Booger McFarland was not bothered by speculation about his job