He might also think that calling any particular person a racist would result in a media firestorm. (After all, he was pilloried just for rallying his troops after an electoral loss.) Maybe he didn't even really intend to imply that anyone was a racist; he may just have wanted to clarify for these individuals.