Rush can pine away all he wants about how he's not a racist but it doesn't matter; a lot of people simply don't believe him, and they have good reason to feel that way.
Are they right? Doesn't matter. Rush has made a career out of making controversial statements and if you live by that sword, sometimes you have to die by it.
It's not about conservatism; as was said elsewhere, if John McCain or Mike Huckabee wanted to buy in on an NFL franchise ownership, no one would bat an eye; Rush has to know this, but on the air (or in print) he's not about to make that concession.
He can't; it wouldn't fit the audience expectation of him as a defender of conservative values and principles.
The irony, of course, is that conservatives like to say they're champions of accountability and that liberals constantly make excuses, constantly place blame on everything and everyone but themselves and their own liberal values and principles.
Yet, here's Rush blaming everyone but himself for his own setback.