http://www.cdc.gov/hiv/topics/surveillance/basic.htm#inciden...
If you follow the links, one of the large slices you don't account for is "high-risk heterosexual contact", which makes up about 1/3 of new HIV infections. That's people who are not part of one of the high-risk groups, but have had at least one partner who was (which may or may not have been known to them). So the proportion of people who have some rational reason to worry is relatively high, unless you are absolutely sure that none of your sexual partners have themselves ever belonged to one of the high-risk groups. That's a particularly common route of HIV infection among women, since not all bisexual men self-identify as bisexual or disclose that part of their sexual history to female partners.
But the threat that AIDS represents is much larger. Why spend a lot of resources fixing a tiny hole on a large boat? The hole isn't leaking much? Well, it's the scary potential that mandates treatment and prevention.
If we assume that HIV rates among blacks are distributed the same way as in the general population, then we find that black gays alone account for about 40% of HIV infections in 2008, while making up 0.1-0.3% of the population. Unfortunately, the data I linked to is not granular enough to prove this.
A comparison: gay sex was about 55-170x more dangerous than straight sex in 2008 (depending on whether 3% or 1%of america is gay). Unprotected sex is only about 6-20x more dangerous than sex with a condom (depending on which set of wikipedia numbers you pick).