Not quite accurate, read the article:

"By 2009, the median net worth for white households had fallen 24 percent to $97,860; the median black net worth had fallen 83 percent to $2,170"

That's MEDIAN, not AVERAGE. Average is even worse, as you say drawn up by the few very wealthy. But with median, the wealth disparity is already astounding. It's a racial issue. We can debate the reasons, but the fact remains. Middle class and even lower class white households still have much more wealth on average than their black counterparts in the same income group.