The biggest reason decaf brews, tastes, and grinds so different is the processing essentially causes the beans to expands (so you can extract the caffeine out of the center of the very dense, green coffee) and then you need to dehydrate them back to the proper moisture content for roasting. You take a green coffee that is previously extremely dense and non-porous, and make it much much more porous. This leads to roasting difficulities and brittleness when grinding which seems to lead to fines.

I'd agree, less caffeine in the bean without decaffinating would lead to a better tasting coffee (if you want the lesser caffeine).