You're right that collecting these fragments is useless for the purpose of street view or Google maps. But it's also completely useless as a method of spying on people's personal affairs.
You can't build personal profiles based on that data, but that is what Google does for a living. They have an incredibly dense set of personal information from gmail, adsense, doubleclick, search, etc. What's the additional information they could possibly gain from arbitrary street view payload fragments?
I'm usually very concerned about the impact of Google's information collection on my privacy, but in this case I just don't see how it fits in with any kind of goal they might have.
My diagnosis would be one of institutionalized compulsive hoarding. Maybe Googlers just don't have it in their genes to let go of any information they got, regardless of how useless it may be. Or maybe we need a new term: BBDR - Blind Big Data Rage :-)