If the answer is collusion, then that only raises more questions on what else they could all be working to sweep under the rug.
But really the bigger question is: surely if this was somehow fixable by a software ECU update, they would have done so? Defeat devices are an old game in the car industry. But as long as those recalls consist of merely software updates, it's clear they are just preparing the next, even bigger con while apologizing for the current one, and EU governments are happy to look the other way.
If I were driving, I'd look for a car that, based on my own testing (e.g., during a test drive at the dealer), measures sane emissions and drinks AdBlue. But for all I know, such a car does not exist. I really wish they make them take all the dirty cars from the streets, mandating a retrofit AdBlue system to allow them back onto the street. With an allowance for those that have an AdBlue system already to measure at a test drive, i.e., with a probe shoved up the tailpipe, and the mechanic/owner driving a loop around the block or so, maybe with some mandatory measuring for the first 100km after installation or so, and if the car at any point during this fails the standards, don't allow it back on the road. The effect from this would be that teaching the car to cheat would be next to impossible, especially if you could mandate another test e.g. at any point between 10000 and 15000km, as best fits the owner schedule. While it should be possible to engineer a cheat for this, it would be hard to conceal as an accidental bug, which would be what they'd have to make it look like to avoid potential criminal liability (they could certainly create suitable criminal statutes along with the testing requirements, and make sure these hit those that actually do it, along with everyone up the chain. Make it conspiracy or so, and catch all at once. Some prosecutor would have a lot of fun.)
This has been an open secret for years. Anyone who wanted to know, easily could.
Perhaps not the existence of "defeat devices", but open disdain for and cavalier circumvention of the rules by all manufacturers.
> Did no competitor try to legitimately meet emissions requirements
Probably. German manufacturers openly share products for comparison with their competitors (realising they can't avoid it anyway). I think it's a safe bet that everyone knows what everyone else is doing.
It's beyond belief that Toyota's research division hasn't both tried to build clean-and-efficient small diesels, and taken apart the european ones. They just didn't put them into production.
For politicians, this was a very convenient non-tarif barrier. Since Toyota wasn't making such engines, you can help VW without attracting WTO fines. And you look good on TV talking about euro technology. And, of course, striking a strong pose about CO2 also looks great on TV.
They are (apparently) entering a (high emission) engine protection mode at a time that is not allowed.
It isn't impossible that their excuse, they read the requirements differently, really was in good faith.