Engineers move job; they talk to each other. Every car company had to have people who knew the truth and, if they were being unfairly competed against, could publish this truth.
They were all aware of what VW was doing and it wasn't a surprise, lots of companies did it [0] a lot of times [1]. Usually though the penalties were light. Many times they involved US companies and the strong local lobby pushed for the lighter fines. Like the heavy duty diesel engines case where 7 truck manufacturers were ordered to pay $1bn together for doing the exact same thing. Which in the grand scheme of things was peanuts compared to VW's $4.5bn (plus the tens of $bn of extra costs). So the expectation was always "it's just the cost of doing business, pay it and let bygones be bygones".
[0] https://www.nytimes.com/1998/10/22/us/record-penalty-likely-...
[1] https://en.wikipedia.org/wiki/Diesel_emissions_scandal#Previ...
Mazda did, in fact, admit to cheating in emissions testing.
In other words, yes, pretty much the entire automobile industry cheated on those tests to varying degrees. The current "iteration" was also by far not the only one.
But yes, had the research group not performed their on-road tests, it would have only been a matter of time before an honest car maker would have realised their competitor's sums don't add up and perform or fund their own independent investigation.
(Perhaps this is exactly what happened behind the scenes? It's known that CARB ordered the research due to discrepancies in US v EU numbers—but someone pushed them to look at the numbers closely. They put a lot of work in to perform these tests so it couldn't have been on a whim.)
This is revealed in these quotes from the article:
German and his colleagues at the International Council on Clean Transportation (ICCT) in the US just wanted to tie up the last loose ends in a big report, and thought the research would give them something positive to say about diesel. ... German’s group also forwarded the findings to the US Environmental Protection Agency (EPA) and California’s Air Resources Board (Carb). “We were definitely scared. We wanted EPA and Carb to take over.”
AFAIK the technical solution (Urea injection) works fine, but it's expensive and VW didn't want to include it in all cars.