I think that's a bit simplified. In pharmaceuticals, the regulation is absolutely vital for the innovation loop, because it sets an objective standard for demonstrating progress. If you cut out all pharmaceutical regulation, you would get limited, biased trials 'proving' that a treatment worked, and then an even more massive marketing department behind it. There's a multibillion dollar industry in skincare, but are skin treatments any better now than they were 50 years ago? Nobody really knows, because large scale, long-term trials are expensive, and there is no regulatory requirement to rigorously demonstrate how well your product works.
Similarly, in chemicals regulation, an unregulated industry works towards the cheapest, most effective solution for their problem, but ignores the externalities. If the pesticide increases your yield, why would you care if it lowers the insect life which pollenates the crops for the neighbouring farms? The neighbouring farms aren't paying your bills. If your furniture treatment is cheaper than the alternative, who cares if it causes marginal damage to the health of the people using it over 20 years? In any case, it will be very difficult for anyone to tie any damage back to your product, so the reputation risk is limited. Again, the regulation is required to set a standard of progress - you don't just want innovation in cost and efficacy for the manufacturer, you want innovation relative to an objective standard which takes into account other impacts.
I do agree, though, that reducing cost for testing is also very important, to reduce the barrier to entry. Making testing too expensive also does damage. The EU actually had a clinical trials directive which proved to be too onerous, made the trials too expensive, and some measures had to be repealed. So it is a balancing act, but to me it's clear that both sides are important.