https://www.nspe.org/resources/licensure/what-pe
Every single design, bridge, etc is certified by a Professional Engineer. Your ass is on the line. I don't mean just being fired from your job - but jail time.
Per the website:
"A century ago, anyone could work as an engineer without proof of competency. In order to protect the public health, safety, and welfare, the first engineering licensure law was enacted in 1907 in Wyoming. Now every state regulates the practice of engineering to ensure public safety by granting only Professional Engineers (PEs) the authority to sign and seal engineering plans and offer their services to the public. "
I suspect, and hope, software used critically have the same. Yes it's more bureaucracy and overhead...but being in software, I know any joe schmoe can walk on and develop code without oversight.
I'm not saying if you're making the next twitter, that you need a PE. Or that every developer needs to have a PE license - but like other industries there is one PE per X amount of non-PE engineers.
Due to the Threac-25 incident (https://en.wikipedia.org/wiki/Therac-25) there are standards for medical software (https://en.wikipedia.org/wiki/IEC_62304).
I'm not in medical so I don't know how widely adopted, or how strict it is adhered to. But it might be IEC 62304 needs to be branched out.