How do you know this?
Just taking a wild guess, but I'd think a company with billions of funding, and a ton of people trying to find flaws in what they are producing, would have some processes in place to incentivize quality as well as quality.
What you are suggesting is that a company that produces a product that is based on balancing trillions of floating point numbers makes core business decisions in the most simplistic black and white terms. "Hey, lets just go with a one and a zero on this." Bizarre assumption.
Maybe I'm just good at prompting, and I'm not trying to trick it, but I don't see this "superficially convincing bullshit." Can you show me a chat where you have sincerely prompted it and gotten something that matches that description?
I often see responses that are better than I could have given even if given hours to research and compose them, and I'm a pretty good writer and researcher.
Here, since I'm asking you to share one where it fails as you say it does by creating "superficially convincing bullshit", I'll share several where it succeeds.
https://chat.openai.com/share/523d0fec-34d3-40c4-b5a1-81c77f...
https://chat.openai.com/share/e09c4491-fd66-4519-92d6-d34645...
https://chat.openai.com/share/233a5ae2-c726-4ddc-8452-20248e...
https://chat.openai.com/share/53e6bda1-fe97-41ce-8f5c-89d639...
https://chat.openai.com/share/19f80ea9-e6be-4ac3-9dd4-7ea15c...