For a certain definition of "software": when only doing one training run costs an 8 digits sum (requiring hardware one order of magnitude more expensive than that to run) I kinda dispute the "all they do is software".
It's definitely not "all software": a big part of their advantage compared to actually free and open models is the insane hardware they have access to.
The free and open LLMs are doing very well compared to OpenAI once you take into account that the cost to train them is 1/100th to 1/1000th what it costs to train the OpenAI models.
This can be seen with StableDiffusion: once the money is poured in training the models and then the model made free, suddenly the edge of proprietary solutions is tiny (if it even exists at all).
I'd like to see the actually open and free models trained on the hardware used to train OpenAI: then we'd see how much of a "software edge" OpenAI has.
And my guess is it'd be way less impressive than you make it out to be.