For example, if we had governments with strong tech departments that could fund helping the development of LibreOffice, then suddenly even if someone wants to use MS Office, that's still a bargaining chip to get a better deal because there's a viable alternative. Or to develop something like OpenProject, Kanboard etc., alternatives to the likes of Jira, that might be enough for many out there, while also possibly benefitting from community contributions. People love to complain about how Jira supposedly sucks, so that'd be a good opportunity to step up and make something "better". Or using open source technologies like PostgreSQL or MariaDB/MySQL for developing their own internal systems instead of always forking over a bunch of cash for Oracle or MS SQL by default.
If you want a government that's cost efficient, then invest in making it be so, treat the software landscape as an investment opportunity - spend some money now to save a bunch of money later. The same way how an app can be a home cooked meal, some software could be a public utility.