Kubernetes is only the software layer, a lot of the cost is in the hardware/infrastructure, and in the salary for those experts.
For instance, the cost for a pair of redundant symmetric gigabit fiber is in the thousands a month and may require tens of thousands of construction costs. These quickly add up, and the upfront costs can quickly reach six figures.
For that matter, have you checked the price, in qty 1, of a server that will absolutely destroy anything reasonable from a major cloud vendor in terms of IOPS to stick behind that switch or router? Even if you believe the numbers on the website of a major server vendor and forget to ask for a discount, it’s still quite reasonable in comparison to a major cloud.
You can always save by going on-prem, assuming you have no uptime requirements. But the moment you sign an SLA, those savings go out the window.
Then if you are going to have remember that cloud networking is pretty beefy and if you want k3s to do distributed storage you will need some pretty beefy network hardware.
There are a lot of things hidden in the cloud costs that people forget about.
The one thing running your own stuff does allow you to do is make choices and trade offs. If this switch goes down and we have 6 hours of downtime to replace it what is that worth etc.