The adoption failures are mostly networking issues specific to their cloud. Performance and box limits vary widely depending on cloud vendor and I still don't quite understand the performance penalty of the different overlay networks / adapters.
Consider a traditional monolithic application. In comes your HTTP request in one end, a bunch of cross thread communication happens, and database queries come out the other end. With that, you have 2 points of network communication.
Now with a micro-service, you might have 4 or 5 applications that are needed to replace the above monolith. Throw in a service mesh on top of your cloud providers SDN, you've turned 2 points of network communication into 20 or more. The 5 micro-services talking to each other and the service meshes talking to each other. Add on top the additional processing overhead of maybe 1 to 2ms, you've just added at best 10ms round trip time to get to your databases and some more CPU. And to what benefit? TLS? You can do this in your application, or trust your private network is private. Tracing? You can do this with PID matching and watching the kernel's networking stack.
For what I do, in theory, many things should not impact results. In practice, anything that upon measurement impact results is stripped away. Think A/B testing but for every single component - including the major version of say the python interpreter.
That's how you end up running many things baremetal.
I'll say the future is not serverless but cloudless
For most companies and their operating environments cloud gives flexibility in capacity planning, freedom to experiment wildly, access to practically unlimited storage[ß], plus reliable perimeter load balancing. And of course, for most companies engineering costs far more than compute. Paying the premium for cloud environment makes business sense if it means your development teams don't have to spend time waiting for available resources.
Funnily enough, even a number of high-end trading firms (including HFT shops) are moving to cloud. Number crunching for models, backtesting, CI, analysis and simulation pipelines, ... All of those require resources and rarely need to operate in tandem with the real-time trading systems. The same flexibility, ease of capacity expansion and freedom to bring transient resources online as needed saves on expensive development time.
If you're big enough, or need to operate fast enough then cloudless and baremetal are going to bring better ROI. Most companies aren't at these extreme ends, though.
ß: within limits; if you need to produce and store petabytes of changing data on a daily basis...
Sure, that's a low number but if you already have 150ms of processing, adding another 10ms might cause issues.
Also, if you're disabling the resource controls on kubernetes- you're kinda defeating the whole point.
Some of the old hands always keep a colo + baremetal in the back pocket for always on and base testing; only pushing to the cloud after due diligence in comparative testing. That's a more realistic approach than 'cloudless' for scale.
Defense in depth exists for a reason.
For most people, in most practical scenarios, you have to hang your hat on something.
Yes, take basic precautions, but if you lacked the chops to keep your private network private, then you have little or no chance of preventing the ensuing attacks.
Do you have any pointers/write-ups with more information or plans in this direction? I would be interested to learn more.
https://kubedex.com/google-gke-vs-microsoft-aks-vs-amazon-ek...
It's pretty easy to dig up speed tests of the overlay networks, but a lot of these are just rating userspace overlay networks. The new hotness is the plugins provided by the cloud vendor which integrate with their SDN, and I haven't seen a good benchmark for those yet.
Most interesting reading will be to look up managed kube networking plugins on github and look for open/closed issues with lots of stars.