I'm hoping that this article acts as a catalyst for the Dutch government, and other EU governments, to move everything away from American clouds.
I'm hoping that this article acts as a catalyst for the Dutch government, and other EU governments, to move everything away from American clouds.
Is the idea that they're more ready to listen and take action because of recent executive changes in the US, even though the cost of doing so has gone up by 100-1000x and the possibility of a joint retaliation from US tech giants and the government working in concert is now much higher?
I hope you're right, but one of the rough dislocations of the present moment is the disconnect between how europeans conceive of their sovereignty and the reality of their economic, military, and cultural fragility in their relationship with the US and US companies.
No amount of grandstanding rhetoric and appeals to "courage" changes that if there are any serious economic consequences (caused by US/corporate coercion or otherwise), the government would likely fall and be replaced by someone more amenable to the status quo. What feels like a small price to pay for someone focused on security long-term may be an unacceptable price for someone focused on short-term outcomes in their political fortunes.
If those very basic precautions had been taken, migrating to a Euro cloud, or a private environment (open cloud stack) would be trivial.
If not, a lot of people should be fired...but granted, there are a lot of stupid people out there...
All that said, I'd say the concerns around this are vastly overblown.
There’s a million little proprietary APIs and the temptation to glue one to another, especially circumstances like AWS where they use lambdas for basic functionality that should have been just provided by the cloud provider itself.
I believe so, yes. I don't think Americans realize how profoundly the last few weeks have affected European political thought. It'll take a while before you see concrete changes. Europe is like a mammoth tanker, slow to change direction, but practically unstoppable. I believe that it's more likely now than ever before for European governments and businesses to sever their dependency on American technology. Lots of comments in this thread explain how hard this is, how big the feature gap between, say, AWS and OVH is, but as a European entrepreneur I gotta say, this looks a lot more like an opportunity than a problem to me.
- Compute (vm, k8s, containers, faas)
- Storage (disks, file shares, S3)
- DBs (relational, document)
- User management and access control
- SDN
- Configuration management
- Secrets management
- Key management
- CDN
- DNS
- Domain and cert registration
- Email / SMS
- Messaging broker
- Streaming broker
Preferably all in the same place and at least somewhat integrated with each other. I'm not spelling out logging, auditing, IaC and other supplementary features but rather core functionality.
That seems to me like a minimal set of services a cloud provider must offer so that clients would work on "service assembly" instead of "building from scratch" or "integrating integration-hostile products".
Ie, are you a Europe-wide entrepreneur working to move the whole unstoppable tanker away from american clouds, or do you just have contracts with EU entities in Brussels and perhaps a few EU members like Germany or Denmark?
Do those contracts really help you navigate the digital contracting systems of Italy, Spain, Greece, and Croatia? And is your timetable for growth going to line up with their elections that could result in contract negotiations stalling or even existing agreements being frozen?
Though I've cursed it for years, I'm increasingly glad our org's cloud migration has been so slow that we've only now rolled out the first apps. Pretty much everything we've build can be run anywhere we want, so if it's time to drop the ball and go back to onprem, we've not wasted anything but time on setting up the base
On premises.
Admittedly they have a new CTO who according to our support agent is very focused on improving that, so here's hoping, because otherwise their tech offering is very convenient.
:/
I’m not sure what this looks like outside of the US, but colocation providers offer racks of machines, or to host your machines, while providing access to cheap bandwidth and peering capabilities. It’s absolutely possible to move away from the major cloud providers. However, it will require a degree of investment within your organization to support these deployments no matter which you choose, which could be a new investment compared to using AWS, GCP or Azure.
It's not just kubernetes and openFaaS, what about that thing that's a virtual appliance and requires a VM, now you need KVM. Network and firewalls? Storage as in fully replicated cannot ever lose a byte or have it unavailable storage? Object as well as block. Databases, point in time restores/backups/automated maintenance for postgres and then you've probably got a mssql server for that one app, and mysql for that other app.
It becomes just a fairly massive task back in the real world.
I'm not too familiar with the whole range of AWS offerings, but I really think aside for DBaaS and FaaS OpenStack can cover pretty much everything someone would need, especially combined with Ceph for storage.
All opensource.
It actually takes work to setup and run we are not just installing some packages and then pretending you can scrap aws.
S3 has 99.999999999% durability as standard.
I see your point that it's not technically 100% but, as close as can be reasonably achieved.
I guess you can say the code is still backdoored / untestable but it seems that could be audited.
> People also fool themselves that special keys and “servers in the EU” will get you “a safe space” within the American cloud. It won’t.
The problem isn't sneaky backdoors, the problem is that the King of America can order Google to shut that thing down and Google will have no choice but to comply.
(Googler, opinion my own)
So that seems to be the value add. Of course the software will eventually need updates...
I also don’t think anyone can count on extra-judicial demands from the current executive branch.
Am I wrong or misunderstanding something?
I feel like people like to fear monger, but cloud businesses won't bend to US requests for anything that resides outside of the US.
I guess the King of America could still shut down the ability to provide support updates.
They are about to go live in a few months.
This is a good option IMHO, and we're about to migrate some of our workload (currently 100% on AWS) on it.
We use EKS, RDS on standard PG, SSM and S3. S3 is a standard now, SSM can be replaced by something else fairly easily, EKS and RDS are just managed open-source software. So it's mostly an added burden on the devops side.
Don't build them. Vendor lock-in is a real problem: even if there are no political issues, it's a business risk because they can charge you whatever they want.
Also, the cost of migrating off these things is usually overestimated. It's an HTTP request, for crying out loud.
But my concern is for those that have built something as Azure/AWS only, who are now stuck with the bed they've made. Sure, there are lessons to be learned here, but if the volume of these is too high, then there will be pushback on any meaningful change since it will be too expensive
Replace (most, not all) crusty apps with a web version - sounds good to me. Put it in the cloud - that's optional.
For hosting their government's own specific computing needs, and assuming a respectable GDP, they can build their own datacenters (pretty trivial) and hire contractors to build cloud computing environments (more challenging).
Open source cloud isn't too hard. There's OSS for about 80% of software needed for a cloud computing service provider, and you fill in the rest with proprietary and custom stuff. There's already several providers (one in the US, several in the EU/other countries) that offer "public cloud" using OpenStack. They literally give you, the customer, your own OpenStack cluster, and bill you for what you use. It's insanely easy and powerful. Yet everybody still uses the more popular providers (DO, Hetzner, Scaleway, etc), despite the fact that they all have proprietary interfaces, without anything close to feature parity with OpenStack. I guess people really like vendor lock-in and lack of features.
The hardware is more challenging to source; the chips all come from Taiwan or China, and the US and China make most of the good hardware.
For private business in their country, they might offer grants and tax incentives to EU companies to build out more local cloud hosting services. But since it's the EU I'm sure it's massively more complicated than that.
All of the talented people left around 2013 if memory serves me right.
- The one I have experience using is Genesis Hosting out of Chicago. Their website looks like it's from 1997, because it is from 1997... But they provide a nice OpenStack solution that works well.
- I haven't used Vexxhost, they seem to provide something OpenStack-related, but their website is all marketing bullshit, so I have no idea what you actually get.
- RamNode seems to provide access to the OpenStack API.
In Europe:
- OVH Public Cloud is still short on details, but based on some verbiage buried in the marketing BS, it looks like you do get an OpenStack interface.
- Open Telekom Cloud by T-Mobile seems to give you an OpenStack interface.
- Acville Cloud is based in Romania.
- Cyso Cloud (formerly Fuga Cloud) is based in the Netherlands.
- IntoVPS seems to provide its services on OpenStack, but no idea if the API is open. They build a custom OpenStack console called Fleio.
There's a lot more listed here: https://www.openstack.org/marketplace/public-clouds/
- Outside of the Telco sector, OpenStack is basically dead.
- Even within Telco, everyone sees the writing on the wall of OS being dead and is looking to make the jump.
- OpenStack is a cluster of poorly-interoperating, poorly-documented products -- The customer experience is fucking terrible.
- DO, Hetzner, etc all offer a superior product.
- None of those products even come close to touching the features of the big three clouds or even Oracle.
If your needs could be well-served by DO, Hetzner, etc., then your needs could be well served by racks in a colo.
Past that scale, American cloud providers are really your only option if you want that level of automation.
(Or Chinese cloud providers, but largely assuming that's a non-option)
I know OpenStack is a tire fire to maintain, I've worked with it for large-scale on-prem data solutions. But if a company wants to kill themselves to maintain it for me, I'm happy to pay for the privilege.
But that's also the point of my other comment in the thread -- a French company builds basically all of the physical infrastructure that datacenters run on. This attitude can be applied both ways.
I assume you were unfortunately a victim of Mirantis/Fuel/Puppet/Mcollective... or one of the 'converged' solutions.
While I wouldn't call OpenStack "fun" Especially in the Essex to Icehouse era, where vendors seriously impacted the code stability...It is just a well documented collection of separate components that interact using REST api's and RPC like calls over a message bus.
Nvidia, Cern, JPL, and lots of smaller companies that need private clouds and have the expertise are still running OpenStack.
For me the main value is the ability to have portability between public and private.
If you just use the ansible playbooks included in every OS repo, it is pretty easy to roll your own deployments that are quite easy to maintain if and only if your company is mature enough to follow that model and isn't subject to the soicotechnical issues that plague containers too.
While the workflow changes, the hard parts of OS and k8s, including networking, monitoring, etc,.. are exactly the same.
As a random example of what always screws this up let me point at kubespray, which is not unique at all.
Note the: > Remove docker requirements https://github.com/kubernetes-sigs/kubespray/issues/6400
That is because, like many projects, they didn't respect the natural boundaries of the node components, and they are now paying the price for that debt.
k8s and OS from an infrastructure point of view are equal in complexity. It isn't instantiating a container with CRI foo, or libvirt command bar that is the hard part.
It is the distributed computing, virtual networking , resource allocation, federation, API's etc... that is hard.
Note, if you think that the "OS is dead" for all needs, especially in the telco space, you may want to dig into what containers actually are. They are just namespaces running on an OS, and it will still be horses for corses as to what is appropriate.
Especially if you are using the easy ways of instantiating hardware for k8s, almost all of them are highly insecure by default and you are going to have to dig into the same style of systems with similar components or you will have a leak of data at some point.
I wish there was something better than OS, but if you use a dev mindset and not a glass house IT mindset it is a very useful tool that may be the least worst option for you for some needs.
It's about the endless bugs and regressions and laundry list of stupid problems caused by inadequate processes by OpenStack developers.
For example, let's say you're running Cinder v3. Cinder 3.59. You want to get the volumes that you have attached to an instance, so you curl the API:
/cinder/v3/<instance id>/attachments. You get a 404.
You get a 404 because you didn't pass this header: "OpenStack-API-Version: volume 3.27". Because Cinder defaults to Cinder 3.01 behavior even when you're running 3.59. Attachments were only added in 3.27. So even though you're trying to curl a route that wouldn't even exist in 3.01 and you're running a version clearly later than 3.27, the API responds as if it's Cinder 3.01 unless you specifically tell it to do otherwise.
And this is just one of the laundry list of stupid situations that I can remember off the top of my head.
When the thing isn't otherwise failing all the time.
It is common for message based systems for the target system to own the contract, and they have both the / and /v3/ endpoints that you can grab the version information from.
This is documented BTW:
https://docs.openstack.org/api-ref/block-storage/v3/index.ht...
While I personally prefer the URL method, when versioning through custom headers, if you bump the API without that custom header, you will break way more than returning correct behavior for the minimum supported version, enforcing backward compatibility for API's is generally considered a best practice.
Note:
> If the OpenStack-API-Version header is not provided, act as if the minimum supported version was specified.
https://specs.openstack.org/openstack/api-wg/guidelines/micr...
Once again, fully documented, expected behavior.
Coming from IT land, the answer is simple: you don't use them in the first place, and you grit-and-bear the replacement cost if and when the time comes. This is a negative on my research notes, slide decks, and papers when it comes to evaluating various cloud platforms for our workloads, and yet it's also the number one reason we're forced into a specific provider (some leader loves their proprietary tooling, and forces us to use it).
Look, I'm not saying these proprietary tools are bad, per se, just that they have a steeper cost than initially presented to the consumer in terms of architecture complexity and inevitable migration. The very first question you should be asking before consuming niche or proprietary products from vendors is, "Can I do this in a standard way that's more portable?" For stuff like Azure Functions, the answer is emphatically yes - but it comes at the cost of managing additional infrastructure, which is often the main reason companies want to use those tools in the first place (a misguided notion about throwing out infrastructure to save money).
As for the solved problem of compute (VMs and Containers), well, literally any cloud provider should have that ready to go. The question is whether or not your org is willing to retain the talent needed to build and support your clouds internally, or if they'd rather pay higher outsourcing costs with vendor lock-in instead.
The networking stack in Azure or AWS are so different that they require a different mindset to work, especially securely. If your networking needs are simple you are very lucky.
Often there are proprietary solution to proprietary problems you would otherwise not have in the first place.
I used AWS for a long time but I am back to hosting myself. What arcane network requirement would that entail? I don't think there are benefits even for government scale problems.
The private links in Azure are particularly specific.
After you translate the vocabulary, the process is pretty similar until you get to security items, like ACLs or packet-inspection firewalls. You're still setting up VLANs in the form of subnets, routers in the form of transit gateways, sites in the form of VPCs, inter-site connectivity through peering connections, you get the idea.
If there's one thing I've learned in my IT career, it's that most "new" ideas are just rebrands of existing concepts, and that the real expertise comes from being able to translate marketing-speak into concrete, interchangeable fundamentals. Public Cloud is, largely but not entirely, no different in this regard.
Each time the discussion on moving to a US based provider was a big consideration, particularly the use of managed services that involve data was a hot topic. Part of the risk assessment was considering what the consequences might be if the US government became a bad actor. It was seen as high impact but extremely low probability. Starting to look like we got that part of the assessment wrong.
I think it will take time for the impetus to move to US clouds providers to slow and reverse but I'm not sure I'd be surprised if it does happen now.
Was that before or after 2016?
by the course of looking for programming job, i have scanned hundreds of job-ads, incl. governmental. everybody-and-his-dog requires AWS/Azure/GCP knowledge as if it matters thaaaat much. These cloud-y things have become a mandatory buzzword, and i am not talking about sysadmin/devops.
In my last gig the system was kept cloud-agnostic, so moving between providers or on-prem be possible at any time. And i as CTO kept that good thing, although had to resist some pushes. But seems such cases are few - most places now dream of hyper mega-giga-scale and Lambdas and Big-queries.. while doodling few thousands of requests.
Lets see if there's any wind change.. vendor-lock is a real thing, with much deeper (architectural or life-cycle) consequences than usually perceived.
Someone knowledgeable should have seen this before, this is a core issue when setting up a strategy for digital systems. And this isn't an issue between "purists" and the rest, that is a false dichotomy. The decision was simply to outsource infrastructure to systems you have significantly less control over.
Might work for 15+ years or it might not. I doubt anything will be done now, investments are probably too high. But it is an issue with lacking foresight.
Between countries and the main task for intelligence agencies is industrial espionage. The Dutch government, like many others, decided that exposing themselves is no issue.
I disagree that it has become a problem only now, this is due to his narrow view on politics and a bit naive in my opinion.
I'd rather have my data end up with Google/Amazon/CIA than it ending up everywhere on the internet due to poorly configured DIY servers (and at twice the cost probably).
It is Russian gas all over again.
Europe wants to improve its economy by growing their consumer tech industry. Some of these products like Google Analytics (the example he is upset about) are really hard to replicate (writing to a database on every visit to your website is an expensive thing to do, significantly more expensive than hosting the website!). So they've been slowly increasing the tariffs (disguised as privacy regulations) on US tech firms. It's gone poorly, even EU governments (let alone EU businesses) still use products like Google Analytics, and US tech firms have been able to engineer their way around the regulations, again doing a better job than EU governments who have been busted countless times for breaking GDPR with their own systems.
No one cares about any "data sharing agreement" or a "Privacy and Civil Liberties Oversight Board" no one has ever heard of that has never done anything. Its a tariff with various ways to pick winners and losers.
The only thing thats changed is there is a higher chance these privacy regulations will be recognized as tariffs by the US.
But EU citizens genuinely care about privacy, in part because of decades of totalitarian and near-totalitarian regimes.
There is another risk underpinning this, I'm not familiar with this so it's mostly hearsay on my part, but foreign firms in the US routinely get completely screwed in US courts, and fear the seizing of their data in discovery processes or other ways. The data sharing agreement was made to provide some degree of clarity or assurances in this regard.
I've met managers who are convinced that if they're not careful, their IP and business data will get stolen by their US competitors through various legal or less-legal means. EU executives have been detained for days at the border on suspicions of terrorism to coerce them into selling US assets. I can't judge if this is paranoia, and maybe those companies could make use of better protection against Chinese hackers but there's certainly some truth to that.
The EU's biggest exports to the US are cars & pharma. I guess the VW diesel situation could be seen through that lens, or the GLP1 compounding rules.
You're going to need a citation on that sort of thing, because I'd expect it to be a much bigger deal.
> To all the people saying that this is nothing new
An EU state wants to boycott or regulate American business? I don’t know how that’s new.