DigitalOcean Region Information
snapshooter.com
snapshooter.com
One of the best perks of working at Tumblr was once a month they had a data center tour for employees, you got to go to NJ for a day to see the facility, all of the power equipment, cooling, network equipment, and the servers themselves. Really awesome and eye opening. Pictures don’t really convey the size of these places.
Seems the URL is down however https://wikileaks.org/amazon-atlas/
For example, here's an AWS facility in us-east-1. Pairs of narrow white buildings, ACs on top, generators in parking lot, switching station next door, nondescript name: looks like an AWS data center to me. (Thanks to the WikiLeaks doc I happen to know for sure this really is AWS, but you can tell just from the satellite.)
https://www.google.com/maps/@39.0271558,-77.4541392,698m/dat...
If you have the WikiLeaks document you can just google the "IAD50" type codes and generally find them, but that's no fun.
Check out Amazon Atlas on wikileaks for some more information
I don’t really see operating your own data center as a GOOD thing. These professional data center providers serve 911 and financial clients, while gcp/aws serve selfies and cat pictures. The later considers their consumers cattle for the slaughter, while these professional colo providers don’t have that conflict of interest.
It's kinda silly to suggest nothing important runs on aws/gcp, there's plenty of huge businesses running on them, with equally large legal departments ready to string amazon/google up by the balls if they breach the terms of their contracts around SLAs etc.
But SLAs are a joke - your business should be making $100 for every $1 you spend on compute, the fact they'll refund your $1 is trivial compared to your $100 loss.
I think the reality is that if stores are closed or websites are down, people will just come back later. Maybe not so much for ad serving (if you're bored at work and Reddit is down, you're not going to make time after work to go view ads there), but for actual businesses, it might not be as big a problem as people would like to imagine. But if it is, that's why you can spend millions on a multi-cloud solution.
When I have a service with AWS, I have software I need to relocate.
The needs and constraints of relocating software from 1000 servers is very very different to the needs and constraints of relocating 25 racks of equipment.
Unless you have 1000 AWS ‘pets’ instead of ‘cattle’, in which case you’ll wish you could just lift-and-shift.
You’re ignorant - countless major serious businesses, government organisations, and NGOs are running on AWS.
https://aws.amazon.com/blogs/publicsector/tag/nasa/
First hand experience here.
That's an absurd stance and you should know better. It's common knowledge that hugely important companies run on GCP and AWS.
There may be something else to it... they are launching a Spain region [1] this year with three data centers. My guess is that it will be both heavily subsidized and much cheaper to operate than a similar project in France.
[1] https://aws.amazon.com/blogs/publicsector/coming-soon-aws-la...
Haven't been in this space in a long time, but when I did work on such projects in Switzerland mid 2000s having anything outside the country would at least raise eyebrows. Back then we had to run most things on-premise, but from what I have heard, they opened up to the cloud to a degree.
See also the tags of the article "[...] Education, Government, Healthcare, Nonprofit, Public Sector, [...]"
A smaller provider usually sees a financial advantage to lease or build traditional datacenter space. “Smaller” can be quite large… I worked with an enterprise team that built out a large private data center circa 2015. Something on the order of low six figure server count. At that time the dollars worked out to build vs. cloud, mostly due to security control requirements. YMMV.
Tidbits from Peter Desantis’ keynotes and before that James Hamilton at re:Invent indicate that like Meta, Amazon doesn’t use 19” form-factor racks, has in-rack UPS solutions, DC power delivery and a sled-based server architecture. All these videos are on YouTube and the last 3-4 years (2018 onward) are really interesting and worth watching.
I’m short I’d guess Amazon and others partnerships with Equinix (and equivalents) are about two things: time to market, and some markets not being worth the investment [1] to drop 3x 30MW facilities into.
[1] https://www.datacenterdynamics.com/en/marketwatch/cutting-da...
There are various degrees of "operate your own datacenter" in the datacenter market.
A lot of facilities on this list, like Equinix, are retail datacenters. They have things like 24/7 "smart hands" who you can call to go repair a server, replace or move a networking cable, things like that. These are the most full service options. They also offer various network connectivity products, and if you are willing to pay for it, you could go as far as ship them servers or a rack and have them do much of the install for you. These are the most expensive options, but, at least in the case of Equinix they also typically have the best network connectivity. This is your typical collocation, or even carrier hotel.
So if you go into an Equinix facility in a lot of markets you will see everyone with racks or cages full of equipment in there. Though the biggest players are mostly going to use them for network POPs and maybe something like a CDN node. So you might see Amazon CloudFront equipment in there, but not much else.
Often the thing to do is put your public internet connectivity, peering, and perhaps backbone links in a well connected place like this, but the bulk of your compute is in a much larger and cheaper facility elsewhere.
There are also retail collocation facilities that aren't super well connected (when I say well connected I mean you can get access to 10s or 100s of providers just by being in that facility) and those are a lot cheaper than Equinix, and often full service. They tend to cater to smaller customers. Though I've seen Akamai have a ton of racks in some surprisingly crappy faculties.
Then DRT (Digital Reality Trust) operates more in the wholesale datacenter market. They can rent cages for multiple racks, but are even more used to renting out whole datahalls, so you have dedicated space. In a lot of areas they can't really offer things like 24/7 smart hands. They aren't going to handhold you nearly as much, and space and power tends to be a lot cheaper, but your minimum buy is going to be a lot larger. 20-100 racks maybe?
CoreSite kind of straddles the line between Retail and Wholesale. They offer a lot of full service stuff, but can also rent you out a whole data hall or even building at a campus if you need it. No one would ever confirm this, but I'm pretty sure one of the big 3 cloud providers has a big chunk of, the CoreSite Silicon Valley campus. If you search Google Maps for core site silicon valley you'll see a cluster of about 5 buildings, all labeled. One building right there is suspiciously unlabeled, but if you do the the Satellite view, it's clearly a datacenter.
Take it another step, and you have companies like Sabey who offer a datacenter shell. Basically a building with power, generators, etc, but you do everything else, or they customize it to your specs and help operate it. Yet again cheaper per unit of compute you are doing, but a lot high minimum spend to make sense. You'll probably be measuring things in MW (megawatts) before you are looking at something like this. Sabey also sells to retail datacenters. I spent far too much time at the Sabey Tukilla campus south of Seattle from 2009 - 2011 in an Internap facility. I had to go through Sabey security to get into the building, and then Internap security to get into their space. Most everything inside that space was taken care of by Internap (now seemingly branded INAP).
Then you have build and operate your own datacenter, which also has various degrees. I don't know how it works now, but I know a few years back AWS had (has) a TON of buildings in the Ashburn Virginia area that are us-east-1. I was told by someone who worked in datacenter real estate that they typically would buy the land and get a datacenter shell built, then sell it to another operator and lease it back. Then bring in all their own power and compute.
I also heard that a few years ago when Oracle was building out their cloud, they were just buying up all the retail datacenter space they could so they could get to market faster. They were paying an arm and a leg for it, but that was the fastest way to get online.
So the economics of all this depend on what you are doing, why you have the space, and how much actual equipment you need in there. At smaller scales, say a CDN, or just networking equipment, it often makes the most sense to just lease retail space. At larger and larger scales it makes more and more sense to in-house more of it.
I've never worked at one of the large cloud providers, but I'd expect they have some smaller regions that are retail or wholesale datacenters operated by another provider. But the large regions are mostly owned and operated.
Owning real estate simply isn‘t their main business purpose and has many downsides, especially in matters of scalability. The main pro reason for owning them is cost. Owning the buildings makes you less flexible but saves you money in the long-term.
One more point in favor of leasing is: the relevant „neighbors“ are just next door. If you own the real estate, it‘s probably in the middle of nowhere, network-wise, and you end up spending lots on laying fibre. Whereas with leased space, other providers are likely on the same campus and all you need is a few 100 meter or one km of fibre to reach the next building or even just the next room. If you are small enough, these savings on connectivity can even end up making it cheaper to lease.
Might be a surprise to many, but it‘s primarily the budget providers (Hetzner and OVH) that own their own buildings.
Most others just have long-term leases with whoever owns the real estate (usually Equinix or Interxion/Digital Realities, as they more or less own most big DCs around the globe).
Also, I guess many miss-understand what „not own the DC“ really means. These services aren‘t the kind of retail-style colocation offers. Those providers rent entire rooms or even buildings there, run their own infrastructure in them and frequently have their own staff and offices on site too. These lease agreements behave more like a usual office or warehouse lease, not what some here might think.
There are a lot of very expensive facilities on this list. Ones that I would not want to put much equipment in if I was at the scale I presume Digital Ocean is at.
But maybe it's a factor of organic growth and starting each region small, then having to grow into it. You can often get space for like 1/3rd the cost of Equinix but with a lot less services and maybe less quality. Equinix makes a very nice datacenter but you sure do pay for it.
If I were building a CDN or an edge compute product like Fly.io or Render, I'd certainly have a facility list that looked like this, because I'm charging a bit of a premium for excellent connectivity to population centers. Though I'd do some kind of managed hosting first until I had the scale to even get into collocation.
But for a bulk cloud provider I'd try to only use these facilities for things like network POPs, DNS services, CDN, etc. Then lease fiber to another facility that was a lot cheaper for the bulk of my compute. But that does take a certain threshold of scale to get there, and a longish planning horizon to make it happen unless you're growing at insane rates.
Uh, pretty sure the Bangalore data center is not in Mumbai.
If someone tasked me with building a data center, I'd probably start by looking at Google Maps and looking for the nearest big, open field on the outskirts of a city.
2. Office buildings often have some lower-quality space (e.g. areas with no windows) so you don't have to use super-premium office space.
3. Server racks are extremely dense, so they can make a lot of revenue per square foot compared to a lot of land uses. Good luck running a yoga studio that makes more revenue per square foot.
Of course there are disadvantages to city data centres - limited room to expand, and nobody's going to offer you subsidy for "creating jobs".
The other reason is you have employees there who can go fix things when they break. In the early days of Square we had everything in a single datacenter in Mountain View and our office was in San Francisco. There were times when something broke in a bad enough way that someone had to drive to the datacenter. It was a while before we built the scale and redundancy to put a facility on the east cost, or in places where we didn't often go.
But you're right, per square foot it's often quite expensive.
At a certain scale you still buy space in the city for connectivity, but then you buy IP transit, or even dark fiber, to connect to a cheaper and larger facility. That could be in the same region, or it might be out in some farmland with cheap power and good tax incentives.
We've seen lots of datacenters built in Washington and Oregon because the electricity is famously cheap hydroelectric. Like $0.02 - $0.03 per kWh. Last I looked a few (5-7?) years ago, industrial power in the Bay Area was closer to $0.10 per kWh and in Ashburn VA it was $0.08.
I remember going to lots of fashionable dotcom parties in offices there in the late 90’s and early 2000’s. The story at the time was that it was the most redundantly connected building in the city.
Obviously both my two VPS are located in EU (Frankfurt) since I’m Italian and I appreciate the GDPR.
And even if you don't reside in the EU, you can reasonably believe in privacy and abide by the GDPR.
The EU has ruled, now let them enforce it!
"I'm just a guy who shows up, they hand me some cash, and then I go up on stage and play my guitar. Why do I need to read up on accounting?"
Some software developers are in that state of naïveté about the data they collect about their users.
Even if it's just a blog, it may be collecting and storing data about readers in many different ways: ads, tracking pixels, comment logins, etc.
Now, is it realistic that it will be illegal for EU providers to use services that are owned by US companies? Very unlikely that this will be the precedent (yes, I know, there are cases where some courts deemed Google Analytics and Google Fonts illegal).
However, I would personally also feel more comfortable using services from countries where the government can't subpeona all data at will.
Co-location targets the corporate world. Corporate world loves it to just hand off the servers and have it run for them.
But if you go to a company such as Interxion/Digital Realities, they will just send you away (or more specifically, send you to one of their tenants that offers co-location services), because they are primarily real estate companies.
They rent out empty rooms designed to be used for servers and even office space on their campuses. These are normal lease agreements and their purpose is that you put your own staff and infrastructure there. And that's what most mid-sized to large cloud providers actually use.
They wouldn't even have the employees to touch servers or network equipment. What they take care of is ensuring that the AC is working, that the shared Diesel aggregates are ready in case of a power outage and that there is security personal patrolling the campus, but they won't enter rooms rented out to customers just like you wouldn't expect your landlord to go into your flat.
(Equinix is a bit special on that point, since they offer both. You can get co-location too through Equinix, but it's probably not what AWS, GCP or Azure is using.)
Colocation facilities are interesting (I've been in a lot of them) and they generally (physically) fall in two buckets:
1) Repurposing of old telecom (and even telegraph!) buildings. CoreSite (a colocation provider) has a few floors (last I looked) in The Telegraph Building in Chicago[0]. If you look at the pictures of the exterior of the building it still says Western Union on the side! There are some interesting aspects of these - most of them are located in downtown areas. If you start looking around in major cities you will start to notice gigantic nondescript buildings with minimal signage, almost no foot traffic, and almost no (if any) windows[1]. If they have any signage at all it will typically be "Something Bell", Western Union, AT&T, etc. These facilities are interesting for a variety of reasons. Because they are repurposed telegraph and telecom buildings going back to the days of mechanical telecom switching[2] and DC battery banks they have floor load ratings that are just incredible. Something like 20x or whatever of a typical high-rise. Downside of these facilities is that on a square footage basis they typically can't support the cooling (and often electrical) requirements of modern compute equipment which is much more dense than the extremely heavy but relatively cool/low power mechanical switches and other equipment from decades ago. If you look at the picture of the Telegraph Building you will notice it's literally right next to the Chicago Stock Exchange. Needless to say these facilities are really popular for HFT stuff.
2) Purpose built facilities. These are typically located near population centers (for access to peering and other fiber) but out of downtown areas. Examples here are CH7 and CH3 from Equinix[3]. These are still close enough to major peering points and fiber distribution so access and latency isn't terrible but being in more "suburban" areas. Here they have more space to build, real estate is cheaper, and interestingly since 9/11 there has been a movement of sorts to move major hosting facilities outside of densely populated areas/potential terrorist targets.
Then for your Google, Facebook, NSA, etc of the world they can build in fairly remote (rural) areas where they have essentially unlimited space and significantly higher security[4]. The kind of thing where law enforcement and private security can setup perimeters where if you get anywhere near the actual hosting facility you better have a reason to be there - as opposed to a downtown facility where thousands of people walk and drive by everyday without ever knowing it's there.
[0] - http://wikimapia.org/30789188/Telegraph-Building-CoreSite-Da...
[1] - https://nypost.com/2018/06/26/nsa-using-att-data-center-in-h...
[2] - https://www.youtube.com/watch?v=xZePwin92cI
[3] - https://www.equinix.com/data-centers/americas-colocation/uni...
As far as Chicago goes from what I remember someone at the company had a bad experience with DRT so we never looked at 600 S Federal but we did look at Equinix at 350 E Cermak (which REALLY stands out) and a handful of others that weren't memorable.
Thanks for the added detail!
And then the building Google owns in Manhattan: https://en.wikipedia.org/wiki/111_Eighth_Avenue
That had a bunch of carrier hotels in it, I'm not sure if Google managed to kick them all out yet, since many had long term leases. But the building has a bunch of truck elevators in it... literally can load a truck up into the building. And a ton of fiber connectivity's which is why the carrier hotels were in there.