How to colocate your first server at a datacenter

I like to keep control of my data and manage my own systems, instead of using some black box where I'm at the mercy of a cloud provider.

You're not going to be able to troubleshoot or optimize down to the hardware level in the cloud. If some cloud service doesn't work or is not available, all you can really do is wait and hope they get it working again, while if you manage your own systems you might have a chance of fixing it or at least finding out what happened and work to prevent it from happening again. Some applications just won't work on the cloud or exhibit mysterious bugs, failures, or bizarre behavior.

Security and data ownership is pretty much out of your hands when your servers are in the cloud. You can only hope and pray your cloud provider is doing a good at securing your data and isn't stealing or selling it themselves. You generally have zero visibility in to how security is handled by your cloud provider or whether a security compromise has taken place.

And then there's the issue of vendor lock-in, which becomes more and more likely the more unique cloud features you use.

Of course, for maximum control, you wouldn't rely on a colo either, and just host your servers in your own server room(s).

Mostly agree, although in this case I say mostly because the colo facilities where you can easily rent a single unit of space are far and few between these days. If we were still in an environment where there were tons of hosts like Verio still around, I'd say there's nothing wrong with coloing a single box.

For sure once you reach above the $10k/month level there's a very good chance that it will make sense to colo. You can fit a LOT of hardware in a single full rack, and they're like $750/month plus bandwidth and power costs (so around $1200-1700 all said and done per month).

> redundancy, backup, remote hands, environmental monitoring, hardware maintenance are just not worth figuring out until there is substantial cost savings to realize.

So you don't need to monitor temperature sensors any more with a VM, but most of the above are still costs with cloud - flaky RAM, redundancy, backups, monitoring, etc. There are also the things you previously didn't have to worry about - crappy resource isolation turning your scratch disks into 2kb/sec joys, total ineffectiveness of the CPU cache, managing a now-essential network fabric to tie pieces of your app together where previously it all fit on 2 master/slave machines, etc.

Of course if your application isn't simply some stock PHP/MySQL application, and you want to really "embrace cloud", then the time you saved fighting a subset of hardware problems is replaced by a fixed development cost integrating with someone else's higher tier APIs (S3, Dynamo, etc) you can then never escape even if you wanted to.

I've never seen any realistic numbers comparing the use of traditional hosting facilities, say, providing managed servers, to the new generation VM stuff. Any material I've seen has been sponsored crap involving some multinational.

My own experience is similar to yours - hosting your own hardware is a pain in the ass. However there is middle ground, there are many colos that will happily provide managed hardware, and perf/pence, this still tends to be far cheaper than the equivalent in VMs, and increasingly they're coming with similar APIs to order/replace machines

I actually find colo'ing a new server quite fun and enjoyable... even after doing it a half dozen times... as a techy... something is just so cool about putting a server in the same racks as some monster equipment. ;-P

However, I disagree about colo'ing not being economical. If you setup a physical box in a colo with a hypervisor (kvm, xen), then the density you can get out of 1U is amazing, and makes the entire thing much more affordable.

Take this site for example: https://www.ubiquityhosting.com/cloud

16GB ram, 8 coers for $128 a month. Considering most 2U colo spots I've seen hover around $150 a month, seems like the colo is not a good deal. However, the VPS for $128 a month is a single server. Your $150 a month colo, with xen, you can pack 3-7 vm's in the same space, making the cost-per-(virtual)server much lower.

I thought you could only get an IP delegation if you were multi-homing and could justify a /24. Is that still the case?

Do not put your IPMI controller on a public IP without any sort of access controls in place. These controllers are pretty terrible security wise, and it's not a good move.

Yes a million times, console-over-IP and virtualization are must-haves

"if you co-lo provider sucks, you can move and keep your IP space. (This is vital for email, but I recommend it for everyone)."

Why is this "vital for email"?

I agree that obviously if you can you want your own space.

But if you are planning your move you can handle the new ip space by dns ttl. Which is what we have done since the 90's with moves where we didn't have our own IP block. And yes it is a huge pain to be avoided so I'm not disagreeing just wondering about the "vital for email".

Buying the 4-hour support package for your servers is so worth it. You don't have to pay for remote hands since the vendor will dispatch a tech to replace/upgrade bad gear. If you handle your monitoring correctly, you can get a tech sent out to hot-swap a dead drive before you even realize it's failing.

In terms of picking a good colo, find one that has high security ratings in an area that doesn't have fluctuating power. If they're in Florida, make sure they're Cat 5 Hurricane rated. If they just happen to also have an entire floor dedicated to government hardware, or are on the same power grid as a hospital, and have buried fiber/power lines instead of exposed, even better!

I am surprised how few people know about this.

It isn't just a "good middle ground". It is an amazing middle ground that allows you to benefit from AWS services (simply choose providers close to AWS data centres) whilst getting substantial performance/cost benefits.

I would imagine at minimum you are getting 10x the performance compared to a typical VPS.

Yes, always compare with rented dedicated, and only go with colo if it makes sense. I did this analysis last year and rented won big time. Never looked back since. Main points in favor of rented dedicated servers: - Much cheaper in my business case (5-10 TB/month by server, only a few high CPU requirements) most dedicated come with bandwidth included - No need to take care of the hardware in case of failure (opening a ticket versus managing the whole process) - Easier to switch machine if needs changes - Scales faster (as quickly then 2-3 hours with decent automation tools, and virtualization) - No switch, firewall, etc (IPMI should always be behind a firewall, never opened on the Internet) - The upfront cost to buy a server was high, and monthly cost of dedicated was actually lower, due to the lack of competition in the colo sector in my area (Montreal, Canada), prices were (and still are) rising quickly, it was hard to predict the long-term costs

How does the cost work out vs. colo when you consider capitalization?

Not trying to dissuade you or imply this is exactly the same thing, but I wanted to make sure you're aware of Amazon WorkSpaces [http://aws.amazon.com/workspaces/], "a fully managed desktop computing service in the cloud."

I'd like to know more about how you do that. I just looked at Opus' site and the 4U they advertise in Hillsboro is pretty good when compared to what I pay for 1U of hobbyist colo in Seattle. Do you put multiple machines inside the 4U with a switch in the same space? Can I be so bold as to ask for pictures? What kind of physical access do you get? I have a 1U server-grade machine but moving to a more flexible space would be nice, even if it does mean a train trip to go see my boxen.

I would also add that in most DCs your never going to have the power capacity to have racks filled with 1RU servers, so 2RU servers are typically not much more expensive to Rack then their slimmer brethren.

Cost: https://blog.serverdensity.com/saving-500k-per-month-buying-...

i think AWS is good, when you're not sure if you will use all resources of your servers or need scalability. If you know your growth rate, what type of resources are used, etc. it will be cheaper to have servers in a datacenter instead of using AWS. Many companies, upon growing, starting to rent a space in datacenter or even build their own.

AWS comes with a ton of added functionality that may not be relevant to you, especially if you've no need to scale or deploy machines in seconds. That comes at a cost.

It's a shallow orange vs. apples comparison, but look at the price of colocating 1/3 rack at Hetzner (119EUR). If you already own the hardware, have no need to use any advanced AWS feature, have the skills to manage it, etc.. sometimes it make sense.

For personal and/or SMB needs, it usually not worth the trouble (specially regarding skills to maintain it).

So you're right, it's becoming more and more a niche.

Cloud is definitely better if you need to scale out, but if you need either raw performance, have requirements for particular hardware (e.g. Backblaze with their custom disk shelves, certain hardware appliances, tape backup requirements), or if you want the cheapest cost per month then a colo may be right for you. Remember it's cheaper because you're paying for the sysadmin time and remote hands, versus Amazon with their people.

You could also do hybrid approach - colo for core infrastructure and cloud to scale out, but that's more difficult to setup.

At a certain scale of colo or with very heavy security requirements you'll find it's cheaper to have your own datacenter.

Cost is the big one, If you can manage your own metal you can save a LOT. 50K AUD of gear would cost us ~ 10x PER YEAR for the same VM sizes (many are 32gb of Ram, 8+ vCPU, reasonable IOPS).

IP addresses is another. AWS gives you 5 before you have to start applying for them. We do email marketing, so we need a lot of IPs (we give dedicated IPs to each customer). With APNIC (Asia Pac) we have a /22 and a /23 range (1536 IPs) for ~ $2900 per year.

Legal reasons are a whopper as well, Australia has some tight laws around privacy and liability if an overseas partner leaks your data.

We still host a lot of things on cloud providers (Rackspace & AWS in both Sydney and US) sites, but at the end of the day our VMs out perform cloud VMs and are considerably cheaper.

cost! Cost!! COST!!!

I collocate 5x 4u servers with 24 and 36 drive bays, 128GB ram, SSD drives squished in for OS, for a total (multiple raid 5 volumes for each 6 disks) usable space for my project of 375 TB

Power and 2-3gbit of bandwidth is included, as well as remote hands to check up on server (for example IPMI sometimes does not work) or replace drives (pay extra for enterprise grade drives would save you alot of hassle in long term!)

for .... €1500 / month

now go and calculate the cost of that on AWS

I priced and ran benchmarks for my last server upgrade. AWS was 2x the cost of collocating and some outages around the same time. I think it would have been only marginally more expensive since I could scale my server down to current need rather than project 3-4 years into the future.

AWS had the fastest network and but colocation allowed for cheaper CPU and RAM upgrades. As it turns out, my RAM/CPU needs plateaued at 4x my previous server so I have twice as much as I need. AWS got steadily cheaper over that same time and the servers got faster.

Upgrading to a new server is pretty disruptive. It would probably be easier to do slowly over time rather than big jumps.

I can't see a type of datacenter it doesn't apply to. You pay for transit, power draw, and to a lesser extent rack units. If you get a rack or a half rack with your expected power draw and want to spill past that they'll be charging you extra because that's space they can't sell to someone else. Some datacenters also provide cages, so your hardware is physically separated from other people. That'll cost extra too.

The only thing I didn't see a mention of is DC power, whereas your out of the box power supplies on most OEM equipment is for AC. Most of the server supplies nowadays should be able to handle 240V, 208V, and 120V with AC on the same unit, when you go DC you want to consider buying a separate AC power supply for setting up the server in your office (unless you drop ship it to the colo).

Make sure you get a very efficient power supply too, because while you can get the most efficient or power miserly server on the planet, an inefficient power supply will increase the draw significantly. You also want to right-size the power supply, because drawing too little power lowers the efficiency (there's an efficiency curve available for most PSUs that are rated).

A colocation center is a type of data center where equipment, space, and bandwidth are available for rental to retail customers. Colocation facilities provide space, power, cooling, and physical security for the server, storage, and networking equipment of other firms—and connect them to a variety of telecommunications and network service providers—with a minimum of cost and complexity.

Colocation by no means indicates you have multiple servers that need housing nowadays.

The irony is delicious, you'd have fared much better with the first sentence alone.

Your server is co-located with other customers' servers.