Thoughts on Colocation
blog.pinboard.in
blog.pinboard.in
Aside: back in the early days, a lot of YC startups hosted at HE. I think it was one of the first JTV colos, actually.
Not that I really need colo, could easily go cloud, back to dedicated, or even dirt cheap with some shared hosting provider for hosting needs, but I've always liked having my own customized setup in colo.
Here's why:
1) $50 per 1U (power included)
2) /27 IP range
3) 250GB monthly bandwidth
4) 24/7 DC tech support
Pretty basic setup, couple of Dell R610s with gigabyte switch and Cisco ASA. VMware ESXi runs on top of the bare metal, so basically have my own VPS environment.
It's a nice break from coding to learn a bit about the systems admin side of the fence, I like it ;-)
Of course, the times they are a changing, at some point I'm going to need to go Cloud. For now pretty content though...
Provider is SagoNetworks, based out of Tampa, and have a brand new-ish facility in Atlanta.
Started with them around 2003 and have not looked elsewhere, DC guys bailed me out early days while learning the ropes -- tech support is generally very good.
Couple of times a year the shit hits the fan, core router goes down or other mini-disaster, and you get nailed with a few hours downtime. Give them a ring, they say it's blah blah, we're working on it.
Otherwise, everything hums along, no news is good news ;-)
I feel like there's a logical progression from PaaS/VPS => Managed dedicated hosting => colo facility.
But I would love to know when the costs for these jumps make sense.
I'm assuming that you're in a position where you can't tolerate a few hours of downtime in the middle of the night if something breaks.
I feel like for most companies with a tech staff smaller than about 8-10, you're probably going to have all programmers/DBAs who may have some server skills, but not the competencies to go chase down complex hardware issues at 2am. Past a certain size real sysadmins will have plenty of work do the 99.99% of the time when you're not down, so it's natural to switch over to a colo situation.
There are also tax and accounting pro/cons to buying the hardware.
Part of it just boils down to risk tolerance I think, and how comfy you are with sysadmin skills.
I'm responsible a medium sized (< 10 servers) web site that use managed hosting for. Their advice/service is invaluable to me, partially because I trust them enough that I can just hire web programmers without worrying about their advanced linux/networking skills being super high. The host has also helped us debug some fairly deep problems that we don't have a chance at in-house like hypervisor config issues, and even a processor 'errata', AKA processor hardware bug.
It's also nice to have an ops team to ping questions off of, since we're not nearly large enough to hire a dedicated sysadmin otherwise (and they'd be without sysadmin work 50% of the time anyway).
NB: I'm treating 'sysadmin' like it's just one skill set to administer routers and server hardware, configure firewalls and networking, optimize database boxes, etc. This is probably not totally true, but fits for purposes of discussion. YMMV.
If you've had a box there for a few years, there's no guarantee that there's empty space near it for more servers or a load balancer.
For a lot of small companies that have 2-3 servers, 'fanatical support' simply isn't worth it. We still end up having to call them to reboot a box or tell them what to do. Or call them and ask why it's down, when they should be calling and telling us.
I don't believe most software folks realize just how much can be saved with colo strategies, building custom servers, buying directly from distributors, component sharing strategies, etc.
It's a great competitive advantage, 10x less burn rate or 10x the hardware of others. Or fall somewhere in between.
My experience is that the 1-amp-per-rack-unit rule of thumb applies only if every server in the cabinet is under full load (CPU and all spinning disks). This is almost never the case, however: our average utilization is around 8 amps per phase on a 3-phase circuit. (Load limits are per phase, not total.)
You can also get 30-amp circuits in most data centers if you're concerned about it.
Keep in mind, too, that your load is usually balanced across 2 PDUs (assuming you're buying systems with redundant PSUs, which you should) and on 208-230V power, which is more efficient than 120V due to lower resistance. If you configure your systems correctly, the load will be shared across both PDUs under normal conditions. That said, you'll still need to ensure you don't overload the remaining circuit if redundancy is lost.
In summary, don't worry too much about overloading a 30A circuit; there are plenty of full cabinets in a DC for a reason.
As for the California earthquake risk, one option is to colocate your servers in Dallas or somewhere with no earthquake risk. The Maunder article talks about why colocation doesn't tie you geographically to that place.
Everybody forgets the E part of EC2. If your workload is not elastic, or is very small, it's incredibly likely that EC2 is not the most cost effective solution.
this brings the hard costs much closer. (but doesn't close the gap entirely.)
Don't get me wrong, EC2 is an amazingly useful platform and it has been a game changer for our industry in a lot of ways. Improved hosting economics isn't generally one of them.
That is the thing, though, generally speaking? it costs a /lot/ to fill a space. 10% of the total value of the lease is not unusual. So they try to get you in with a low rate, then jack it up later.
Personally, I think a setup fee, then some sort of guaranteed low rate ongoing (maybe tied to the local price of electricity?) would make more sense. It's less costly for all involved if you sit at the same datacenter forever; it's just that the way that these things are sold, the owner of the data center wants to 'capture value' by raising your rates if it's hard to move.
The owner of the datacenter also wants to start you out with a low rate, for the same reason; if they can get you hooked so that it's hard to move, well hey, they might break even the first year, but after year two or three, they are looking pretty good.
Personally? I find this really frustrating. All parties, when working to maximize their own value, actually waste a whole lot of value for all involved. It's an example of capitalism destroying some pretty significant value.
You see this in almost all cases where switching service providers is expensive (and, I think, it's one of the reasons why "the cloud" is so appealing.)
Personally, I really like the idea of a 'condo' model for the data-center, and for renting in general. Yeah, you'd end up paying $100K+ for your rack, but it wouldn't take very long to make that up, and you'd end up owning part of the organization that controls the whole datacenter.
I think the big problem with the condo model is that in regular condos, usually you don't have individuals that own 100s of units, while that's not at all unusual in a data center.
Edited: Also, your "$100/month" quote is pretty fanciful. Who is handing out 32GB with SSDs for that price? I'd really love to know, because the best I've been able to get for a database quality machine (RAID-10 with BBU, SSDs, and lots of RAM) is closer to 10x that.
they're not talking about machines of that caliber.
I know that's not much space compared to the big boys, but it fit our needs at the time pretty well.
If you're looking for colocation stuff these days, and you're not huge and need to be in San Jose, I'd look at some of the spaces in Sacramento.
Other great spots around the country: Dallas, Chicago, Atlanta, and in Virginia around MAE-East.
I do not get that quote.
I have been a WHT member for years and in that time found dozens of excellent server and collocation providers.
Then again, outside of WHT, where else can you find the colo provider gems? Certainly the best rates will be found there in my experience; if you're lucky, provider tech support will be excellent as well.