Linode NextGen: The Hardware
blog.linode.com
blog.linode.com
I was really hoping that was going to be this announcement, however the fact they've titled it 'The Hardware' hints that there might be nothing new on RAM in the next announcement, which would be a shame. Upgrades to CPU are nice, but it'd be nicer to see a RAM upgrade, as almost everyone is constrained on RAM rather than CPU or disk, particularly on newer hardware and with their new bandwidth limits.
I feel rather ungrateful now having said all that. Thanks anyway Linode!
[1] http://www.linode.com/faq.cfm#how-many-linodes-share-a-host [2] http://ark.intel.com/products/64595/Intel-Xeon-Processor-E5-...
Taking Linode's past behavior (history of reinvesting profits into upgrading hardware/infrastructure and passing that value over to the customer rather than just milking higher margins), I'd expect them to someday upgrade RAM when it becomes economically feasible.
https://news.ycombinator.com/item?id=5396634
Not sure where that pic is from or whether it is legit, but if they double the RAM on all plans as it implies I'll be really pleased... there's some intriguing text top right too which looks like it might be some sort of stats monitoring system (longview?).
They had a demo of Longview on the show floor too, from what I gathered it is a package install based daemon to monitor processes and the like, something like the server part of NewRelic but sitting directly in their datacenters.
Keep in mind that the CPU's they mentioned are ~$1600 each in lots of 1, I'm sure they are getting a good discount for volume but that is still a lot of $ per box in CPUs, I would imagine they need to pack them full of VMs to offset their costs.
Well that is disappointing and arguably more important for many people. Not to be a debbie downer or anything.
> We’ve also moved to the latest generation of the enterprise-grade hard disk drives — doubling their cache, increasing their port speed, and decreasing latency and access time
It would be nice to have SSD as an (expensive) option for those that can absorb the cost.
Would be even happier with more / cheaper memory.
also perhaps a more elastic storage option to stretch/shrink your linode disks, take snapshots (not assigned to a given linode) would be great.
They have startup program: http://www.saphana.com/community/learn/startups
It also costs an absolute fortune (only approved hardware allowed) and even the AWS instance which costs $3.50/hour is not certified for production deployments. I don't know many startups who can afford $5000 a month for minimum redundancy.
You might try building a new server on Digital Ocean with VirtIO enabled. It's an experimental option for new servers and you can contact support to have it enabled on existing ones, but should show further improvements according to some other folks in the support community.
To get what I want from my database, I want SSDs. If I move my database to another provider, then I have to move everything else to keep the latency to the database low.
Not having an SSD option even when it's expensive leaves me shopping around for where to move all of my Linodes.
When you're looking at around $8/GB for storage of this type, it's no surprise that Linode didn't bother.
Surprisingly, HP is offering the drives with 3-year warranties, which is a far cry from sometime ago where they were something like a few months.
[1] http://h30094.www3.hp.com/product/sku/10238268/mfg_partno/VM...
It might not be a profit center on its own, but it could land them customers who otherwise would have to skip them.
I would take this further and argue that consumer grade SSD's shouldn't be discarded out of hand either for similar reasons.
The fact is, if you care about IOPS, SSDs are cheaper than spinning disks. In other words, you need to stripe out across way more spindles (and thus consume a lot of power) to get the IOPS you get from a decent SSD.
Even an expensive SSD is _cheaper_ than a HD on an IOPs to IOPs comparison, and IOPs is what matters to most people.
> "we instead decided to go with mainstream SSDs to lower the risk of a random mechanical failure. We didn't need the endurance of an enterprise drive in these machines since they weren't being written to all that frequently, but we needed reliable drives. Although quite old by today's standards, we settled on 160GB Intel X25-M G2s but partitioned the drives down to 120GB in order to ensure they'd have a very long lifespan." (http://anandtech.com/show/6824/inside-anandtech-2013-allssd-...)
In the comments, Anand mentions that he went with the X25-M since the upgrade was last year; if he were doing it now, it would be the Intel 710 or S3700. Also worth noting that reliability, not speed, was their primary motivator.
I wonder if there's a way that Linode could open up that as an option to subscribers? The disks have a shorter lifespan with high writes, but that's measurable, and could be worked into the cost.
MLC (Multi-Level Cell) SSDs are slightly slower, a lot cheaper, and less reliable than SLC (Single Level Cell) SSDs.
Fundamentally, an MLC stores more data in each cell (hence the Multi), while an SLC stores 1 bit of data in each cell.
There's now all sorts of "eMLC" drives which are pushing the reliability of MLC up towards the SLC levels, but they're not very common yet.
Wow.
I was using a whole bunch of c1.medium instances and found the CPU performance varied by a factor of at least 2.
Context switching is less of a problem when you consider that the L1 and L2 caches of the CPU(s) are also shared amongst the 100+ VMs running on it.
If you're running something that is CPU intensive, or sensitive to something else taking over the cache, then running in a VM can be painful especially when you're not in control of what else is running on that server.
There's a link to the site at the top, and a link to the blog below that.
On top of this they mentioned before this process started that they had something new coming that doesn't require you having a Linode on your account.
Edit: though I certainly agree it has been long enough since the last RAM upgrade, and I would love to see one soon.
Linode is expensive but they give you lots of control in your account panel, you know they'll be around for a long time, excellent support, IPv6, and on and on. Not that DO is bad though. They've got SSDs which is awesome, they're cheap, free snapshots, and so on.
I'm a happy customer of both services and I don't think one is better than the other. I think both serve different types of customers. For me, personally, Linode is where I go to host my serious projects - the money making ones. Digital Ocean is where I go for a quick, cheap server to mess around with. I have one "main" server on both services but I trust my Linode more. Maybe that'll change as Digital Ocean matures.
8 cores does give you the highest best-case performance, since you can use 100% of each core if the other Linodes on your host are not using them.
They show the 1024 Linode with 8 CPU's as an example. But they also state that there are on average 20 1024 Linodes on each host machine, making 20x8 = 160 vCPU's being used.
Marketing-wise, more cores sounds impressive. But I wonder if performance per physical host is reduced splitting them into so many vCPU's
The Ivy Bridge based E5 v2 is also coming out in Q3 this year, and supposedly with 12 core models, so it seems that this is a somewhat poorly timed upgrade on Linode's part. They should have either upgraded to the E5 v1's much sooner, or just wait it out for the E5 v2's.
:~# dd if=/dev/zero of=test bs=64k count=3k oflag=dsync && rm test 3072+0 records in 3072+0 records out 201326592 bytes (201 MB) copied, 41.3478 s, 4.9 MB/s
this a 512MB Linode but still...
I hope they fix this with the new hard drives
Linode 512: 3.96647 s, 50.8 MB/s
DigitalOcean 512: 5.87315 s, 34.3 MB/s
(though the digitalocean one feels much snappier)
<code> 3072+0 records in 3072+0 records out 201326592 bytes (201 MB) copied, 5.52118 s, 36.5 MB/s </code>
201326592 bytes (201 MB) copied, 3.05352 s, 65.9 MB/s
Atlanta, GA74.5, 70.5, 55.4, 56.1, 84.8, 75.1, 73.7, 58.0, 67.5, 61.8
Average of 67.7 MB/s
Dallas
201326592 bytes (201 MB) copied, 4.08309 s, 49.3 MB/s
201326592 bytes (201 MB) copied, 5.22498 s, 38.5 MB/s
201326592 bytes (201 MB) copied, 5.85255 s, 34.4 MB/s
201326592 bytes (201 MB) copied, 2.94301 s, 68.4 MB/s
3072+0 records in 3072+0 records out 201326592 bytes (201 MB) copied, 2.31272 s, 87.1 MB/s
London 512:
201326592 bytes (201 MB) copied, 4.93267 s, 40.8 MB/s
201326592 bytes (201 MB) copied, 5.89248 s, 34.2 MB/s
201326592 bytes (201 MB) copied, 5.97411 s, 33.7 MB/s
London 2048:
201326592 bytes (201 MB) copied, 4.50047 s, 44.7 MB/s
201326592 bytes (201 MB) copied, 2.90073 s, 69.4 MB/s
201326592 bytes (201 MB) copied, 3.30499 s, 60.9 MB/s
another London 2048:
201326592 bytes (201 MB) copied, 3.16025 s, 63.7 MB/s
201326592 bytes (201 MB) copied, 2.86327 s, 70.3 MB/s
201326592 bytes (201 MB) copied, 4.0267 s, 50.0 MB/s
Newark 4096:
201326592 bytes (201 MB) copied, 3.19571 s, 63.0 MB/s
201326592 bytes (201 MB) copied, 4.17497 s, 48.2 MB/s
201326592 bytes (201 MB) copied, 4.60173 s, 43.8 MB/s
Atlanta 4096:
201326592 bytes (201 MB) copied, 2.19477 s, 91.7 MB/s
201326592 bytes (201 MB) copied, 2.63818 s, 76.3 MB/s
201326592 bytes (201 MB) copied, 3.04094 s, 66.2 MB/s
Dallas 2048:
201326592 bytes (201 MB) copied, 2.65925 s, 75.7 MB/s
201326592 bytes (201 MB) copied, 3.11197 s, 64.7 MB/s
201326592 bytes (201 MB) copied, 3.45745 s, 58.2 MB/s 201326592 bytes (201 MB) copied, 3.11716 s, 64.6 MB/s
201326592 bytes (201 MB) copied, 2.70099 s, 74.5 MB/s
201326592 bytes (201 MB) copied, 3.28993 s, 61.2 MB/s
201326592 bytes (201 MB) copied, 2.98622 s, 67.4 MB/s
201326592 bytes (201 MB) copied, 2.93612 s, 68.6 MB/s201326592 bytes (201 MB) copied, 2.61587 s, 77.0 MB/s
201326592 bytes (201 MB) copied, 3.06237 s, 65.7 MB/s
201326592 bytes (201 MB) copied, 5.70759 s, 35.3 MB/s
201326592 bytes (201 MB) copied, 6.24756 s, 32.2 MB/s
201326592 bytes (201 MB) copied, 3.98084 s, 50.6 MB/s
EDIT:
For comparison, here's the same test on a 4GB DO instance (Amsterdam DC):
201326592 bytes (201 MB) copied, 6.16116 s, 32.7 MB/s
201326592 bytes (201 MB) copied, 5.08132 s, 39.6 MB/s
201326592 bytes (201 MB) copied, 7.73071 s, 26.0 MB/s
201326592 bytes (201 MB) copied, 7.55657 s, 26.6 MB/s
201326592 bytes (201 MB) copied, 7.9936 s, 25.2 MB/s
Skeptical. I'm wondering if this is just advertising hyperbole because Linode doesn't seem to be large enough (and there is no evidence to indicate any "funding") to be able to "invest millions".
They operate out of a suite in an office park outside Atlantic City NJ.
(I think it's a great company by the way I just don't think they are investing millions it doesn't make any sense given what I know about them.)
I'd like to point out as someone that has made the INC 500 list and knows the process very well that there is no vetting to the numbers that you give them. Back when I did it you simply needed a letterhead from your accountant and Inc went with whatever you said. (Might have changed but that's the way it was). There is obviously no audit. And there is no question that people fudge to get on the list in various ways because it's a good marketing tool.
Linode is not a public company releasing information. Once again I'm not saying the info isn't correct and that they don't have that many customers. I'm simply pointing out that the fact that they issue a press release saying they have 45,000 customers (along with the math that you are assuming gives them $20/customer) isn't necessarily correct.
Another way to approximate their number of customers would be by IP address assignments:
http://bgp.he.net/search?search[search]=linode&commit=Se...
I count 176,640 IPv4 addresses, and that's only direct allocations - their older customers are still using IP addresses owned by Linode's datacenter vendors (SoftLayer, HE, etc.).
If what you say is true, I have no doubt that some companies fake it to get on the Inc 500 list, but given the facts about Linode it seems fairly unlikely that they faked it, and quite possible that they could be spending millions on infrastructure upgrades.
Who buys equipment upfront for cash at that scale?
# grep MHz /proc/cpuinfo
cpu MHz : 3922519.116
cpu MHz : 3922519.116
cpu MHz : 3922519.116
cpu MHz : 3922519.116
That's on a rackspace small instance. 4000 GHz, baby!Linode is the only VPS I've seen that gives a gross number of cores to everyone. Instead of allocating a cpu count correlated somewhere along side plan-tiering, they just give everyone a lot of CPUs (8), and then they use tiers to determine priority, who gets to actually use those cores.
An entry level plan has 1/16th the CPU priority of a top tier user, and pays 1/16th as much and has 1/16th as much ram. If you're not RAM bound, for a somewhat loaded system you'll see slightly better than linear scaling (since you aren't paying as many context swaps) as you move up in tiers.
I think it's a really bad policy to give entry level users scheduling on so many cores: their workloads will never stick on a local cache! Their jobs are going to be bouncing around the system getting evicted from everywhere and bouncing to the next place where they can trash it's cache next. This is a marketing gimmick, a "our number is higher" trick, played on people like you jedbblue!
Some time in the past: the Pentium IV netburst core was going to hit 4GHz, then 5GHz. The mhz wars were on, mhz sold, they were what people saw, first and foremost, and it's more the fact that the insanely-deeply-pipelined core didn't even scale that scrapped the idea than people wizening up. I'm not sure if Linode's scheme here is as bad as all that, but it seems unnatural to me, and I certainly would not consider 8 cores to be a boon in this circumstance.
Edit- this does change things a good bit! A quote from elsewhere in this thread I was not aware of-
We limit the number of Linodes placed on each host
machine. We also only place one plan type on each host. In
the worst-case scenario, you're splitting CPU time evenly
with your fellow Linoders, but are still able to use the
full potential of the host if others are idle.
This is a little different circumstance than what I'd guessed at above as you're not going to be outclassed by higher tiers, you just have to play with peers, and pay to have less of them.For instance if the previous virtual cores were all 1ghz and new cores are all 500mhz, doubling the cores won't do much good if you don't know their clock speed.
I was critical of last Linode's NextGen post discussion, because I didn't think upgrade was impressive compare to others, but this is one is a nice bump, memory is probably what most people will care about more though. So maybe next refresh is the memory?
processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Xeon(R) CPU L5520 @ 2.27GHz stepping : 5 microcode : 0x11 cpu MHz : 2266.746 cache size : 8192 KB physical id : 0 siblings : 8 core id : 0 cpu cores : 1 apicid : 0 initial apicid : 1 fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu de tsc msr pae cx8 cmov pat clflush mmx fxsr sse sse2 ss ht nx constant_tsc nonstop_tsc pni ssse3 sse4_1 sse4_2 popcnt hypervisor bogomips : 4533.49 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management:
"And we’re upgrading all Linodes to 8 cores! Right now. As in all you need to do is reboot to double the computing power of your Linode. By the time the host refresh is completed the average Linode will be running on hardware that is less than 1 year old."
I, for one, thought that it was ready now after reading that. In fact, from the comments, it's clear that for some customers, this is in fact ready:
"BizzarTech: Rebooted my 1024 and moar cores!!! Thank you!!"
<snip>
processor : 7
vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU L5520 @ 2.27GHz
stepping : 5
microcode : 0x11
cpu MHz : 2266.746
cache size : 8192 KB
physical id : 0
siblings : 8
</snip> processor : 7
vendor_id : GenuineIntel
cpu family : 6
model : 44
model name : Intel(R) Xeon(R) CPU L5630 @ 2.13GHz
stepping : 2
microcode : 0x15
cpu MHz : 2133.460
cache size : 12288 KB
physical id : 0
siblings : 8 processor : 7
vendor_id : GenuineIntel
cpu family : 6
model : 45
model name : Intel(R) Xeon(R) CPU E5-2650L 0 @ 1.80GHz
stepping : 7
microcode : 0x70a
cpu MHz : 1800.059
cache size : 20480 KB
physical id : 0
siblings : 8
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 13
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu de tsc msr pae cx8 cmov pat clflush mmx fxsr sse sse2 ss ht nx constant_tsc nonstop_tsc pni pclmulqdq ssse3 sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes hypervisor ida arat epb pln pts dtherm
bogomips : 3600.11
clflush size : 64
cache_alignment : 64
address sizes : 46 bits physical, 48 bits virtual
power management:
Edit: I should perhaps mention that this is a 512MB Linode, on the off chance that it makes a difference.http://www.linode.com/faq.cfm#how-do-i-get-my-fair-share-of-...
> We limit the number of Linodes placed on each host machine. We also only place one plan type on each host. In the worst-case scenario, you're splitting CPU time evenly with your fellow Linoders, but are still able to use the full potential of the host if others are idle.
Probably - according to their FAQ it looks like each host has about ~20GB of memory which is pretty small these days.
"This will issue a pro-rated credit to your account"
It's not per hour billing as such, but if you spin up Linodes and cancel about X hours/days you do get the leftover fees prorated against your account.