Intel Xeon D 12 and 16 core parts launched: first benchmarks
servethehome.com
servethehome.com
I hear this type of argument a lot, but it's so important to include the cost in engineering and talent necessary to keep your datacenter humming.
If you're a small shop with a small future growth expectation then sure, forget Amazon and start racking your own boxes. Just be ready to hire ops staff competent to run your operation. You need to be realistic about both aspects of cost.
Let's say you're renting half a rack with 100Mbps unmetered bandwidth and 20amps of 120V for $500 per month. So you have 16 amps / 1920 watts of continuous power you can draw. These Xeon D's are very lower power, although strangely Pat doesn't report the actual idle and full load draw as measured by a Kill-o-Watt meter. With the 10GBE I can't imagine budgeting much less than 150 watts per machine. If you can put let's say a dozen of these in that half rack, in my contrived example, the "colo cost" would be $41.66 / month or just $2.60 / core.
Across all the SKUs in OP the TDP is 35-65 watts. 20 More watts for the new 1587 SKU compared to the benchmarked 1540 in the anandtech article. So a rough estimate would be maybe ~95 watts at under load? Assuming the board is the same and only the CPU has changed that should be a pretty good estimate.
You're right it mostly doesn't make sense to compare machines in that way due to all of the variables surrounding colo costs, max load wattage is the metric I should be interested in.
We also have a few racks in Sunnyvale, California that we use APC metered by outlet PDUs for and that we actually calibrated by testing with the Extech units and found two of over a dozen that gave us consistent readings across PDUs and ports.
This unit is in a SC515 chassis with redundant PSUs. Even with 4x 7200 rpm Seagate 2.5" drives, a 64GB mSATA drive and a Samsung SM951 m.2 SSD it is still pulling 0.24A on 208V (so 0.48A on 208V). If you were to gain a bit of efficiency running a single PSU, you could easily fit these in 1A on 120V power envelope.
What we have seen several folks do with similar D-1540 machines is actually use 1U 1A/ 120V hosting (which is very inexpensive) to deliver cheap local PoPs for their applications. We have been strongly considering decommissioning our Las Vegas 1/2 cab DR site and moving to this sort of distributed model as there are a lot of benefits with this.
We published more complete benchmark results ( http://www.servethehome.com/intel-xeon-d-1587-benchmarks-16-... )of the D-1587 yesterday, expect more in terms of reviewing the overall platform next week. The Xeon D is very platform dependent for power and the particular unit we have does have the additional LSI/ Avago/ Broadcom SAS 2116 (16 port SAS2) HBA onboard.
I assume you meant to write "(so .48A on 120V)". 50W idle with 4 spinning drives, an M.2, and the redundant PSU really is really very impressive. As you say, keeping full load under 120 watts to be able to fit in a 1U/1amp colo gives an incredible value.
The hardware's not even that expensive! And for $50 - $60 / month in colo cost [1], the reliability of systems now with SSDs, and the amazing orchestration tools that are available, the likes of AWS, DigitalOcean, etc. start to look really expensive.
http://www.servethehome.com/going-colo-series-part-1-decidin...
http://www.servethehome.com/colo-series-part-2-picking-coloc...
http://www.servethehome.com/colocation-years-learning-experi...
Bottom line, if you can host your app and resources on less than a 150 machines in Amazon you win, more than that and you're leaving money on the table. Once you get to the point where you are deploying new datacenters to support your customers you get a huge boost in operational efficiency.
But to the point of many other folks here, if you're all cloud you can really shrink fast if you need too. Lose a millions customers? No problem, just drop some machines. Any business that can price into the monthly customer fee the marginal monthly cost of AWS to support that customer, can keep their costs flat when their customer counts vary. Someone with dedicated hardware is still paying the bills for it, even when their customers leave.
Sadly I don't expect them to share what those costs are. I have always felt that Amazon could, in theory, always out compete with NetFlix with their Amazon Prime Video because they would only have to charge the marginal rate for the hardware. However to date it seems like the content contracts have kept NetFlix on top.
Risk mitigation is part of the cost of infrastructure that needs to be factored in, and if your business has a downturn, or your compute needs turn out to be lower, then you can scale down your OPEX pretty quickly. If you had bought them, you'd be stuck with them. (See Zynga)
The operating cost of doing it yourself is always hard to pin down, and include many many things. OS/Hypervisor licenses, staff, hardware, cooling, redundancy, disaster recovery, power, or Colo fees etc. The advantage with AWS, is that this just a few numbers on a few various services and it is very predictable.
And that is not considering other AWS offering. I've come to really like DynamoDB. It takes some time to get used to, but I've found it's solving more and more problems for me, without having to scale OPS engineers. There is a danger of lock-in, but I guess as I am a paying customer, it's not going away anytime soon, and Amazon is probably not jacking up the price either.
Surely your calculation doesn't / can't include bandwidth charges? And if you have Windows hosts... cloud screws you too.
You're not wrong that it takes a lot more machines than most shops need, and if you can make your scale elastic, 150 machines probably goes a long way! And to your point, developers undervalue their time a lot, and dealing with colocation and sourcing of hardware and backups can be a real drain to save some on CapEx.
AWS/GCE/Azure are all really expensive compared to even good multi-homed transit. If you push more than 10TB and you don't need multi-homing you are already ahead.
If you do need HA multi-homed transit you are looking more at 50-100TB or so but still, there are a lot of places that easily do that every month, especially if you are doing cross-DC storage replication for example.
And we use Linux so no Windows hosts charges.
There are more and more tools that are force multipliers for your Site Reliability Engineer (SRE) equivalents. You can't avoid swapping out dead drives, but with the right systems architecture you can make it pretty painless.
Ignore the whole song and dance when they have to bring in a VP into the call to specially approve the pricing they're trying to pitch to you at. Those are standard transit sales tactics. Or avoid Cogent's sales entirely by working with a reseller.
This goes back to my first point in that going into the DC requires staff with very different skill sets. It's not that it's not worth it, but there's a lot of costs involved part from the per-hour instance cost.
I wish the industry would stop quoting prices they way we do (USD/Mbps, so e.g. $1.20/Mbps); Gbps is probably the right unit now, and it may make more sense to invert.
But there are lots of terms involved in a transit contract; unless you're buying tens of gigs, crossconnect, port cost, commit, etc. may be more meaningful than per-Mbps cost. And pricing often depends greatly on exactly where in a city you're buying (at IX, carrier neutral facility, etc. will be cheapest; off net building or monopoly building will be most expensive) -- and of course Cogent transit vs., say, Level(3) transit are only superficially the same.
Also, having 150 machines in a single location is rarely optimal.
That is just statistics of course, but one of the things I tried to do in the Blekko infra-structure was to mix the ages of the SSDs to mitigate this risk.
They didn't arrive on time.
Good thing I had backups.
What's the practical way to avoid this? Staggering your SSD purchases?
Considering that you can easily do dedicated for 1/7th the cost of Amazon for the equivalent amount of resources, it doesn't take anywhere near 150 machines to realize cost savings, it's more on the order of one complete physical machine as a dedicated server. You can purchase a 2nd one for redundancy and/or for always available additional capacity, so that you can burst to double your usual peak performance and still save 5/7th the costs of Amazon. If you're expecting to require the ability to scale much higher than 2x, you can also purchase a bunch of servers just for 1 month, or even less time depending on the provider, or take a hybrid approach if you really need an extreme amount of dynamic scalability. The reality though is that such extreme scaling is in the minority of cases.
I'm not sure how you arrived at the 150 machines number, but that sounds like 3-4 full cabinets and about right to have a reasonable scale for colo.
I'm surprised how often I see arguments like this as well. If it takes even a few days of time for your employees to manage the hardware and you're only saving a few $100 a year buying the hardware yourself, you're already making a loss.
This comment gave me pause, I feel like I'm falling out of pace with the evolution of commodity server systems. We've gone from being network (10/40gbe) and disk io (fixed with SSD, PCIe NVRAM and Fiberchannel) bound in the the near past to swing towards memory capacity bound?
The more you can keep in memory, the better off you're going to be, even with the fastest SSDs in the world.
It all depends.
In a high-end server, with PCIe flash or NVRAM, you're increasingly unlikely to be IO bound.
Which is silly, because controllers can simply be designed to handle more data, and likewise they can be made more power efficient. So, far the only real limitation has been the interface, which off-course can be changed.
NVMe-backed storage, 10 GigE-backed network IO, and 16-core CPU gets you pretty far nowadays for ~$5K USD, and RAM starts to be the limiting factor. Disk IO and network IO can be saturated pretty easily, and that kind of work usually involves a lot of RAM so here we are.
I assume there's some market segmentation happening here, but it's also a way to reduce board size and pin count.
So i guess the problem is with address space as mentioned above.
So no, I don't think so.
Right now we're living in a golden age of financially accessible super-computing. We don't need to deal with MPI anymore, we just use RDMA and have Infiniband speed fetches to any machine joined to the cluster[see: MS Paper]. I was working on RELION as a favor to my father and got deeper and deeper into it because my docket was pretty open and it was fascinating. Long story short, I sketched up a test setup with Phi, RDMA (which admittedly wasn't used that much, as the problem set would be characterized by anyone as "embarrassingly parallel") and 10GBit. This is all commodity stuff, and we effectively never touch disk -- I cobbled together 18650's in the rack[see: batt] instead of a UPS with a uC which fires off a 'persist state to disk' message on any power interrupt but other than that it's all RAM and blazing fast. (Side-note: anyone who read the Nature Methods special on all the Cryo-EM stuff may be seeing a paper or two with a few new sets of computational methodologies in the near future ;))
Buyer beware though -- sacrificing clock speed for more cores can end up costing you a lot more overall[2]. Licensing policies have a tendency to change around from version to version, so your perfectly licensed Oracle 11g (lets say it was quantified by the physical chip you throw into the socket) might not have a straight forward upgrade path (now by the number of vCPUs). A lot of clients of mine got burned trying to move from Iron to AWS, only to realize that Oracle licensed per _available CPU_. Fines galore. They ended up downgrading to a previous revision Xeon (i.e. 4th gen E5 MSRP $x,xxx instead of 5th gen E5 MSRP $0,xxx with better performance but more cores). Anyone encountering this problem should keep that trick up their sleeve and find some 1 year old off-lease equipment with less cores but a higher clock speed to keep license compliance and still meet your computational demands. Anyways, yeah the latest and greatest might not be the most economic of choices for this reason[licensing].
[1] https://news.ycombinator.com/item?id=10805087 [2] This has so many variables as one can imagine - you can't just go by what people did in 1975 i.e. "this instruction takes x cycles, we can physically count how long our computation will take with a sheet of graph paper" because of pre-fetching, pipelining, cache patterns, the use (or lack of use) of the AVX(2) registers, etc etc. [3]https://software.intel.com/en-us/articles/intelr-xeon-phitm-... [MS Paper] http://sigops.org/sosp/sosp15/current/2015-Monterey/printabl... [Batt] As you probably have guessed, I have no formal EE training. I did however rigorously read all of the safety datasheets, use very high quality Panasonics, and strictly conform to the CC-CV guidelines so my fathers lab would not catch on fire. This is probably not the best idea, but it's electrically safe and isolated. It'd pass UL certification.... at least I think ;) [licensing] If you're a corporate entity who is moving into any sort of cloud or onto new hardware, my firm has quite a bit of expertise in licensing compliance when either a) shifting to new hardware, or b) shifting to the cloud, as well as getting your best bang for buck with existing licenses.
How did your tests go with this? I'd totally use a system like this - properly packaged up of course. Li-ion and Li-poly make me nervous.
The precautions I took were probably overkill but since it wasn't my lab I chose to go pretty overboard. In addition to conforming to the strict charge-discharge guidelines Panasonic thoroughly delineated in their sheets, were to use 1: use "protected 18650's" (actually designated as 19760) which have internal circuitry to keep, say, cheap chinese eBay charge-units from setting peoples house on fire, 2: sourced the batteries from a vendor I've trusted for ages (since the Panasonic 'greens' have such a popular reputation, and knock-offs make it into all sorts of markets including first-tier shops like Amazon and Digikey, I asked my vendor to source directly from Panasonic JP, which he did and provided me with a packing slip). Fear of Li-* is reasonable, and I figure risking permanent damage to ones body and/or life isn't worth the 7 bucks you save buying no-names - go with the 19760's and live life to the fullest! (Pro-tip, the form factor of the 760s are about a mm or 1.2 mm or something longer than unprotected 850s due to the added circuitry -- which I presume is just a PTC thermistor + kill switch, I haven't looked it up though. I'm sure there's a mass difference too if you have access to lab quality scales or are friendly with your local cannibus vendor.
Thrown in the case I have a live K-type thermocouple in all 8 of the 1u's (a pair in each rack) with conservative failure policies should temperatures exceed parameters I set. I put put some nice fusing (HRC.. why? because I was already going overkilling and since this wasn't an off-the-shelf component, and I'm not an EE with extensive PDU experience I decided to go safe. (I also consulted with a post-doc buddy of mine who's actively in lithium battery research and after a good chuckle told me the precautions I took were satisfactory.) It came in at around 5k USD for all 8 units with everything, including ~115 a piece for the Hammond 1u's. Each unit can sustain a little under 4.5 kVA which gives more than enough buffer time for a graceful shutdown
http://www.cpubenchmark.net/high_end_cpus.html
"The King of the Hill" is only 2.3 GHz ?
For servers, doing more things slightly slower is (usually) better than doing fewer things faster, so Intel usually puts core count over clock speed for their Xeon CPUs. For a desktop system that's unlikely to be doing more than 3 or 4 things at once, you can prioritize single core performance and higher clockspeeds.
It's also considerably more efficient to have more slower cores than fewer faster cores. Reducing power consumption and reducing heat is a win-win in a datacenter.
Video Encoding isn't that well multi-threaded either as it's some what linearly dependent (you can't just encode random frames without having the previous frames/key frames done for references which means you can usually do 2-4 frames at the time so run off becomes and issue when you have more cores than frames to encode), 3D rendering is also a mixed bag depending on what type of raster and post processing you use you will get substantially different scaling between number of threads vs pure clock speeds.
In any case video encoding and 3D rendering (at least the ones that one will do on a desktop) will benefit much more from multiple high end GPU's than from an increase in CPU cores, the more or less conversion is that GPU numbers give almost 1 to 1 scaling where additional CPU's (cores) give you 0.3-0.5 on average in the best case scenarios (there are a few cases where CPU scales almost as well as GPU but they aren't that common).
The boost levels are typically defined as "bins", i.e. the max clock if 1, 2, 3, etc. cores are active. Can't find Xeon-specific info, but this page shows some general information on the topic: https://www-ssl.intel.com/content/www/us/en/support/processo...
What one will see, very quickly, is that the new SKUs generally offer
slightly lower clock speeds to maintain a 45w TDP. Maintaining this
figure while adding 50% to 100% more cores and cache is no small feat
and it makes sense that clock speeds suffer. We also see the TDP
figures rise to 65w in order to accommodate more cores and higher
clock speeds.
We introduced the Core * Base GHz and Thread * Base GHz figures just
to show how much of an improvement this is. The new chips represent
double the cores but up to about 62% more clock cycles in aggregate
over what we had as the previous fastest chip, the Intel Xeon D-1541.
We also now, in the same TDP figure, have 23% more raw compute.(I don't know what this measures, though. It seems to be about 1,900 to 2,100 for the newer chips - even the one in my laptop. The score doesn't vary us much as the clock speed, though clock speed seems to be a factor.)
Source code:
Turns out this was an error on my part. All setup now.
Intel's supposed to be releasing the E5 v4s in the next few weeks, which will be Broadwell based. edit: These are also Broadwell based, supplanting the Haswells they released last year.