EC2 High Memory Cluster Eight Extra Large Instance (244 GiB RAM) | Hacker News Reader

EC2 High Memory Cluster Eight Extra Large Instance (244 GiB RAM)

aws.typepad.com

65 pointsjeffbarr22 comments

Almost all the big EC2 outages have been due to EBS, and you have to pay extra for it as well. Netflix steers clear of it [1].

Does anyone know why Cluster Compute and Cluster GPU Instances [2] (including this new one) make using EBS mandatory?

[1] http://techblog.netflix.com/2011/04/lessons-netflix-learned-... [2] http://aws.amazon.com/ec2/faqs/#Does_use_of_Cluster_Compute_...

Likely because they are considered abstracted, throw away machines.

I believe it is a function of the fact that you are "owning" the machine.

I dont think it uses EBS, since the storage is using SSDs. they are "in" the machine, not remote, like EBS disks. If the machine dies, or moves instance, your storage is gone... As for if your boot disk is on EBS, thats a different question and one i cant answer...

Well, the article says "Support for EBS-backed AMIs only." i.e. you can only boot from EBS, you can't boot from instance storage.

Two reasons come to mind:

* the failure rate of an EBS volume (assuming a bug-free EBS software stack!) should be less than any one spinning disk

* these machines don't have disk space set aside for copying the root volumes to them from S3 to boot. (There's a 10GB limit, and to allow for this to be somewhere, you'd have to have another SSD, or partition the existing ones.)

There's a good blog post about it on Rightscale: http://blog.rightscale.com/2012/11/02/new-ec2-instance-types...

esh's "You Should Use EBS Boot Instances" also lays out a number of good reasons to use EBS for boot: http://alestic.com/2012/01/ec2-ebs-boot-recommended

You at least have two 120GB ephemeral SSDs to play with when you're running - a luxury you don't have on the new M3 classes.

It's not really the rate that's the issue here, it's correlation between failures. The chance of losing a third or more of your machines at once is enormously higher with EBS.

Partitioning a disk should take seconds; I don't see why it would be an impediment. As it is, someone can do it themselves but now it'll be a custom job and less reliable.

One way to scale Firebird Database is to put it on memory virtual disk or ramfs

On linux i can mount a partition in memory (Install ubuntu/debian)

sudo mkdir /mnt/ram sudo mount -t ramfs -o size=200G ramfs /mnt/ram mount to show you the partitions mounted

and then move your database into the ram partition

cp -rp /var/lib/firebird/2.5/data /mnt/ram

That large instance could help you with the Firebird cache settings also the extra SSD could do wonders

Inspired by the stack overflow big fat server architecture

http://highscalability.com/blog/2009/8/5/stack-overflow-arch...

Well, two things with that. Three really.

First, does AWS allow ramdisks? It's been a few years since I tried to mount a ram disk on Xen, but it was not possible at the time.

Second, most databases are smart enough to shuffle stuff into RAM for you, starting with indexes and commonly-accessed tables.

Third, the "D" in ACID stands for "Durable". If you want an in-memory store, use an in-memory store. RDBMSes do sorta kinda rely on the assumption that stuff written to disk is actually written to a disk.

Yes it AWS allows ramdisks , here is the output from a machine

root@ip-amazon-ip:/home/ubuntu# mkdir /mnt/ram mount -t ramfs -o size=2G ramfs /mnt/ram mount ramfs on /mnt/ram type ramfs (rw,size=2G)

There are the cache and memory settings Fore Firebird but is better to put the main full database on the fastest storage and that is RAM for the moment

In Firebird we have a nifty feature called Shadow that creates a life snapshots of the active database (Think of it as Replication in real time) so you can activate it and keep a safe database on the SSD/HDD

http://ibexpert.net/ibe/index.php?n=Doc.DatabaseShadow

Recently, antirez pointed out that redis was essentially memtest86; consequently, he has had to spend a lot of time chasing down phantom bugs due to memory modules with stuck/flipped bits. At that time, I think HN was left wondering whether Amazon uses ECC memory in its EC2 machines. Do we know whether that's the case now?

All Xeons and Opterons (in other words, all EC2) use ECC.

Colin Percival (cperciva) already has FreeBSD 9.1-RELEASE running on these!

http://pastebin.com/SCMUJNDt

This instance was announced at RE:invent 2 months ago, and is now available. You can try one out for $0.34 / hour with spot instances.

It does not seem to be possible to launch a spot instance of one of these... the web interface wont allow it, anyway... Still, about $3 per hour to play with a machine of that size should be interesting...

You might have to use the ec2-api-tools cli package to launch spot requests for this instance type right now. The AWS Console doesn't always seem to be updated at exactly the same time as new instance releases.

RAM isn't a cure all for big data applications and presents its own set of challenges. The largest issue is with traversal and data access latencies simply because of the virtual memory management overhead that is required.

I've run into this problem with our btree search indexes on bare metal (~96GB), some more reading:

https://github.com/johnj/llds

Some time ago (mid 2011), I was trying to use an AWS 64 GB machine for some largish work and discovered that I couldn't actually allocate anywhere near all of the 64 GB of RAM I was supposed to have access to. As I recall, writing a simple C program to do binary search to pinpoint exactly what the max amount of memory I could malloc was, it failed somewhere around 34 GB – so just over half of the memory I was supposed to have access to. Makes me wonder how much of this 244 GiB you can actually use when you spin up one of these machines.

I'd like to see a machine with an SSD from Amazon that wasn't so big and expensive.

I don't think that's possible. Big|Fast|Cheap, Choose two. (or one, under some circumstances).

I think you misread the parent. jahewson asked for a machine that is Fast and Cheap and Not Big.

More specifically they want to upgrade a weaker machine with an SSD instead of getting a high end machine with an SSD.

Nothing has been announced, but fwiw, I've been running benchmarks comparing the SSD block storage at Rackspace (http://www.rackspace.com/cloud/blockstorage/pricing/), as well as the all-SSD VMs from Digital Ocean (http://www.digitalocean.com/), and frankly, the CPU & I/O blow away EC2 so far. Compared to Google Compute Engine (https://cloud.google.com/pricing/compute-engine), CPU is on par, but obviously SSD is going to best even zippy local instance spindle storage.

For example, calculating 8th order Fermat prime factors (single thread/core intensive, so ymmv), EC2 XLs are getting smoked by 20-50% vs. Digital Ocean's 2 core 2GB RAM/40GB SSD. Replicated in multiple data centers/zones a dozen times. Not bad for $20/month!

Also, for the low end, remember "core" on EC2 is throttled at 50%-75% for the small & medium instances, translating to lots of stolen cpu time.

Still working out the details on the Rackspace SSD EBS-equivalents, but will post update soon.

But, agreed, given across-the-board I/O benefits of SSD, surprised Amazon isn't offering something in the non-cluster category. And at $2,500+ /mo, the new instances are definitely targeting a very niche use case for burst-heavy batching.

Reply on news.ycombinator.com