HNHacker News
TopNewBestAskShowJobs

werner

2,767 karma · joined September 6, 2009

submissionscomments
werner··on Farewell EC2-Classic, it’s been swell
You are right, it was a typo. It should have been 10.0.0.0/8. Updated that in the blog, also resolved a confusion about the memory in a P3dn.24xlarge.

Thanks for your critical reading!

werner··on MXNet – Deep Learning Framework of Choice at AWS
Amazon has been building technology based on ML&DL for over 20 years and has developed several frameworks. You must have missed the announcement of this open source framework earlier in the year: https://github.com/amznlabs/amazon-dsstne.
werner··on Under the Hood of Amazon EC2 Container Service
Can you elaborate on the intra- inter-* latencies your are seeing? for AZ independent services this should single digits.
werner··on Under the Hood of Amazon EC2 Container Service
No mesos under the hood. You can bring mesos as your own scheduler.
werner··on Under the Hood of Amazon EC2 Container Service
You can choose eventual or fully consistent in DynamoDB. Given that full consistency comes at a higher cost (read from a quorum of replicas) we expose that cost to you.

BTW nobody wants eventual consistency, it is a fact of live among many trade-offs. I would rather not expose it but it comes with other advantages ...

werner··on Under the Hood of Amazon EC2 Container Service
Nope, we have built a new scheduler for you that will allow placement over multiple AZ's, replace failed containers, allow them to connect to ELB's, etc.
werner··on My run-in with unauthorised Litecoin mining on AWS
Luke, drop me a note at werner [at] amazon with a link to the support ticket you created, and we'll see what we can do.
werner··on Introducing Login and Pay with Amazon
Yep, I thought it would be of interest to the HN community, but most importantly I was interested about the comments/feedback/criticism of those who would potentially be using it.
werner··on Multi-Region Latency Based Routing now Available for AWS
And if you are looking for really cool jobs in which you can apply all your distributed systems and cloud skills on amazingly interesting products, beyond just video streaming, check out Amazon.com (75+ pages of 20 jobs per page in software development alone)...

http://www.amazon.com/gp/jobs/ref=j_sq_btn?keywords=&cat...

(sorry Jeremy, coudln't resist) :-)

werner··on Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS
5000 reads per sec of 64Kb items, would make you stream 2.5 Gbits/sec using consistent reads and 1Gbits/sec writes, moving close to 1.5TB each hour. At the end of the month you have read well over 800 TB and updates 160 TB... That is a substantial application you have in mind... :-)
werner··on Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS
noted. (both requests that is).
werner··on Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS
The amount of consumed read units by a query is not necessarily proportional to the # of items. It is equal to the cumulative size of processed items, rounded up to the next kilobyte increment. For example if you have a query returning 1,500 items of 64 bytes each, then you’ll consume 94 read units, not 1,500.
werner··on Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS
you can choose whether queries are consistent or eventually consistent.
werner··on Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS
It is on the requested features list
werner··on Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS
Can do me a favor and drop that in the AWS DynamoDB Forum so folks can look at it? https://forums.aws.amazon.com/forum.jspa?forumID=131
werner··on Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS
As with all the other AWS services you can have your limits lifted upon request.
werner··on No Server Required - Jekyll & Amazon S3
Actually I did do some coding for the conversion, etc. :-) But I like it when doing something new to be able to look at how other people solved similar problems. And it is a bit early for Cactus in that respect, and Liquid feels much simpler than Django templates.

The extension and plugin mechanisms will make it easier for me to start adding my own code without having to modify the core framework. But it is always more fun to add these kind of things if there is a community to give you feedback.

werner··on No Server Required - Jekyll & Amazon S3
Actually that was my fault. I had just switched off the redirect of allthingsdistributed.com to www.allthingsdistributed.com and as such you ended up at the old MT installation. That is now corrected.

You are correct; to map to an S3 bucket you need a CNAME. But DNS doesn't allow the apex to be a CNAME so you will need to redirect that. Route53 solves that for EC2 with the help of ELB. But there is no such solution for S3 (yet).

I am using the www subdomain as much as possible, so the redirect only happens if a visitor actually types in the apex name, in all other cases they will get where they need to be directly. But I agree that it would be better to solve this at a different level.

Launch and then iterate...

werner··on AWS drops bandwidth pricing
My blogpost w background information: http://wv.ly/iLDaqu
werner··on Faults in Linux: Ten Years Later
The data that goes with the research can be found at: http://faultlinux.lip6.fr/
werner··on Just Making Money - A Compilation of DHH at Startup School 2008
For a presentation I needed an additional voice to convince the audience that building a sustainable business was preferable over striving for a massive exit. A short compilation of David Heinemeier Hansson talk at 2008 Startup School did the trick. With all the increased bubble talk it is worth watching again. You can find the full talk at http://www.justin.tv/hackertv/b/259414909
werner··on New AWS feature: Run your website from Amazon S3
Absolutely. Next steps already defined :-)
werner··on New AWS feature: Run your website from Amazon S3
Actually you could run wordpress with WP Super Cache enabled to generate static files.
werner··on New AWS feature: Run your website from Amazon S3
Moveable Type has support for multi-site deployment through rsync (see the MT docs on rsync target, etc). After that the job queue finished MT will call rsync to upload the files to remote locations.

Didn't take that much work to make it work with rs3sync instead (see http://rs3sync.net).

werner··on The Incredible Power of the Amazon EC2 Cluster GPU Instances
You guys are important customers...
werner··on Dynamo: A flawed architecture
Dynamo is still in use, but as with all technologies that have to operate at Amazon's scale, the systems evolve rapidly. The storage systems in use now no longer look like the ones from 5 years ago when Dynamo was developed.

The Dynamo SOSP paper had two goals: 1) show how systems are a composition of techniques and how all of these need to work together build a production system 2) given that it was based on a variety of research results it was intended to give feedback to the academic community about the difficulties of moving from research results to production, and what matters in real-life vs production.

The paper was never intended to be a complete blueprint for easy design of follow-up systems.There just isn't enough room in an academic paper to do justice to that. If it was going to have any role in that the best we could hope for was as a collection of points you would have to think hard about and make decisions about when you were going to design your own storage engine.

That doesn't means I agree with the conclusions of the analysis, on the contrary, I think they are seriously flawed as well (as with everything else that is not absolutely perfect :-)). But I can understand that when people look for the dynamo paper to be a blueprint that solves all their storage needs and provides a perfect available service under all failure scenarios (and solves world peace), that they may be left with a few questions afterwards.

I always thought that the real contribution of the paper was that it made you think hard about the trade-offs you are faced with when you have to design high-available, ultra-scalable systems that are cost-effective and provide guaranteed performance. 5 years later we know a lot more but this stuff is still hard, and we still need to balance rigorous principles with production magic to make it work. But you do need to fully understand the principles before you can make the production trade offs.

Caveat: some of these remarks were tongue-in-cheek fun; I leave it to the reader which ones :-)

werner··on Radio's Regulatory Roadblocks - How the FCC slows new wireless technologies
This might help: http://www.ntia.doc.gov/osmhome/allochrt.pdf