A performance test of SQS | Hacker News Reader

A performance test of SQS

litt.no

12 pointssorenso9 comments

There's some confusion in this post about the testing methodology vs. the implementation of SQS itself.

SQS itself may well be implemented on EC2 instances with auto-scaling (though I doubt it), but that's irrelevant to the test harness, which needs to scale out the number of nodes pounding on SQS.

The important bit is that SQS doesn't exhibit any of the scaling bumpiness of ASGs. It's a region-wide service, which you'd be very lucky to break on your own.

Thanks for commenting!

I'll be happy to answer any question about my implementation and testing.

In this test I'm using a standard SQS service i region EU-west. My integration with SQS is done from a simple application written in Node.js. My application is put inside an autoscaling group which scales up when the load on the servers is above my threshold. At the end of the last test I was pushing messages from 18 different servers at the same time. The request rate was above 6K/sec but the insert rate into the SQS queue only managed to peak aroud 2.3K message insert/sec.

I agree that this is strange. But I've also experienced strange stuff with Amazon ELB: http://litt.no/tech%20stuff/amazon-elb-latency-problems

Best regards, -sorenso-

In our experience of ELB/SQS, AWS has done a good job on these services. A few things we noted:

- ELB takes about ~15 minutes to scale out completely when you hit traffic spikes.

- SQS is simple to use, rather cost-effective and quite reliable. It will add a bit of latency, but as the author notes, this can't replace Kinesis.

- The redrive policy on SQS is really nice to catch corner cases.

Thanks for commenting!

What is your traffic rate in request pr second? I would be interesting to know.

SQS is a simple and easy to use service. I really like it, but as you read in my post. I'm a bit concerned about the scaling. We are expecting to reach a peak traffic of 3K req/sec next year.

Best regards, -sorenso-

> It seems to have a problem with handling more than 2.3K message insert/sec.

We've written to SQS at rates several times this using only 3 EC2 nodes without issue.

The only issue we have seen is occasional and sporadic latency issues reading from SQS which causes the queues to backup a bit but it generally clears within 30 minutes or so.

Hi mbell,

Thanks for commenting!

What kind of application are you using with the SQS (language, purpose, usage)? It would be interesting to know for me :)

This test was only running for 50 minutes, maybe this could be the issue?

Best regards, -sorenso-

We use SQS as a buffer at the front of an analytics system. The enqueuing app is written in Java using Dropwizard and Apache Camel. Average message size is probably ~1KB.