How I Got Stung By Amazon EC2
For those of you who are interested, Social Crawlytics is based on the following technology:
1. Redis 2. FuelPHP 3. Node 4. Pubnub 5. MySQL
To cut the chase, within a couple of weeks I discovered that my pet project was attracting a lot of attention and out of nowhere the app was processing hundreds of large reports a day - which meant we needed more workers. So we developed the app pretty much based on Amazon's AMIs, every time there was a huge spike the app would auto launch new instances and instantiate workers until the spike was over and then it would terminate the extra instances.
This was obviously an expensive decision because we had instantly moved from a "free tier" to basing pretty much most of the app on Amazon's infrastructure. I knew the costs would add up but I sought solace in setting up Alerts so every time a certain threshold was hit I would get an alert. Turns out that Amazon's alert had a bug and and so did Social Crawlytics, basically in a matter of hours we clocked up over £640 bill and I wasn't alerted by Amazon at all.
Luckily, I just logged in to AWS and when I saw the cost I almost had a minor attack. So why did I share this here? Simple! If you are on Amazon watch your account like a hawk, you just never know what the costs would add up to and if you have a bug, it could cost you thousands in matter of hours.
I hope this will help someone somewhere.