Ask HN: Javascript widget scaling issues
Our widgets are receiving around 50/60 requests every second. These include both widgets loaded for the first time, and also subsequent requests where the widget asks for new content. We are running our database on a large instance using Amazon EC2. The database is not the problem, however. We recently moved the code that serves widget content onto a dedicated small server instance. Immediately traffic has been phenomenal and CPU usage is very high right now.
We operate under a LAMP environment using PHP as our server-side language and MySQL as our database platform. We take advantage of memcached by storing the content for 30 seconds at a time (this makes sense because our widget refreshes every 30 seconds). This means that on most occasions the content that is served comes directly from memory (with 0 queries being executed). After a request is sent, we deliver back HTML content. We know that it would be smarter to return simple JSON containing only the newest updates. This is something we will implement.
It seems that with 50/60 requests coming in every second, even though the majority of the content never makes a request to the Database, the sheer volume of traffic is what is killing the CPU.
Do you have any suggestions on how to handle this smart?
Move to a Large instant on Amazon to serve widget content? Scale horizontally with small instances and load balance them on Amazon? Could PubSubHubBub be used to push new content to the widgets? I would love more information. Is memcached having any effect on CPU - since there is such high traffic? Should the widget be served using Java?
Thank you ver much!