The concept is a live interactive system with 100,000+ members online, logged in, and actively posting for about an hour at a pre-determined time of the day (or night). Each person may post 2-5 times a minute and this can add up to 1/2 million hits a minute, but I said I need a million hit per minute capacity because it is possible that 200,000 people might be participating in some of these events. This is a high-profile venture and "bogging down" would be an undesirable occurrence in the middle of an event.
Each post from a logged in member will contain a value which typically consists of no more than 5-10 characters. A cookie of 5 characters will accompany the posted data to uniquely identify the visitor. When the server receives this data it will append a new record to the database with the post value in one field, the login cookie in another, a date/time stamp in the third field, and an incremented value in the fourth field to identify the order in which each post is received. I think that's all the data that needs to be appended to the db in each new record.
(Note that this is my current concept of the database schema, if you folks know a better way to do it please tell me, thanks.)
After the new record is appended, a search is performed and the post field is sorted, then the relative position of the visitor's new database record is identified in the results (example 49,501 records from the top). This 'position' information is used by the script that dynamically generates the HTML, which is then passed off to one of the HTTP servers so it can return the page to the visitor's browser.
Since the returned data must be created dynamically based on the most recent database update, it appears that the database itself may be the limiting factor here. I can equip the database server with SSD drives to accelerate its performance but this still may not be fast enough. Plus there may be an issue with database size itself since it will grow extremely rapidly with 16,000 new records appended every second for an entire hour ...
I have not calculated the size of such a database after an hour of such appends, and I don't know whether existing SSD drives have the capacity needed to store the volume of data appended at this rate, or even if today's database systems can use multiple SSD drives if more capacity is needed. These are issues I may have to deal with if something else does not come up first that makes the whole concept look absolutely impossible.
And yes, I am absolutely going to need this capacity from the start. This is not a system I can "grow into", it must be capable of this performance from the very beginning. Assuming that such a system can be created its cost may not pose a problem since the concept will be very attractive to investors based on the huge revenues it can potentially generate. If there is no practical solution at this time I'll have to re-think the entire concept and try to come up with an alternative that works within today's server / database / page rendering limitations. I'm just hoping there's an existing solution so I don't have to change the concept because as-is it is unique and has unbelievably huge upside potential.