The way they use MySQL is also interesting IMHO: they populate a relational database in order to be able to build new indexes in the Redis side, using the relational DB for the stuff it is best at, generating new "views" of the data easily.
(Relational DBs are also good to do a zillion more things of course.)
I remember seeing YouTube storing videos using some sort of specialized hardware from NetApp or something and they still got hit by the limitation of Linux filesystem (number of nodes?)
Never did manage to see the final product though!
He said that he was confident that DB2 and Oracle could handle the load. They responded that SQL Server was much more capable than he thought and asked him about the workload.
Their response to his answer was "we can't even simulate that, maybe we're not ready".
I assume that they got better.
Though, oddly, they've been late to the game on social.
reviews, 'discreet' dating sites, sugar-daddy match-making sites, etc.
Porn does "social", but only anonymous social, which is naturally handicapped.
If someone created a social network which allowed better privacy controls, the porn sites would love it.
And I suppose one could argue that war is often just a manifestation of a desire for sex or power/money as an intermediary to sex.
Edit: Otherwise I don't see what the goal does to make the butchery of your fellow man any more or less obscene than it inherently is.
In Prometheus Rising RAW mentions that the reason monks take a vow of chastity is because it instantly frees them from one of the major hamster-wheels of the human condition.
Of course, I'm not planning to become a monk :) But food for thought...
The legal definition of obscenity requires that a work is meant to be arousing and has patently offensive sexual conduct, in addition to having no redeeming artistic value.
Therefore, adding sex into something offensive makes it more obscene.
http://www.wikisummaries.org/Prometheus_Rising
It fucked my mind when I read it :) It's a bit far out sometimes though, but it's a cool read.
It amused me that you were testing a porn site -- the homepage / potentially-misused zrange-issue.
* http://code.google.com/p/redis/wiki/Benchmarks
* http://redis.io/topics/benchmarks
I love how they use sInterStore as a replacement for a fat SQL query.
I suppose if each video's metadata causes a redis lookup and there are 300 videos displayed per page that would do it, but you'd think you could batch those lookups. It seems a fairly inefficient.
Either that or they have some background jobs causing redis lookups that aren't a result of page views.
There were 76 requests made as observed from the net tab in Firebug. The majority of these requests seemed to be for those video images. Note that I'm using an adblocker and refuse to check out the site without it.
Note also that they said 300k queries. There could be constantly-running background jobs doing conversion, pulling those preview frames, and crunching numbers on the metadata.
Tomorrow our site, with over 100 million page views per day, will be relaunching running on Symfony2. This is a complete rewrite. Prior to that it was running the Catalyst Engine written in Perl.