Thanks Omar! Definitely couldn't have done it without all the support from the local community and friends - Waterloo is an awesome place.
5,399 karma · joined April 2, 2007
[ my public key: https://keybase.io/igrigorik; my proof: https://keybase.io/igrigorik/sigs/eMcFb-lAFqkBGZKfdzJcQrC5a2dTBj_8cTAQqsXUhaE ]
Super excited to join Google and take it to the next level!
Now if only TCP was message oriented... For that, I guess we have ZMQ.
Alternatively, there is work on sending an "upgrade" header in your regular HTTP response: http://code.google.com/p/chromium/issues/detail?id=69688
The NPN route is obviously the best in terms of performance, since the protocol can be negotiated as part of the TLS handshake..
Long story short: you can definitely run your own SPDY server and Chrome will auto-detect and use the protocol.
Rubinius is a little bit further behind, but I've been able to run our Goliath stack on it, and that exercises quiet a few syntactic changes. So, I think both are close.
Now.. How you actually achieve that is a whole different story. You could, in theory, throw a job into some external work queue and poll that, or if your runtime permits, spawn some threadpool and periodically check that, or.. spawn a process and wait on that. In other words, it all depends on the actual operation.
In the case of PDF generation, if you rely on some external tool, you could use a mechanism like EM.system('shell cmd') to spawn a process and wait for it to return you the data.
As far as middleware goes, because you are reusing the same "app chain" between multiple requests, you just have to make sure that your middleware does not rely on any instance variables since those will get clobbered by other requests. Check the wiki page on middleware, we have a few examples around this.
Last but not least.. the EM reactor runs on a single core, so you're basically in the same deployment scenario as Thin or node.js. Having said that (always a caveat! ;)), Goliath can run on JRuby and Rubinius.. so in theory we have non-GILed environments there, which means we can start multiple reactors and run across multiple cores from within the same proc. This is not something I've experimented with in practice yet, but in theory its possible.
Thin would be an alternative to Goliath as well, although we chose to switch to a different parser and also to add the Fiber logic/wrappers right into the framework.
The way I think about it is: if Thin is an app server, then Goliath is more of a minimal framework which you can use to go from start to finish in if you need to bring up an API endpoint. That includes, configuration, routing if you need it, validation, etc.
ruby-1.9.2-p136 :011 > /My name is (?<first>\w+) (?<last>\w+)/ =~ "My name is Ilya Grigorik"
ruby-1.9.2-p136 :012 > first
"Ilya"
ruby-1.9.2-p136 :013 > last
"Grigorik"
Only one wrinkle: you have to submit your own challenge to see the solutions.
If anyone is up for it, would love some help with the client/parser: https://github.com/igrigorik/vimgolf
Short of going into a rant on the topic, please do check out: http://www.igvita.com/2010/06/24/weak-consistency-and-cap-im...
As the narrator says, this is not an usual pattern for people to migrate from one thing to next. And if that's the case, then chances are there are many many more people which may end up never finding "that calling". In a way, the above story illustrates the anti "pure hill-climbing" approach: the initial assumption/premise is that you should find a hill and then go really, really far in hopes of making a big contribution. Whereas Ed Witten's story shows that picking one single hill is not necessarily a good idea (local minima!) ... In fact, every once in a while (or systematically), you could jump to a different hill and still come out as a winner. Now the real puzzle is:
a) You could make the argument that your chances of finding the optimal hill for you, on the first try, are very slim. Hence, you should jump at least a few hills.. i.e, your real aptitude lies somewhere else.
b) Jumping too much will/may cause not enough time with anyone subject to make significant contribution - hence it's an optimization problem.
c) The "cross-discipline" question: its often acknowledged that bridging different knowledge areas is very helpful in terms of making interesting discoveries. Does that, in turn, mean that you should jump at least a few hills? If so, how many, and how much time?
Fun questions to ponder... Clearly there is no black and white answer either! Depends on your field, context, etc.
As for conclusion, I think I'm going to have to make another go at it in another post.. :-)
Just for sake of completeness, you can basically copy-paste that code into Ruby as well via em-websocket: http://github.com/igrigorik/em-websocket
Also, to avoid some of the callback muck that a few have brought up: http://github.com/igrigorik/em-synchrony
(same idea, closures + coroutines to abstract the callbacks).
That's not a benchmark, it's a proof of concept: http://github.com/igrigorik/async-rails/blob/master/app/cont...
/widgets/http makes an http request to /widgets, which does a 1s sleep query inside of mysql. All of this is happening on the same reactor loop, and finishes with ~2s, aka, the sleeps are not blocking the reactor and hence the "async stack".
It's a start.
As for Goliath - we will. Time is the scarcest resource in a startup, and that's why haven't opened it up already. We need to get some documentation in place, and remove a few dependencies.
Point is though, there is nothing inherently novel in our stack! All off the shelf pieces: thin, async-rank, async middleware, em drivers + em-synchrony.
Hmm, not at all. A message queue and a job queue are not necessarily one and the same. What we need is real-time scheduling, with up to the second resolution for each job. We don't run a batch "go fetch all of these pages" jobs, rather our system is always running, always fetching content. For that, a heap/work queue is required.
It seems like the primary concern for Domas (presenter) is concurrency within InnoDB. Just to get it out of the way, he dismissed MyISAM, and went right into the low level optimizations & tweaks they're working on at Facebook. High level points are:
- they're using XFS, and he stressed understanding the block math and how it relates to your record sizes, etc.
- using "correct" scheduler has a big impact
- swappiness = 0, etc.
For InnoDB itself, it seems like they're carving out entire chunks of functionality that he claims "never worked correctly" anyway. For example, deadlock detection is something they disabled outright because it gave them a large perf boost. Likewise, for read-ahead functionality within InnoDB. Those were the two he went into a lot of details.
Concurrency + threads: there is no definitive formula, but he suggested setting thread concurrency to be the min concurrency of your lightweight requests (claimed: threads are cheap).
For ops / day-to-day, they focus on slow servers and dive right in: custom tools + gdb + freeze the server. They are playing with the idea of providing stored procedures interface to app developers since it'll avoid the round trip time between different queries (TBD). Also, they sometimes go very aggressive (timeouts of 3s, or even 1s), to force internal app devs to respect the DB layer.
Also hinted that they're using "smart caches" which are capable of buffering counters / updates and doing one update instead.
All in all, their biggest problems seems to be thousands of (app) servers that are always connected, hence the need for high concurrency...
(I'm sure I'm forgetting a whole lot, but that's off the top of my head)