377 karma · joined November 6, 2010
We don't have a batch MapReduce problem and so Hadoop isn't really an option and, having used it in the past, not sure it's even a great choice.
Also, most of the data we're working with comes from content being shared and not number of users, so for our case in particular, dealing with a lot of data is an issue from day one.
Lastly, " solve the problem first and abstract out the framework later" I think is one of the core misconceptions in computer science and engineering. Abstraction lets you build faster if done correctly.
Again, if we only needed a simple key-value store and only had to deal with managing user data/events, we have built this kind of library out. Many of the reasons we built this library is because of the kinds of data aggregation we encounter in relevance ranking and machine learning.
I've talked with him about this choice and he has a lot of sound reasons for making this move backed by his years of experience teaching in various languages.
What's your line of thought? I use Clojure many hours every day and while I think that LISP languages are awesome, I don't think they're a good choice for a 1st programming course either.
They note in the paper this distinction, but it should be marked in the table, because I'm sure 99% of people don't see that. Sloppy!
You can see a demo of the site at condensr.com. There will be substantial changes to the site including predicting the best dish at a restaurant on the site soon. An academic paper should follow shortly.