Short Chat Server in Clojure
kirindave.tumblr.com
kirindave.tumblr.com
I didn't see this one in my reader, though the author is a member: http://news.ycombinator.com/user?id=KirinDave
The problem I've got with posting stuff from 'the same feeds' is that it tends to drown out the non-mainstream (and non techcrunch ;)) stuff, which is what really makes HN interesting.
Between all the spam posts and the 'feed pushers' a lot of really good content never makes it to the home page and so is lost to a great many readers.
Personally I've switched to the 'new' page as the homepage of HN long ago, but even there if you look the other way for one or two days it's hard to catch up because of all the fluff.
I too despair over cool stuff sliding off the 'new' page into oblivion -- the only surefire way for an article to make it to the front page is for multiple people to submit the same URL, which can get rather unlikely for the most valuable posts. I usually try to wait until a reasonable PST hour to submit odd things, just to maximize the chance that clueful users will have the chance to give it those first few upvotes.
I do think some countermeasures should be added to news.arc to handle the mainstream spam-stream. I think a good first step would be good to stop counting link-karma towards user totals, but I don't have a clear idea of what else would help.
The scala one is a webapp that is sitting on top of a lot of specialized machinery provided by lift.
They are both short chat applications in the new cool languages on the JVM, and both are neat code samples, but other than that they don't have much in common.
Perhaps you could write a basic chat server in Scala to show us? Really, I found this exercise quite fun. The programmer's equivalent of cotton candy.
http://dhotson.tumblr.com/post/271733389/a-simple-chat-serve...
Any difference in functionality?
The data model is slightly different, too. Clojure has very powerful concurrency-safe data structures which made my data structure slightly more obvious. It's just a hash table of lists, and the lists contain tuples of [Name, StringWriter].
If you know Javascript and you don't know Clojure, it's probably natural your eyes follow that example more easily. I wouldn't say either version is particularly more convoluted than the other (although it's obvious we both made some decisions that favored brevity over readability).
Mainly just a curiosity question, I realize not many people will ever have to deal with thousands of concurrent people chatting.
I'm probably going to extend this exercise by switching from the default contrib library networking and standard java.io, and instead use netty. I'm told that can give me superior performance and scalability, and I don't think it'd take much more code. But the rest of the code should be fine. I'm quite certain, from profiling, that the core data structures can scale to thousands (and even millions) of users. Clojure's refs and agents are extremely well-implemented.
But, I'm not sure the node.js version really would scale to thousands of users. Every time someone connected hits enter the entire user set is looped over, synchronously. I suspect this latency will add up. The reason that code is so simple and direct is that everyone is just executing slices in one thread. In my version, the rooms have a thread pool (implicitly), so it really comes down to how many user threads java can allow.
There are reports on the mailing of broadcast echo tests (which are basically the same thing) running 10k-20k concurrent users. The main problem seems to be running into OS limits for the number of open ports.
I am fairly certain that the architecture USING the evented architecture wouldn't scale though.
In general, the single threaded evented model tends to scale better than the process/thread per connection model for large numbers of connections. Having to context switch between 10,000 threads/processes typically means the OS spends a lot of time doing context switching instead of IO.
I'm not sure how Clojure implements agents, it may already handle having 10,000 concurrent agents efficiently. I'd be curious to know.
Also, there's a great article about network programming, which you may have read. It's called "The C10K Problem", which talks about scaling to 10,000 concurrent connections.
http://www.kegel.com/c10k.html
.. really good article, well worth a read if you haven't already.
Right, the evented model works, and you get that for free with Node.js. I dunno if that specific codebase would handle "thousands of users" though. Test it, it'd make a cool blogpost. :)
> I'm not sure how Clojure implements agents, it may already handle having 10,000 concurrent agents efficiently. I'd be curious to know.
Well right now it's 1 thread per connection, which I will have to change. As for agents, they dispatch on a threadpool with a number of threads corresponding to the number of cors in the physical hardware. So that part of the code should scale of fairly arbitrarily (although a specific room with many, many people may experience some latency, and many such rooms might bog down the thread pool).
The only reason I didn't was that the original node.js was usable right from netcat or telnet without prefacing anything with /say or any such command. If I had multiple rooms, a /say <targetroom> <message> would have been required and I didn't think that was very fun to type.