Show HN: Comet server w/end project, barely breaks a sweat at 1000msg/s | Hacker News Reader

Show HN: Comet server w/end project, barely breaks a sweat at 1000msg/s

splinter.com.au

13 pointschubs10 comments

How is it good to have 1million queued messages but only utilize 5% of the cpu?

I think the argument is that queuing should be cheap and the consumers should actually use the CPU. He has no consumers yet, he just fires messages like crazy and they are left pending.

That leads me to the question what this benchmark is good for.. Someone tried to measure ~something~, but it's useless right now and targets localhost.

I'm not impressed.

Thanks for having a look! The main thing i wanted to check for as of now is the performance (in terms of memory usage) of the hash/dictionary/list library i'm using (klib). In which case, i'm personally impressed with the work of klib's author (who, as it turns out, has chosen to be anonymous!)

What led you to klib? Looks interesting . . . given the fasta/fastq format parser in the source code base, we can guess that super high performance/low memory footprint development is pretty critical for the klib author.

The macro-oriented dev stuff is kind of new to me, but I'm sadly years away from writing C.

I googled for C hash libraries, and came across one of those 'shootout' blog posts where they compare memory size and cpu usage, and klib looked to be a winner. Plus it's only one or two header files to make it work. Documentation was poor, so i contributed some. So i'm happy with my choice. Yes, the macro-oriented stuff is a headache, but the raw speed it gives you is worth it. And don't think you're years away from C! I work in C# and javascript mainly (and recently, obj-c) so, for me, this is as much a learning exercise as anything.

You get big props from me for

- replying with further details

and

- responding in a very friendly way although my tone was neither constructive nor friendly.

Thank you for that great response.

Holy moly! What a turnaround. You're welcome :)

Those are orthogonal stats, true. More relevant is queued messages vs RAM used, which i'm quite happy with: 1m messages = 65m ram. Also think about the 1000 msgs/sec vs 5% cpu, which i'm also happy with. Thanks a lot for having a look!

For a long time I thought there was a limit of 65535 concurrent connections, but I learnt today that it is for local ports only. Now I'm confused. Doesn't accept() create a local port?

Yes, but the limit is: 65k only for a single incoming IP address. So 1m is fine, presuming the clients all have different IPs. So, for testing, i'm going to have to do 1m/65k = 16 servers running concurrently at 64k connections each.

Reply on news.ycombinator.com