Mongrel2 | Hacker News Reader

Mongrel2

mongrel2.org

170 pointsrams80 comments

Usually when people announce a product and give you little else but a future feature list, I'm sceptical.

Not with Zed, though.

This is the exact sentiment that makes me respect the guy. He's an asshole and loves to complain REALLY LOUD to eat up the attention.

But he's dead sincere when it comes to making things. He's like _why's evil twin.

Agreed, but for one thing. He IS NOT an asshole, not in real life. He's actually super-nice. Online he has the asshole persona, but that's it.

The Internet is real life.

I'm with petercooper. If you act differently online, you're still responsible. His recent attack on Giles Bowkett was an assholey thing to do even before considering whether either one of them had a point.

That said, I'm not too worried considering his history with the Ruby community. I'd still have a better opinion of him if he didn't antagonize them though.

I'm absolutely certain he's a great guy in person. Asshole doesn't preclude you from being nice, personable, and even entertaining. Instead I take it to mean that when the bad side does come out it's nasty.

Giles Bowkett has never shrunk from attacking others (like his blog post "Chad Fowler is a Vampire", which was a long personal attack against Chad Fowler that Bowkett seems to have since deleted) so it doesn't seem that asshole-y to treat Bowkett as he treats others. I once participated in a thread in which Bowkett swore at and belittled a number of participants who criticized a blog post of his (he subsequently deleted his postings so the recipients wouldn't be able to respond).

You get what you give. I tried for a long time being nice and gentle online and I've found that there's a class of hackers who act like bullies anyway. They'll slander you, try to ruin your job prospects, lie about your personal life, everything you can imagine. Seen it first hand done to myself and to others and it's stupid.

It's ridiculous, but it seems the best way to keep this in check is to be kind to the gentle people, and totally destroy the haters.

Also, it's pretty fun tearing into these guys because they're usually pretty bad at their rants. :-)

I find it bizarre that you can come onto a forum like HN and insult me, call me an asshole, and effectively slander me and that's quite alright. You're totally not an asshole at all.

But if I out-rant a dude who compares me to Lady Gaga and tries to kick me when I'm down for no other reason than to drive traffic to his poorly written blog and sell more videos, then I'm an asshole?

You have the same double standard all geeks have. You let the blow hards, pundits, and bullies get away with treating other people poorly, lie, and hype themselves endlessly, but when someone calls them on it you call them an asshole or worse.

You're right. It's easy to talk about people as though they're not people online, even when you're secretly trying to praise them. I'm usually finding myself agreeing with you when reading things like that and even loving the vitriol while wincing.

So I apologize for writing timidly previously. I apologize for standing behind your back and spreading my opinion in an unbalanced fashion. I made the classic mistake of forgetting I'm actually in public.

I'll say it to you directly. Zed, I'm glad you stand up for yourself, both against my clumsy commentary and all the other blowhards who strike at you. I'm even glad you do it so vocally because it is a damn good story: the wronged who doesn't go quietly into the night.

The only negative side is that your voice is corrosive and bitter and if it weren't I'd feel a lot more comfortable about agreeing with you. As it is I feel like I'm nodding along as the bully takes out that annoying kid who really just had it coming. It feels good, but society is built on not just sitting and watching that happen. Worse, I hate that I end up remembering that feeling whenever I read about any of the cool things you do.

So yeah, I still think you're an asshole. It's unfortunately not what's fair or what's owed but instead how it looks. I'm sure you're ready to tell me why I'm wrong, but don't worry because I've already got a truckload of cognitive dissonance going on. It doesn't help that I think Fret War and Shedding Bikes and Lamson and Mongrel are all great projects and am completely sure that you're a great friend to those who earn your respect.

I just see it as the image you've intentionally built for yourself. Starting with ZSFA and now eking into oppugn.us.

Zed's writing has generally struck me as being pure larrikinism (especially under the banner of ZSFA). Unfortunately I'm not sure that there is an analogue for larrikinsim in American culture and it's not a concept I can easily explain; there is a subtlety to it that makes it both difficult to convey and difficult to execute. If you Google it you might be able get an idea of what it's about and perhaps get a different perspective on Zed's writing.

You can download the code by logging in and visiting the 'timeline' http://mongrel2.com/info/2d2129a2db

I do sometimes just throw an idea out there to see if there's interest. No point wasting time these days if nobody gets it.

In this case, I felt like reprising Mongrel and finally that it would make sense, so I whipped up some code, got it working, and made a viable chat demo with it:

http://mongrel2.org/chat/

Based on that, seems folks get it and it'll work so I'll start hacking on it.

I'm surprised that he's doing this... after so much vitriol about the Ruby (and Rails) community and being "ripped off" with Mongrel, I would have expected him to never touch it again. Granted, it's written in C, but I'd expect it to be heavily used with Rails.

That said, its good that he is writing it. The more software is written, the higher the likelihood that _something_ will be great, and when Zed Shaw's writing it, it's probably pretty good.

His primary use case is different for mongrel2 compared to mongrel however its based off the same HTTP parser and is a webserver so I guess the continuation seemed appropriate. Being completely written in C its going to be blindingly fast. I have a feeling it'll surge past unicorn, thin, passenger and everything else out there once its refactored and past beta.

Don't you think AGPL licensing will slow adoption? Nobody seems to be able to clearly define what you have to open source if you use AGPL'ed code.

I think that you simply need to buy an alternative license. This is also the way neo4j is monetized.

AFIK you just have to publish the source to the changes you made to mongrel2 itself if your server is public.

And mongrel2 is not embedded. You connect to it. No linking. It's a server.

Unicorn, Thin, and Passenger are all mainly written with C/C++ as far as I know.

Unicorn and thin are both written in ruby with the core being the ragel HTTP parser which was first implemented in mongrel. Thin took that ragel parser and added eventmachine to the mix which made it much faster the mongrel. Unicorn went a step further by leveraging the kernel in such a way that things got even faster. Macournoyer, creator of thin was working on thin-turbo, a rewrite of the thin backend in C but stopped a while back. Passenger I guess because its nginx or apache module has to be C but their focus was never performance, more ease of deployment. Passenger 3 will have performance improvements and they've got benchmarks here http://blog.phusion.nl/2010/06/10/the-road-to-passenger-3-te... but I'm thinking mongrel2 will one up it.

The advantage of Passenger is that it takes care of dynamically allocating instances of your app, adding them and removing them as needs be. I don't think Mongrel ever did this: you had to guess at how many instances you might need and hope for the best. (At least afaik - no one has ever corrected me when I've said that in the past).

You're correct, Mongrel had predetermined number of instances.

I actually wouldn't compete with those servers, because they are very language specific. I'm aiming for Mongrel2 to unify all the languages in the network architecture so that you can do what companies actually do: use what works.

I could easily see Passenger or others providing a 0mq connector that Mongrel2 can talk to. Passenger is bad ass at running Ruby, so no point in trying to do that better.

I'm not a Ruby guy, so this might be a stupid question, but: why would you need to run more than a single instance of an app on a given server?

That just refers to running multiple processes instead of multiple threads. Rails wasn't thread safe until recently and many ruby libraries still aren't (including the mysql driver). So it's common practice to run a "pack of mongrels" (a few processes simultaneously) and then load balance between them in case a long request ties one up for a while.

Mongrel2 will let you effectively design part of your app to benefit from Rails (for example) and part to use 0mq and do a more evented style of design. All within one configuration / approach.

Ruby 1.8 uses green threads, so running multiple processes lets you use all your cores.

As mentioned in another post, Rails used to not be thread safe. Right now I'm running a Rails site in production with multithreading enabled, but I'm sure many libraries in Ruby are not thread safe.

I don't understand the benefit of dynamically allocating instances. Why should you be running fewer instance than your server can support?

Because you have multiple applications.

How do you guard against the situation in which both applications require the maximum number of workers?

You can't, obviously, any more than you can do much more than monitoring and jumping in with a new server instance if your site suddenly gets 1000 times the traffic it normally does.

It doesn't takes 1000 times the traffic to max out your workers. Imagine if your database (or some other dependency) had abnormally long response times for a minute.

I would think that by spinning up more workers than you can support, you slow down all responses. This increases the rate of backup, which increases the need for those extra workers, which slow things down, etc.

By keeping workers below the ceiling at all times, requests back up, but you don't degrade performance. I think this makes things much easier to recover.

Correct me if I'm wrong about your scenario, since I've never deployed it on a high traffic site.

Oh, well you certainly keep the total number of workers below a maximum.

All I'm saying is that I have N sites that get different amounts of traffic. If more people are hitting one, I want it to get allocated more passenger instances to handle the traffic. With mongrel, I'd have to, a priori, decide how many mongrels each site gets and hope for the best. With passenger, it dynamically allocates them to handle traffic on an ongoing basis.

Also: notice the AGPL license. If mongrel2 hits a sweet spot for the targeted types of web apps, Zed has a good chance of monetizing this project. I appreciate Zed's writing and work - so good luck to him.

The use of AGPL has some interesting ramifications. For example, I still don't understand why AGPL projects like MongoDB can be used as part of large networked systems and this not be a violation of the AGPL if the entire system's source is not released under AGPL also. AGPL is "networked viral."

Straight from the horse's mouth (http://blog.mongodb.org/post/103832439/the-agpl):

"To say this another way: if you modify the core database source code, the goal is that you have to contribute those modifications back to the community.

Note however that it is NOT required that applications using mongo be published. The copyleft applies only to the mongod and mongos database programs. This is why Mongo DB drivers are all licensed under an Apache license. You application, even though it talks to the database, is a separate program and “work”."

Thanks. I had read that also a while back, so this sounds like the MongoDB developers offer a sort of waiver to the straight-up AGPL, effectively making it the GPL. In any case, kudos to the MongoDB developers; I have used it on almost every project I have done this year.

Its a little different than the GPL. If you modify some GPL server code, and open that server up to "public" connections, you don't have to contribute back your changes, as long as you don't distribute the server itself.

With the AGPL, however, you do. Some people, though, interpret that to also mean that any of your code that connects to an AGPL server must also be open-sourced; eg, you web app, if it connects to mongodb, you must provide the source. As far as I know, there's been no legal opinion if thats the case, and the mongodb devs have stated thats not how they read it, and thats not what they intend.

I don't think anyone could argue that if you had an AGPL HTTP server that the AGPL would be applicable to browsers that connected to the server. So if the Mongo protocol is presented as an open protocol it seems unambiguous. Though if someone created an "open" protocol to someone else's AGPL code with the clear intention of bypassing the AGPL... it might get weird and awkward.

Right, but I don't think anyone is arguing that.

Its when you modify the AGPL server code, and open that up, you have to release your modification.

So in you AGPL HTTP server example, the clients never are required to be open, but any changes you make to the server must have the source available.

This is a very cool idea. I think it's actually more along the lines of nanite or fuzed (ironically) than mongrel1.

I think Zed's approach is clever... to create a middleware layer that can intelligently (and without adding tons of config complexity) let people build fast, complex apps that fulfill requests using a variety of backends.

From the description ("… designed for web applications over serving files") it seems that you still need a web server as proxy in front of it for mid-sized projects; to serve static files. (Bigger projects use a server/CDN just for static files.)

It'd be able to serve files just like normal, but at large scales smart companies use a big CDN and don't do their own file serving. The economies of scale are just not there to do it yourself.

Mongrel2's advantage is its focus on apps. Combining HTTP, jssocket, and websockets seamlessly on one port means you can get the best of req/response and async operations in your GUIs.

He's been an advocate of building stuff over being a language snob.

I remember reading in his book teaching programming to beginners that the important thing is to build stuff, not to nerd out about language features.

That's because I find programming languages are marketed similar to religions and work more like Ponzi Schemes. My goal is to change the culture so that programmers start to think this "community/religion" attitude is weird.

Basically, I'm a linguistic secularist. :-)

Mongrel2 will be very language agnostic, in the same way that Mongrel was very "Ruby Framwork Agnostic". My experience there was that making the server fairly implemented for all created a boom in server frameworks. I'm looking for the same result with Mongrel2, but with many different languages.

Zed is a busy guy!

I like that the configuration language is a sqlite database; sounds far easier than parsing ini files, for example.

ini files are

1. a straw man. The Format is very generic, only sections and key-value-pairs. A fairer comparison would be against httpd.conf, or other languages actually used for web server config.

2. Easy to parse, because there are tony of implementations already.

3. Static, like a SQL database, although Zed[1] and others[2] argued we need Turing-complete extension and configuration languages like Ruby.

[1] http://vimeo.com/2723800 [2] http://lambda-the-ultimate.org/node/3606

An even fairer example would be ngnix.conf

He says that a config file is half-turing, but an sqlite database can have no turing. Won't this make the configuration far more complex?

Just wait till someone uses an ORM library for configuration.

But how would one represent a configuration like "don't require auth for /, but do require auth for /foo/*.jpeg" in a table? It would be great to just use a row for each possible URL, but with overlapping wildcards, folder hierarchies and multiple levels of allow/deny it gets very complicated very quickly. Granted, the allow/deny and order that Apache uses isn't great, but at least a configuration is understandable by reading it because the admin can lay out the configuration file with whitespace and comments in a way that makes sense. The equivalent might be achievable in sqlite, but it would be very challenging. It's basically a step backwards towards a poorly featured, confusing GUI configuration mechanism like the old IIS configuration used to be.

well you could always just represent it as:

insert into rules (rule) values('don\'t require auth for /, but do require auth for /foo/*.jpg')

The use of a db is more to create a standard networkable interface for other clients to read/modify the config than a statement about the semantics of the config syntax.

You could do that, but would it really be much different from this?

data = config_file.read()

"insert into config (config) values ('%s')" % data

> But how would one represent a configuration like "don't require auth for /, but do require auth for /foo/.jpeg" in a table?*

Are you confusing offline representation with a runtime structure?

I was only being half serious (or half joking, depending on how you look at it).

I don't like that idea, because of one simple limit. You cannot diff databases (easily). Of course you can dump them, make sure the linebreaks are there between records, make sure they're sorted, etc. etc. But checking the whole config in or out of a VCS is much more useful in general. It also provides an easy to browse record of changes.

If diffing is important, why not use Git as the repository back-end, then?

It doesn't matter which VCS. If you check in text, you can review the changes. If you checkin databases, all you get is a blob. Git is not magic and will not solve this problem.

The sqlite database might get stored as a blob, but a json, xml, or sql dump of the database would be just as plain text as anything else.

Think of it like this. Zed is combining Apache's nonstandard configuration language with it's internal runtime representation of such. He's doing it in a way that's standard and manipulable at runtime. So changes to the runtime configuration are persistent. You can fix that. That's the beauty of it. It's an incredibly simple and flexible idea that builds upon existing, robust, and standard software. If you want to diff config files, then you could slap together an XML, JSON, or any other representation of your config data and write a loader for it.

Isn't reading from sqlite all the time a bit bad for the performance?

Wouldn't the server only read it after a config option has been changed? I don't think the server is going to check this database every time a request comes in.

adamtj said "manipulable at runtime", so I guess he meant the changes are available directly after an UPDATE.

How can you tell if the config is changed? You either poll it every time with SELECT (SQL) or see if the timestamp of the database file has changed. (If SQLite works this way. I never had the need to check on the file status.) This costs time as well.

Other web servers have a config file and reload it after getting signalled.

But: It's a bit early to say what Zed intends with SQLite. Maybe mongrel2 gets a configuration interface like Cherokee and only reads the DB on startup and after getting signalled.

I'm aiming for operational understanding of the data model used over being able to directly work with the config file. If it's a .ini (or other conf) usually you have no idea how the config maps to internal data structures. It's a mystery why some stanzas in nginx work on FastCGI but not Proxies for example.

With a sqlite3 based "config file" you'll be able to use real, turing complete languages to configure it, create control pannels, distribute the config reliably, and you can run .schema to figure out how it's structured.

It's also incredibly reliable and has many great features for this kind of thing.

Isn't the nginx's problem solvable with consistent implementation and complete documentation too?

I guess running some programming script isn't that bad to configure the server - but right now, people do a lot of basic operations on configs via shell scripting which do the deployment. Using bash/sh to interact with sqlite doesn't sound very inviting.

Possibly, it's a littler early to tell, but I'm sure I'll make it easy to use from the shell. Could even be possible to create converters from other config file formats to the Mongrel2 data model as well.

body{margin:0 6em;}

There, ftfy

Don't forget

font-family: sans-serif;

Instead of "sans serif".

Bonus point, it's licensed under AGPL.

AGPL is always good for misunderstandings and flame wars. Nice show. ;-)

Looks like the project web site also has a custom ticketing system and repo browser. Why not.

I believe that all comes free with the scm he is using, fossil.

Oh. Right you are. I hadn't heard of it, but it looks interesting. Certainly a big time-saver to get all that for free.

Zed loves Fossil SCM: http://www.fossil-scm.org/index.html/doc/tip/www/index.wiki

Fossil SCM is written by D. Richard Hipp. The same great guy behind SQLite.

This is a pattern I'm seeing from Zed:

1) loves small software (do one thing do it right)

2) loves C + glue code (perf in C, everything else script)

3) loves solid code

Some examples: SQLite3, Fossil SCM, Mongrel1/2, Merb (hinted by him), etc.

Wouldn't be surprised if he uses the tools developed by Dr. Hipps. Perhaps he prefers tools written by like-minded people.

What can I say, UNIX philosophy at its best?

Note: I like Dr. Hipp's software too (and OpenBSD). These tools are strong, solid, high quality, and represents everything that's good about software product. Unfortunately they're not the most "popular" thing out there.

Fossil SCM does not do one thing that does it right, it does many things actually, ticketing, wiki, scm and more I think.

So is this a rebranding of NoNoSql, or something new?

Umm it's a webserver

Yikes, -4 for an honest question.

Looks like he renamed NoNoSQL to MulletDB: http://mulletdb.com/index

MulletDB uses ZeroMQ and sqlite, and communicates with JSON, so I thought maybe this was the same project heading in a new direction.

It's not really an honest question because he didn't bother to click on the link.

Reply on news.ycombinator.com