Complete Noobs Guide to Hacking Nginx | Hacker News Reader

Complete Noobs Guide to Hacking Nginx

blog.schmichael.com

123 pointsschmichael31 comments

I don't want to sound rude but the title doesn't seem appropriate at all.

Your article is about how you hacked nginx for your specific need (eventually fix it if it finally merge to 0.8) but not a "complete noob guide".

That being said, nice fix !

Sorry! I intended it to be encouraging for C dabblers like myself, but it's definitely not a "guide." It's my first time posting to HN, so I'll try to be more accurate with my titles in the future.

Sorry for the tone I was just frustrated !

Get a room.

> sudo make install

Judging from my understanding of the article, you are using Ubuntu. Please, please, please don't ever teach "complete noobs" installing anything this way. I won't insist on full course of backporting with apt-get source, hacking on debian/* and doing dpkg-buildpackage, but even silly `sudo chown -R $USER /usr/local && make install` is more appropriate than Slackware-ish `sudo make install`. And, while far from being perfect, checkinstall(1), is easy to use and should work fairly well.

I had to manage several Debian GNU/Linux-based systems with ton of software installed this way, and it was... well, quite painful. Package management is there for a reason.

The nginx source code is actually very readably and very well organised; I've never programmed professionally in c, but I feel quite comfortable reading and grokking nginx source.

Why wouldn't you just use POST? Being "RESTful" is even more ridiculous than validating strict XHTML, since, apparently, it causes you to build an unnecessary patch for your webserver. Just because a feature exists doesn't mean you are bound to use it (PUT, DELETE). The distinctions are nonsense.

Missing the Point (™)

Even if you allow that PUT is not a wild goose chase, he is still making trivial patches, the first of which is not correct, and the second of which he states he did not verify was correct, even after being rejected the first time. So if we discount that he is demonstrating how to build a bad API, he's still demonstrating how to be a bad participant in the patching process.

I'm adhering as best I can to the (nearly completely undefined) development practices of the project I'm working on. My assumption is that upstream developers will continue to guide me through the testing and improving of my patch. Even in a loosely run project I would hope my patch would be applied to the development build for wider testing long before it's included in production.

I'm using Nginx now, and developing a RESTful API.

I was planning to use the HTTPDavModule (http://wiki.nginx.org/HttpDavModule) which adds the HTTP and WebDAV methods PUT, DELETE, MKCOL, COPY and MOVE.

Did you try this before? If its compatible with proxying, then it wouldn't be necessary to patch the server for HTTP PUT requests.

Nginx supports PUT and DELETE without the dav module. It however requires PUT requests include a Content-Length header and does not support Transfer-Encoding: chunked (although the chunkin module adds this support).

The problem I was wrestling with was specifically PUT requests with no entity body and therefore no Content-Length (or chunked transfer encoding).

I've tried to explain in other comments why asking our users to add a Content-Length: 0 to empty PUTs isn't feasible.

If your RESTful API supports PUTs without bodies, you'll need to either use my patch or ask your users to always include Content-Length: 0.

Such distinctions have been discussed to death over the past 10-15 years, but the most obvious benefit is to allow developers to know something about an API without having used it before: PUT, DELETE, and GET to resources are all idempotent operations that act on an entire resource. A PUT to a collection acts on the entire collection. On the other hand a POST to a collection creates a new resource in that collection.

You can know all of those things and more about a RESTful web service without reading a line of documentation. So I would personally consider the distinctions between HTTP verbs very useful and quite underutilized.

The UNIX filesystem is another example where distinctions between seemingly arbitrary terms helps a user know more about a system without knowing details about a specific implementation: /home is for user data, /etc for configuration, /var can probably be mounted non-executable, /usr/local mirrors /usr, etc.

Despite a wide range of variation amongst implementations, someone familiar with these distinctions can navigate fairly comfortably on a wide range of operating systems and distributions.

On the other hand, let's pretend my patch is worthless. I would hope that doesn't lessen the usefulness of my post. My attempt was not to enforce some pedantic standards compliance but rather introduce developers to how to diagnose problems and come to solutions.

Can any standard browser actually send PUT or DELETE? What buttons do I press in firefox to send PUT?

Yes via Ajax, supposedly no via html form method.

You could build an Ajax button easily. You couldn't build a basic submit button to do it (supposedly--on my phone so no cross browser testing).

Your point is taken that browsers don't really speak HTTP. There are plenty of clients that do, generally for programmers working with APIs. These are at issue.

Right, there's a wide variety of Java uploaders that use PUT instead of POST.

> Can any standard browser actually send PUT or DELETE?

Not via a regular form (let alone link)

> What buttons do I press in firefox to send PUT?

It wouldn't be a button, it would be a form@method

Out of interest, why is validating strict XHTML ridiculous?

I think he is referring to sending it to the browser as MIME type application/html+xml, which causes the browser to throw up an ugly error page on any markup error. This idea was quickly abandoned since a tiny mistake could cause your entire site to be broken.

Or possibly he is referring to actually validating XHTML, since errors often reported by the validator do not matter as far as rendering goes. If this is the case, I personally disagree: validate your damn HTML.

That's, probably, because most (IMHO) sites are hacked "live", by modifying code right on the production server. And there're no automated testing involved, at all.

With proper development process (with development->staging->production cycle), and fair tests coverage, there shouldn't be any problems with strict XHTML validation. Except for human laziness (I'm guilty too, of course) and lack of easy-to-use "here, install this toy, fill in this five-line config and deploy with just one command" tools.

Yes, with the exception of sites that allow user-generated HTML, you could be fine. My take is that having strict validation on the users' side does not add any value to my projects. text/html works just fine.

Nginx is not “an established C project”, it’s the insane product of Igor’s twisted mind. It’s a brilliant totally asynchronous giant state machine – which is how it’s so blindingly fast – because everything is non-blocking, and everything is super efficient.

I'd like to see a diagram of that state machine!

What's the problem with sending "Content-Length: 0" as part of the PUT headers?

Browsers?

If this is the case, I think the article could have made it more clear that the problem was with software not under the author's control. All the article mentions is "the application registering the device," and I assumed that it was their application which was not sending the header. So a browser won't send a Content-Length header for a PUT request if there is no body?

I'm not criticizing the article itself; it was a nice beginner guide to making quick changes to an existing C project. My only criticism is that the first step when considering whether to make a change to a code base is to fully evaluate whether that change is appropriate, if it is fixing the problem in the right place. Perhaps the author did this and the article simply failed to mention it.

do browser support put at all?

Yes. Most browsers support all of the HTTP methods via the XmlHttpRequest object.

http://annevankesteren.nl/2007/10/http-method-support

The good news is that browsers aren't the issue for us at Urban Airship. The bad news is that it's worse: it's an infinite number of HTTP libraries of varying quality that hit our API. Some from pretty much every imaginable platform capable of HTTP: from mobile devices to VBA in Microsoft Access:

http://help.urbanairship.com/discussions/questions/161-using...

If the HTTP standard allows it, we probably need to support it (or degrade in a standards compliant way) because someone will try to use it.

Or sending a small meaningless body? On the other hand, he got to play with C which is often fun.

A couple of reasons:

1. The pragmatic one - our current web server doesn't require hacks like that, so there's lots of existing code out of our control that would break.

2. The pedantic one - at Urban Airship we try to offer as standards compliant a RESTful HTTP API as possible. Asking our users to work around shortcomings in our system is something we avoid as much as possible.

awesome post !

Reply on news.ycombinator.com