Varnish Cache v3.0 - Features and Additions in Brief | Hacker News Reader

Varnish Cache v3.0 - Features and Additions in Brief

blog.unixy.net

32 pointsjjoe16 comments

VMOD is to Varnish as loadable module is to a kernel. The new Varnish provides a programmable interface that enables expanding the VCL (Varnish Configuration Language).

VCL is the only fault I can find with Varnish. Is a new language really necessary here? Why not extend via lua or scheme?

It's probably only a matter of time until someone writes a Lua-to-VCL translator/compiler.

I've heard fantastic things about LuaJIT, but would Lua be fast enough to be useful at the speeds that Varnish provides? I've used Varnish to handle huge loads and it did so without breaking a sweat, wouldn't having to go through Lua take away much of that advantage?

I would have thought so, once the code has been called a few times the JIT will kick in. There will be a small overhead for actually making the C to Lua call, which you can't get rid of, unlike the other way round where you can call C from Lua with the FFI with no overhead.

If you want to see what the overhead is, the Lua module for Nginx is comparable if you compile with LuaJIT, should be fairly easy to benchmark just adding a small bit of code. Actually I tend to use this combination instead of Varnish where I actually want to mess with stuff, like do authentication while processing requests etc.

I can imagine how this would be extremely beneficial in terms of ease-of-use. I hope someone creates a lua module for Varnish using VMOD, which I don't think should be very hard. Maybe I should give it a shot, actually.

You're complaining about the DSL but have no issue with varnish as kernel module?

First thing I do with a production system is build it a new kernel, with a tight feature list, and disable modules.

I think you read it wrong. Varnish is not a kernel module.

It's a compiled language for speed and so that people can use inline C to write simple extensions.

Inline C can offer more than just simple extensions.

I've extended Varnish 2.1.* to include a custom authentication layer that basically means I can drop multiple different applications behind Varnish and have the equivalent of a Web Account (of the ilk of Google Account, Yahoo Account, etc).

The functions provided by the shared library I've written require a whole load of VCL to grab hold of parts of the cookie and to pass things into the functions, before then handling the return values and re-writing headers (that the backend servers then look for).

It all works really well, and I haven't found VCL to be an ugly domain specific language to work with, especially when you consider that it translates directly to C which is then compiled.

When I think of what it took to write an Nginx mod, VCL with Inline C is a dream.

Ah, the infamous second single sign-on system! I was going to send you this link, good thing you saw it. I'm very excited about the prospect of a Lua Varnish module, I think it would make this sort of hackery (writing server modules directly for various tasks) much easier.

The exact same sentence applies to Scheme. Example: http://wiki.call-cc.org/eggref/4/bind

VCL gets the job done, is compiled to C, has less overhead, has less power giving you less potential to shoot yourself in the foot (unless you have inline C blocks) and it's easier to learn than both lua or scheme.

And why don't people ask the same thing about Apache or Nginx? The reason for that, IMHO, is that with VCL people see the potential of fine-grained request, response and cache control through a real programming language, but that's only because VCL is a DSL OK-ish enough for that purpose (something to think about).

I know the title says "in brief", but it's a little too brief for me. From the horse's mouth:

http://www.varnish-software.com/blog/varnish-30-changes-esi-...

http://www.varnish-software.com/blog/bans-and-purges-varnish...

http://www.varnish-software.com/blog/streaming-varnish-30

Something to pay attention to from that streaming post: if the asset you're streaming isn't already in the varnish cache, only the first user to request it will get it streamed. All other clients will be blocked until the first stream is complete and cached.

> The object will be set to busy while it is streamed. Other clients will be put on hold. Furthermore, the rate at which the object is fetched by the client dictates how fast Varnish will fetch it from the backend.

Unless I'm reading the above wrong, [re-warming your cache for these large objects would seem more prudent than relying on streaming.

There's also a nice write-up by Redpill Linpro employee and Varnish contributor Kristian Lyngstøl:

http://kristianlyng.wordpress.com/2011/06/09/varnish-3-0-0-r...

About the same day as Apache Traffic Server, another http proxy, hit version 3.0 as well http://trafficserver.apache.org/

Has anyone used ATS, any recommendations over Varnish?

varnish

Thanks

Reply on news.ycombinator.com