As someone who has never used Varnish but has used Nginx's cache to some degree... whats the benefit of placing varnish in the middle vs going with Nginx?
Aka serving a cached response while the cache is getting refreshed (even to the request who initiated the refresh). Currently when nginx has to refresh the cache it will "hang" the current response until it gets refreshed (the requests that follow, even while the cache is getting refreshed, will get the stale version tho).
It's the difference between your client being amazed by how fast your api is, to "wtf does page X takes 3s to render from time to time" comments.
The way to work around this issue with nginx (for simple endpoints) is to write scripts that will hit the endpoints to make sure the cache is always hot, but it's a sad, half working hack (the CEO can still hit that cold cache page himself, if lucky enough).
[1] - https://www.bountysource.com/issues/972735-proxy_cache_use_s...
Once you start breaking your stack apart to 1) add redundancy, 2) isolate hardware for different workloads, and 3) scale the components independently of each other, the situation gets a little more interesting. Sure, you could run different clusters of Nginx for each of SSL termination, static content, cache and compress, etc. and have them reverse proxy to each other (and some people probably do this), but when you get to this evolution of your architecture, it's worth evaluating the different options for each layer of the stack. There's an argument to be made that e.g. HAProxy is a better load balancer, Nginx is a better static content server, Pound is a better SSL terminator, Varnish is a better caching and compressing reverse proxy, etc. (N.B. that I am not saying these things, I am just saying that they can be said.) If you're not serving static content at all—let's say you're presenting an API written in Elixir/Phoenix—it may not make sense to go with Nginx in the first place!
Regarding Varnish, specifically, I've found that it offers unparalleled power and flexibility when it comes to caching and compression. Yes, that power and flexibility comes at the cost of more complexity, but it's there when you need it. It offers different storage backends, including a pure memory backend; the ability to serve gzipped content directly from cache (for supported clients); PURGE and BAN HTTP verbs; synthetic responses; query string sorting; the ability to serve stale content while updating; ESI; and probably a whole bunch of other stuff I'm forgetting and/or have never used. Nginx and Nginx Plus may support some or all of these things, built-in or as modules... I'm not sure.
It would be interesting to see data that compares the two, and see how close they come in different scenarios, and how tuning/configuration might affect the performance.
At that point, talking about metrics is likely procrastination.
While in your experience the performance differences were negligible, but that doesn't mean that will hold true in all usage scenarios. For example, maybe they perform similarly when caching many small files, but one struggles with serving longer running requests.
Parent above was saying "both are at the same speed / none offer any speed advantage".
He is then asked for any metrics to support that statement, which is not an extravagant request and should be encouraged more.
Your answer that "At that point, talking about metrics is likely procrastination" and "one will be faster than the other in any given scenario" serves nothing other that further his statement while deriding the need for said metrics to prove that this is not turning anecdotal evidence into facts, and frankly serves nothing to the discussion short of saying the point should be accepted without verification and that asking for verification is useless.
A point of view I really cannot agree with. I do not know which is faster, nor if any of them is faster than the other or not, but if somebody claims one way or the other in a factual manner, he should not find it a waste of time to find an actual metric that shows it.
There are other criteria.
http://www.bbc.co.uk/blogs/internet/entries/17d22fb8-cea2-49...
For caches, the bugaboo is latency, and event architectures are not good choices when small-as-possible latency is an issue. It is here that heavily threaded architectures really shine. Sure, you can likely get by with less resources w/ events, but (1) performance will suffer and (2) as thread implementations improve, threading will only get better and more "stingy" re: resources.
* More fine-grained control of your caching is possible
* Easier to express normalisation of requests (increase your cache hit rate and protect your underlying origin from malicious requests by discarding cache busters)
* Inline C means you can do things like move your authentication to the edge
* In theory if you have enough RAM you can go faster than nginx's on-disk... in practice the sweet spot for the gain is small and they're both on par
But then... it comes with disadvantages too. Like most Varnish services would never let you do the nice Inline C stuff because no-one in their right mind would run untrusted code in their environment where it could impact another customer. If you see a provider do this (at any price point), avoid them.
If you make your own VMOD(can take from a few hours to days), make sure to send a PR and add it to the directory above (IOW: share it) :)
Not in theory, but in reality. Even if you don't, varnish's file on-disk implementation is just as good as nginx's if not better.
The idea behind a cache is to increase performance. If that is the priority then you would pick a design and implementation geared towards that problem set. Sure you can say "Well, nginx or httpd is 'good enough'" and that would be right. But when nginx or httpd aren't good enough, you need varnish. And even when you don't need varnish, it is often a better architectural design to break out specific functionality (best of breed).
I will take it with a grain of salt or do your own tests, if you really care.
In terms of introspectability, Varnish (afaik) is without equal.
I believe that the power and utility of VSL are so underestimated and overlooked that I don't know where to start...
2: More control.
3: Better compliance