105 karma · joined July 29, 2012
It's even simpler if your CLI can take a stdin argument, since you can just pass the buffer as a stdin parameter.
Thumbor's filter infrastructure allows you to change the image incrementally.
[1] https://github.com/thumbor/thumbor/wiki/Brightness
[2] https://github.com/thumbor/thumbor/blob/master/thumbor/filte...
At globo.com we have near a billion images (we are a big portal). Can you imagine pre-generating that many images every time a new format gets added?
We serve everything with thumbor with a Varnish cache in front of it and we're very happy with it. It has enabled our designers to work with any image size they can think of.
If you guys need more info, please check thumbor's docs: https://github.com/thumbor/thumbor/wiki/
Maybe we should have a different storage strategy if the data is too big? File storage? I just meant for it to be simple.
If you are going to use redis for storage then you'll need to fine tune it to the processing you are doing (we have).
We use tornado for the stream (the task processor). That means that only one user gets to run a task simultaneously.
That said, the stream is just an http application.
This means that you can scale it as easily as you would any web app.
We thought of parallel reducers and it does make a lot of sense. The reason they are sequential is to get a first release out so we can juggle ideas with people. If you care to contribute we'd love it. Even if you just create an issue.
r³ was designed from the ground up to adhere to HTTP. That means it's pretty easy to scale using our old and well-proven techniques: caching and load-balancing.