Pilbox: an image resizing application server | Hacker News Reader

Pilbox: an image resizing application server

agschwender.github.io

52 pointsagschwender41 comments

I was recently looking for services that could handle resizing and cropping. Personally I prefer the ones that act like a proxy, so I can store the images in my own S3 bucket. Here are the services I found:

http://www.imgix.com/ http://cloudinary.com/ http://www.cdnconnect.com/ http://cloudresize.com/ http://cloudimage.io/ http://www.blitline.com/ https://transloadit.com/ https://uploadcare.com/ http://www.resrc.it/

You can add ours to the arena: http://thumbr.io/

See also thumbor [1], it's been used and developed by the biggest news portal of Latin America.

> thumbor is a smart imaging service. It enables on-demand crop, resizing and flipping of images.

> It also features a VERY smart detection of important points in the image for better cropping and resizing, using state-of-the-art face and feature detection algorithms (more on that in Detection Algorithms).

[1] https://github.com/globocom/thumbor

Pilbox looks great. Added it to this list of such image resizing services:

  https://github.com/adamdbradley/foresight.js/wiki/Server-Resizing-Images

Many, many implementations, and each goes about it differently. There's unfortunately no widely adopted standard for constructing a URL to request a resized/altered image.

It would be great to see traction on something like this RESTful Image API Specification:

  https://github.com/riapi/riapi

Adam, the guy who built foresight.js in your link above, runs a CDN with image resizing features built in: http://www.cdnconnect.com/

It has some other cool features like optimizing images/serving webp ones if supported by the client. It can also convert Illustrator and PSD files into svg or jpg/png files. Worth checking out in addition to Pillbox.

This is a perfect use for http://www.docker.io, btw. I'd love to be able to run 'docker run pillbox' and have it download and install all the dependencies.

(The point of docker is that everything pillbox requires on can be separated completely out from the rest of the system. pillbox requires many libraries. it could require nginx + a special config for proxying. I don't want to do 'apt-get install pillbox' and have it install all this stuff system-wide. I want to keep everything pillbox needs in a separate environment and communicate with it via a port.)

I think nginx should be included for a proxy cache in front of pillbox. Docker would make this simple to setup.

Let's work on it together.

email sent!

I was looking for something like that (hosted) yesterday and I stumbled upon cloudinary[1].

The do the resizing part for you as well as the hosting. But the best thing they offer is face recognition. It's incredible! I haven't been in live with a service since guthub.

Their free tier is probably enough for smaller projects (but without backups in your S3 bucket)

Thumbnails with focus on the face looks fantastic!

[1] http://cloudinary.com/invites/lpov9zyyucivvxsnalc5/bwrrdazeh... (affiliate link. Without: http://cloudinary.com)

I wonder what they use for the face recognition, and if that could be added to pillbox?

Looks like pilbox already supports some sort of facial recognition. From the changelog:

0.5: Facial recognition cropping

We built something similar in node.js (as express/connect middleware)

http://github.com/sydneystockholm/imgr

You can also do this right in Javascript on the client side using ImageMagick.

see http://manuels.github.io/unix-toolbox.js/ and https://github.com/manuels/unix-toolbox.js-imagemagick/tree/...

Being able to resize images via url parameters is one of those great ideas, but I've always been surprised by how long it's taken to catch on.

Awesome.

I've got a feeling it's something that many people have been doing on even mildly complex sites for a long time, but people seem to be trying to SaaS it up recently.

protip: Dynamically generating content by URLs is also a nice DOS attack surface if you don't rate limit in some way or try to detect abuse.

Definitely, true. The purpose of the signature and allowed hosts functionality is there to help mitigate that risk. Also, since the image service would be run separate from the main application, it shouldn't impact the performance of your site. Likewise, in a real production environment, this would run behind a CDN or Varnish, the DOS would just prevent new images from being generated until it could be addressed.

Honest question: Why is it such a great idea?

Wouldn't it be easier/quicker to either:

a) Have a command-line tool to resize images? Alternatively, one that's integrated into the shell so that you can access it from context menus? I.e. right-click -> "resize image"

b) Just open it in mspaint or your open-source image editor of choice?

Pretty sure the idea is to make this available as a service, for UGC-type sites.

These services are becoming more interesting as responsive web design takes off and everyone tries to format their images based on client agent needs. Often the exact dimensions and treatment of the image can't be determined before request time depending on the user agent's screen size, resolution etc. Also, why spend time as a designer cropping and scaling images when you can automate it all and make it on demand?

This thread caught by eye since I have been working on the very same idea for the last year. http://www.pixtulate.com

It's simpler to store the original image, then generate URLs like http://image-server.com/resize?url=http://s3.com/original.jp...

This lets you change the size of thumbnails by only changing the URLs.

I've been doing it for years, works great.

Doesn't this mean that the server needs to manually process the image on each request?

You should stick a CDN in front so this doesn't happen. Ideally once the server generates the image, it would also be cached there.

You probably want nginx in front of pillbox. Nginx setup as a basic http proxy cache.

So, CDN -> nginx -> pillbox.

This is because CDNs can send lots of concurrent requests to the origin server if lots of people are requesting the same new image simultaneously.

PHP sites have been doing this with TimThumb for quite a while. https://code.google.com/p/timthumb/

I wrote https://bitbucket.org/btubbs/thumpy at work and we've been using it in production for a couple years. It's very similar to Pilbox.

I wrote a similar image service (resize, crop, border, color analysis, format, and other fun features) a few years ago. It served about 6M image requests a day from 4 servers, plus a whole lot more from the CDN.

If you're using a CDN, just make sure they cache on query string parameters. Our resize controls were part of the URI, but the other commands were part of the QS. Our CDN stripped queries, so the first image would be cached and any calls with different QS would return the first hit.

A possible alternative that I never implemented was using the request header instead of the query string for additional commands.

Edit to add: Also, if you have lots of resized images on a page, be careful when Google or Bing scraped you. Your CPU and IO will go through the roof as your servers go crazy trying to dynamically generate all the images.

So what happened to it? Are you still running it? If not, why not?

Not running (probably). We were acquired by a competitor. Last I heard, they upgrade their image server code with the bare essential features to migrate over our clients. A lot of good software was buried in that acquisition.

Kind of strange the bots would spike you. Wouldn't anything on a page have been used before and thus cached on your server as a file? I've written all this stuff before as well, including overlaying site logo on images that were hotlinked to boot, but I always cached to the file system anything I had to generate the first time it was requested and simply served it after.

Short version: We had direct access to all the large news image providers and analyzed the images for topic and story identification. We used this data to dynamically generate hundreds of thousands of pages. Way too much data to cache to file (our storage cluster was many many terabytes). We used a CDN to cache the images. When a bot would scrape, it would hit all the old pages that fell out of the cache. Perhaps our fault for having such a large sitemap.xml.

> Being able to resize images via url parameters is one of those great ideas, but I've always been surprised by how long it's taken to catch on.

Our video CDN customers can use a custom URL to get a thumbnail of any size from any key frame in their video files, fantastic for building "trick play" responsive HTML5 video player UIs without having to manage a single image.

Just one of the countless little nice-to-haves that sets a VDN apart from a CDN.

Who are you using as a VDN?

His message indicated that he works for a VDN provider, rather than be a user of one.

This would seem to be his employer:

http://www.advection.net/

Cool. I bet this a pain point for many projects. I had the same problem and ended up cooking up something similar for Django [1], but I like that this one leverages Tornado.

[1] https://github.com/hcarvalhoalves/django-rest-thumbnails

Do any of these support International Image Interoperability Framework: Image API 1.0 http://www-sul.stanford.edu/iiif/image-api/ ?

1000 Memories released a cool Java service for this called Photon.

https://github.com/1000Memories/photon-core

Similar idea, same tech, more mature (but more complex?): https://github.com/globocom/thumbor

Just to make you laugh : I worked as a webdesigner for a retail chain, and we have bought Adobe's Scene7 which does quite the same job. It was a 102 000$ / 2yr deal.

I've used Scene 7 for the past five years and think that it is a great product. The cost may seem expensive if you just look at the dynamic resize/crop technology and hosting, but it is a good solution for digital asset management for e-commerce. Our photo studio and re-touchers upload directly into Scene 7 and it has simplified the entire workflow.

I agree that Scene7 is a good product, but really overpriced, though my company was happy to pay for it because it was like a magic patch applied on their shitty worflow.

Wouldn't this be just a bit more awesome in some use-cases as a PNaCl app?

EDIT: I get it now: People would change the URL to dynamically request an image of just the size they need.

Awesome! My teams have needed this. Thank you

Reply on news.ycombinator.com