Pillow-SIMD: improving the performance of image processing | Hacker News Reader

Pillow-SIMD: improving the performance of image processing

github.com

108 pointsigordebatur44 comments

re: https://github.com/uploadcare/pillow-simd#why-do-not-contrib...

They can create three versions of each affected function (fallback, SSE4 and AVX2), place them in separate files (one file for each set of compiler flags), compile each version with its own compiler flags, then link them all together, and in the main module (which is compiled for generic cpu) run cpuid and set global function pointers to the right function implementation.

Then always use the global function pointer to call the right implementation of the function, and only expose calling the global function pointer if the function is exported from a shared library.

They do need to make sure that function pre and post conditions are preserved in all versions and that memory alignment/layout required by optimized functions is created by the generic code.

I think x264 does this.

You can even use the GCC ifunc attribute and get the dynamic linker to handle selecting the right function.

Certainly seems like this is the right way to go: http://clang.llvm.org/docs/AttributeReference.html#ifunc-gnu...

Any pointers for similar functionality for Visual Studio?

Code in separate libs then this inside #ifdefs.

#pragma comment(linker, ...)

Thanks!

In practice, it's very hard to make Python's distutils do this. I tried to figure it out years ago, see http://stackoverflow.com/questions/15527611/how-do-i-specify... . I also asked on the distutils list. No one had a clean solution.

The best solution I have is to interpose my own "CC", which looks at what is being compile and adds the correct compiler options for the right files before calling the real compiler. This is hacky and inflexible.

I'm curious how scpiy and numpy manage it. I'm pretty sure that they have solved some of this before.

In my short search, I didn't see evidence that either package has run-time dispatch based on the processor type.

I don't see "cpuid" used in the scipy code base, and the only intrinsic I found is specifically in a MS Windows #ifdef.

Numpy doesn't use cpuid. It does have compile-time options for intrinsics, but not run-time.

Entirely handled by the underlying BLAS library, OpenBLAS and MKL have this feature but few if any other libraries do. I believe FFTW has something similar, but scipy doesn't use FFTW.

It's possible to do automatically this by using a SIMD library such as libsimdpp (https://github.com/p12tic/libsimdpp). Everything is mostly abstracted away and you just write the SIMDified code once and add a CMake rule that builds the same file for several architectures, sets up dynamic dispatch and links everything together.

It'd be interesting to see this compared to Halide [0], which conveniently already has a resize example [1]. Parallelization and data locality for example can make big differences.

[0]: http://halide-lang.org/

[1]: https://github.com/halide/Halide/blob/e9ece5ee8ee9cb62295d5e...

I just discovered today that our resize example is actually absurdly slow. Working on a better one. So far it's about 8x faster. The other apps are much better tuned.

(I'm one of the main Halide devs)

Cool! On another note, are you aware of any other places with a lot of Halide code? A repository for common but not-obvious processes like brightness changing, denoising, contrast adjustment and such would be really cool.

I don't really want to defend ImageMagick, but does Pillow come anywhere close to it in terms of format support? A magnitude better performance is great, but sometimes you just need to operate on and convert strange files (X bitmaps, ASCII braille come to mind).

Many sites have a simple requirement of "allow user to upload a jpeg, resize it to create user profile pic for the user". No need to support anything but the most common formats for this, but you need security and you need speed of resizing.

There are a lot of formats, some of them are better exercised than others. It's not as complete as imagemagick, but we're getting one or two formats per release lately.

You could make a hybrid solution where Pillow supported formation (which should be the vast majority of your input) go through its filters while other formats are run through ImageMagick to be converted into something Pillow understands so they can be stuffed into the front end.

If you support JPEG, PNG, and GIF you will have 99% of the user data supported. Very few people upload RLE TARGA to webpages these days.

Pillow does have Targa RLE support, but test coverage is sketchy.

ImageMagick has also better default handling. For example, in a previous project, we were automatically resizing jpeg images. Surprisingly, some resulting images were blurry. After some investigation, we found that these images had been photoshopped and Photoshop was using a different sampling factor (one of the variables used in jpeg compression). ImageMagick automatically detected this sampling factor and reused the same, but Pillow was using a default sampling factor and was blurring the image.

Did you submit a bug? We're using libjpeg, so the support for that functionality should be there.

This is a bit OT, but if you only care about downscaling, cropping and/or rotating JPEG images by multiples of 90 degrees, you might also want to take a look at jpegtran-cffi (https://github.com/jbaiter/jpegtran-cffi).

It does all of the above operations without decoding the image, giving a significant speed boost.

I just benchmarked it against pillow-smd on my laptop (SSE4 only) and it's 1.3-1.8x faster, depending on the operation.

I wonder why this isn't benched against graphics magick (which I have found to be ~2x faster than imagemagick)

Both of the implementations you mentioned are not best known for its speed, rather, its their versatility (or bloatedness, some may argue).

With Python, is there an alternative for pexif.py to parse the orientation data for reorientating images? I used Thumbor in production for a while (which relies on Pexif) and found so many bugs in it that I had to make a hurried switch back to PHP.

Just slipped this into an image resize-bound Python script I'm using. Immediate 3x speedup. Glorious!

(But yeah, would love to see this merged upstream. Lots of Python libraries have sorted out harder CPU/GPU dependencies.)

Does Pillow have good security tests like fuzzing? Is it safe to pass arbitrary user input (file upload) to Pillow and Pillow-SIMD?

Fuzzing is slow. I've tried afl against a small subset of images and I was getting about 10 tries per sec per core.

There have been some bugs that look like people have been successfully fuzzing, but there's nothing organized.

The goal is certainly to be safe against user input, I would say we're there, but we're better off than we were 6months ago.

At the next major rev, Image.DecompressionBombWarning probably should default to error instead of warning.

Why did Pillow fork from PIL in the first place?

> If you have ever worried or wondered about the future of PIL, please stop. We're here to save the day

What's the backstory here?

PIL is dead. No updates at all since 2009.

I don't the whole backstory, but I was a PIL user before Pillow was a thing. PIL was old and crusty and had barely been updated since about the Python 2.1 days. While I don't know the details, I've always assumed that PIL was abandonware and the Pillow guys decided to reinvigorate it.

It didn't install cleanly with plone, so Alex Clark forked it to clean that up and make it work with pip and the other setuptools. Python 3 compatibility was added for pillow 2, and since then it's been a steady stream of additions and bug fixes.

(I'm one of the maintainers of pillow, but I hadn't seen the smid fork before.)

> What's the backstory here?

No backstory, the development of PIL gradually slowed down and ultimately stopped in 2011 (the last official release having been cut in 2009), I've never seen any info as to why that happened, it just did. Possibly the difficulty of implementing P3 compatibility while remaining pre-2.7 compatible.

PIL was always an idiosyncratic package with a completely custom setup script, a difficult install (including setuptools-incompatibility) and weird-ass modules (probably owing in part to 1.5~2.7 compatibility), as well as a fairly slow release schedule (bi-yearly at the best of times).

And so in 2013 it was forked (not entirely unlike LibreSSL from OpenSSL) and a dedicated group started on maintaining and cleaning the stuff, progressively phasing out pre-2.6 compatibility and adding Python 3 support, etc...

Very easy install:

  wget http://effbot.org/downloads/Imaging-1.1.7.tar.gz
  tar xvf Imaging-1.1.7.tar.gz
  cd Imaging-1.1.7/
  python2.7 setup.py build
  sudo python2.7 setup.py install

Alas, those simple days are over.

EDIT: Preemptively, I know how to verify checksums.

Thankfully, you can still do that if you don't like pip.

What about compared to opencv's image handling functions? I've seen Opencv run about 4x faster than pillow for certain operations.

Opencv doesn't provide convolution-based resampling. Only supersampling with INTER_AREA flag, which is comparable to BICUBIC for high-ratio downscaling. So, such statements don't make sense without exact code :-)

Is there any chance the ImageMagick developers would consider adding such optimizations?

Which operations you do the most? I'll try speed up them too.

How about versus imagemagick with openCL?

What if you don't care about speed, but you do care about not embedding strange scripting languages from the mid-1990s that can write files and make network requests?

Then fork it and make a command line tool that links to the C code.

Then you can use Bash or another strange scripting language from the late-1980s, presuming you're okay with it writing files, too.

https://github.com/imazen/imageflow works for that and still gives you speed. :P

Reply on news.ycombinator.com