Hacker News Reader

Comment by pubby | Hacker News Reader

Unfortunately SSE doesn't speed-up huge data much. It's only fast when everything's in the cache.

I coded up a 5x5 Gaussian blur using NEON instructions. I used the standard seperatable filter technique, one horizontal pass and then one vertical pass. Benchmarking effectively measured 2x the memory round trip time on the raspberry pi. My second implementation rotated 8x16 image blocks using register permute instructions. Only the block borders needed to be cached, meaning everything shared between blocks for a 640x480 image fit in L1. Despite being substantially more complex, I cut the execution time by 30%.

Lesson learned, sometimes the easiest performance gains are found by not being naive about memory access. The extra instructions were inconsequential.

Curious if you have encountered Halide (http://halide-lang.org/) and if so, your impression or thoughts. The main advantage is being able to easily express optimizations like the one you mentioned, allowing you to experiment with different optimization parameters/ideas more quickly.

I actually drew inspiration from halide. I think it performs well because memory access / cache misses are more expensive than a few extra instructions, but it has its limits. For example, I'm not sure how you'd implement the in-register 90-degree rotation in halide. Therefore you'd probably wind up with an extra round trip to L1.

Impressive, but definitely beatable given sufficient free time.

Not sure what you mean, I can process non-cached sequential data from RAM over 20 GB/s by using SSE/AVX.

There's no chance you could achieve same by using scalar instructions. SIMD can access memory a lot faster than scalar.

Random access is another matter. The trick is of course to avoid non-sequential access patterns.

Isn't the bottleneck in any sequential access case the memory bandwidth?

IOW: Are the scalar instructions slower than memory bandwidth?

you can saturate memory bandwidth without SIMD, since you can issue at least 2 8-byte scalar loads per cycle. it does not leave much room for actual processing though

Isn't that just a prefetcher win? AVX/SSE lets you do more computational work per cycle but I don't see how it would improve memory bandwidth/access.

The CPU's load/store unit is usually designed with SIMD in mind, and the access width is 16 bytes (or 32 bytes for Haswell-and-later Intel CPUs). This means you get more bandwidth by using SIMD.

Do you have any explanation of why you think so? And how much is hugh data for you?

I work with image processing, >1GiB/s. We optimize for each platform for hand, many embedded platforms but also x86_64. Most of our work is detectors and loss-less compression and we could not do these things in realtime if it weren't for SIMD/MIMD.

I was not very specific. What I meant to get at was that if your bottleneck is memory, then optimizing for instructions is not going to help. is_sort is an example algorithm that's far more dependent on memory than it is on instruction throughput.

If your bottleneck isn't memory, then yeah, SIMD is a real boon.

It’s not all about compute. SIMD instructions allow you to load more bits per instruction cycle. When you are doing sequential access on a modern processor, this can make a huge difference. It allows you to use the memory bandwidth to its full capacity.

It's rare to have a meaningful algorithm where memory is the absolute only bottleneck. Not much else aside from memcpy and even that can see a benefit from SIMD tuning on many systems.

Especially since high performance software is glad to have even a 1% boost in speed.

Reply on news.ycombinator.com