Hacker News Reader

Comment by old-gregg | Hacker News Reader

Can this simply be an artifact of terrible disk I/O on AWS or overall difference between ZFS/ext3?

Do you think the results would have been similar if you were to use no-compression-ZFS instead of ext3 on a proper database hardware?

Basically trying to figure out if the low performance of uncompressed dataset is specific to AWS/ext3. Thanks.

View on HN

Another issue is that ZFS is extremely aggressive with caching data in ram (L1ARC). That can eat up memory you'd rather give to the database heap and also tends to skew benchmarks.

Placing limits on ARC (25% for me currently) limits that effect.

Yes, but postgres' design in that area actually helps. Postgres relies on the OS caching the data tables for the most part. There is some caching in shared buffers, but generally that's not a huge portion of the memory of your db system. (10-25%, and not more than a few gigs)

I run Postgres on ZFS, and simply limit the amount of memory dedicated to the ARC.

I have seen similar advantages comparing XFS to ZFS+compression on a local server (Centos 6.3, ZFS on Linux 0.61).

Using a 2 disk striped volume for PostgreSQL 9.2, I get an average of 2.5X compression (as reported by ZFS), and a 1.5 to 2X time reduction in database restores (single threaded or 8 jobs in parallel).

Given this development box has relatively slow 7200 RPM disks, the tradeoff of more CPU time for less disk transfer makes sense.

Edit: My use case is an OLAP server. I can't state how the tradeoffs affect OLTP performance.

old-gregg makes a great observation here. The addition of a ZFS benchmark without-compression is needed to isolate the compression as a factor in the speedup.

That aside, I thought this was a wonderful article with non-intuitive findings. Very interesting, CirtusDB [edit: er, CitusDB]. :-)

Citus

old-gregg's point is valid. Since we didn't benchmark ZFS without compression we can't say for sure how much of the performance improvement is attributable to compression vs. just ZFS.

As far as AWS goes, we have noticed ephemeral disks connected to the same instance can exhibit fairly large performance differences, and attempted to control for that in our tests by reusing the same disk for each test run.

I always question benchmarks on ec2 because of "noisy neighbor" effects - maybe that's in the noise, but someone trying to replicate your results would perhaps see significantly different results depending whether their VM landed on a busy node or not... Tag this one #ymmv

+1 on testing uncompressed ZFS

I did see a blog post about MySQL with similar results (at least compression was a significant win) some time ago - disks are so slow compared to what throughput modern CPUs are capable of on these sorts of compression algorithms.

You're also testing on a c1.xlarge that gives you excess CPU compared to I/O, so it's potentially biasing your results.

While I can't claim that we logged CPU load while running these tests, I can say that I watched the output of top and iotop and that the CPU load was relatively light. It's also worth pointing out that Amazon describes the I/O performance of c1.xlarge instances as "high". We also considered using an hs1.8xlarge "High Storage" instance for these tests, but eventually decided that we were more interested in testing against conventional disks as opposed to SSDs.

Did you use instance storage? EBS? Provisioned IOPS?

There are vast differences between those three.

Using compression with zfs on solaris derived platforms serving as a san/nas backend for vsphere appears to speed up every workload backed on rotating storage. Well, not if vmdks use guest full disk encryption, but that's understandable.

Reply on news.ycombinator.com