The company I work for had a similar experience with Cassandra. We were running it back in the 0.6 days. Once any of our nodes got around 500GB on them the performance would tank.

It seems that they later fixed the performance in 1.2 (http://www.datastax.com/dev/blog/performance-improvements-in...) but by that time we moved our data over to HBase and haven't had any regrets.