I... agree. But my argument was not "use NoSQL". I never suggested to use NoSQL. I suggested to use Spark as the compute layer.

Spark has a sophisticated [1] query engine, which seems perfectly capable of pruning partitions and pushing down index scans into data sources. Yes, Spark can't do writes, so you'll still have to build a transactional KV store that sits underneath the query layer, but you won't have to implement SQL from scratch. (This seems to be similar to the approach taken by e.g. SnappyData [2] and Splice Machine [3].)

[1] https://databricks.com/blog/2016/05/23/apache-spark-as-a-com...

[2] http://snappydatainc.github.io/snappydata/

[3] http://doc.splicemachine.com/2.0/GettingStarted/Architecture...