Makes most sense to compare Impala and Spark architecturally. Ibis will eventually integrate with Spark. We've been focusing on Impala integration for reasons cited here: http://blog.cloudera.com/blog/2015/07/getting-started-with-i...

In particular, we're working on byte-level shared-memory integration with Impala (which is implemented in C++ with LLVM runtime codegen — the project's tech lead, Marcel Kornacker, was the tech lead for Google F1's query engine) to run user-defined logic without data serialization / memory usage overhead. This also opens up Python's HPC / scientific computing stack and existing data libraries to be run in a Hadoop setting without Python-JVM interoperability issues.