The current Big data ecosystem is a difficult place to be for companies that bet the house on hadoop (cloudera, Hortonworks, MapR) for sure.

There are new technologies coming at a breakneck pace. Check out apache Flink if you need streams and the microbatches in Spark don't do it for you. The ML lib there is not that far ahead yet though.