There are approaches for using Spark to distribute hyperparameter tuning and cross validation: https://databricks.com/blog/2016/02/08/auto-scaling-scikit-l...

However, for the example in this post, I would recommended using the logistic regression provided by MLlib to scale up.