Oh wow... I didn't even know these limitations (and in fact ,they are not very Google-able).
Can you talk about these limitations and your experience?
Can you talk about these limitations and your experience?
https://issues.apache.org/jira/browse/SPARK-1476
https://issues.apache.org/jira/browse/SPARK-6235
You'll hit this bug when your model size is larger than 2GB.
BTW, Recomputation of RDDs may result in duplicated accumulator updates. So please do not use accumulator in your trainer for gradient summation. They know that, but they said they will not fix that.