How do you think of Google is putting their best effort to head back to China ? They were working real hard and getting shut down.
155 karma · joined May 21, 2012
In these experiments, p is quite small(About 1K). So the sizeof the weight vector (or weight gradient vector) is only about 4K (for float) or 8K (for double). It should take less than 0.1 second to transfer these vectors. So there are no need to use tree allreduce or bittorrent to transfer them. These big inventions from VW and Spark become useless and burdensome. And also, in such scene, Spark's accumlator has no advantage than the tranditional map-reduce.
Another point is: Though correctness is more important than performance, it's very difficult to get a correct implementation even for the most basic problems. e.g. Most implementations of OWLQN(which is for L1 regularized convex problems) are wrong. More serious, sometimes the system you relied on is problematic or unstable by design. e.g. This Spark issue https://issues.apache.org/jira/browse/SPARK-5490 may never be fixed.
I would also recommand google's sensei(https://github.com/google/sensei) as an alternative to be evaluated. I recommand it just because it comes from google. But it's not well-known yet.
Another Story: My friend Li works for Alibaba,the largest internet company in China. Li is a software engineer with 8+ years working experience, and his salary is less than 3000 USD per month.
But Beijing's House price is much higher than Silicon Valley.