Google just released its internal tool to collaborate on AI | Hacker News Reader

Google just released its internal tool to collaborate on AI

qz.com

91 pointsa_d24 comments

I was hoping to see a project which allows users to collaboratively train an AI model, so that open source projects can compete with corporate clusters of 100 GPUs. (e.g. to create useful speech recognition engines etc)

We've also been working on ML collaboration at https://www.tinymind.com. In addition to training models collaboratively, we have free, hosted Jupyter notebooks for both Python 2 and 3 with popular ML frameworks that you may want to check out :)

Interesting. I like the simple design/interface you've put on top of the various ML engines, including how to tweak them. Overall approach reminds me a bit of Digital Ocean's friendly UI for servers/hosting. Haven't tried the collab features yet.

Looks like I can not really use it just yet, need sign up and then be put into a waiting list.

Is it true that Collaborate is on Python 2.7? Since everything is switching to 3.x these days it would be nice they can update Python version as well. This tool is built on top of ipython-notebook(jupyter) which supports 3.x officially already

Does anyone have a more in depth source or a link to the source code?

I'm currently using a slightly hacked Jupyterhub to do what I assume is the same thing, I can't work out from that article (maybe I missed it) wheree the kernel actually runs with this?

It's not useful to me unless you can self-host and hack it. If that's possible and they add python3 will probably end up putting it into prod.

GitHub.com/jupyter

Thanks for replying but I know what Jupyter is and I'm already using Jupyterhub. My question was on more details/source code for this Google service that they've built on top of Jupyter which looks equivalent to a hosted Jupyterhub.

I think it's a completely new UI that some people at Google built from scratch, which is much easier for them to do realtime sync with using their API (https://developers.google.com/google-apps/realtime/realtime-...). I think they felt there were also some usability issues with the standard Jupyter UI when sync was added in. (Source: I visited the group writing this a few years ago at Google and they showed me a demo which looked just like this.)

For googlers, how does this differ from datalore.io?

Looks the same to me...

I already signed up for their free CPU cycles.

doesn't work in Safari

Use Firefox

> ...supports Python 2.7 and has to be used on Google Chrome.

Looks like that's not possible.

> With Colaboratory, users create notebooks, or documents, that can be simultaneously edited like Google Docs.

I wonder how Google came up with this "feature". If there's one thing I don't need, it's other users editing my code behind my back. Probably a case of "just because we can".

Questionable usefulness aside, technically, I think it is a very cool feature! :)

(Roughly speaking: IPython + Gdrive = Colaboratory)

There is something that really is IPython + Gdrive, and it it's https://github.com/jupyterlab/jupyterlab-google-drive

Colaboratory is perhaps more like: Jupyter kernel + Gdrive + lots of new frontend code

Would you say the same thing about the collaborative editing features in Docs?

> If there's one thing I don't need, it's other users editing my prose behind my back.

To me, this sounds like a fantastic teaching tool, a fantastic pairing tool, a fantastic interview tool, and a cool way to collaborate on scripts where Git would have too much friction.

I think this sort of tool can even scale up to projects larger than a single file — I’ve used Floobits (https://floobits.com/) to co-op edit whole Django webapps in Intellij. It’s super fluid and productive to use co-editing tools to very quickly collaborate on new feature build-outs, either pairing together on a single file or splitting work between Frontend and backend concerns.

> Would you say the same thing about the collaborative editing features in Docs?

Yes, a simple "sync" button that pulls in the updates and merges the differences (like git) would do for me, and in fact would cause much less confusion than somebody editing behind my back.

Then perhaps this isn't for you? Collaboration tools such as these are fantastic for some teams and use cases.

Don't use the tool, then.

Jupyter is already huge in the research world. Your use case isn't representative of the world.

Your view appears to be that if you don't need it, nobody needs it, ergo it's dumb. But as Shakespeare wrote, "There are more things in heaven and earth, Horatio, than are dreamt of in your philosophy." If you don't need it, maybe it's for somebody else.

There is a lot more value to collaborative features than just giving other users the ability to edit code behind your back. I implemented the collaborative Jupyter notebook capabilities in https://cocalc.com back in 2014 and have observed, rewritten and used it since then, and there are many additional benefits to collaborative features.

(1) If you have more than one computer, you can open the same notebook on multiple computers at once. This better uses your available screen real estate.

(2) Similarly, you can open the same notebook in two browser tabs, which provides two views on the notebook at once. This is very handy when working with the long linear format of Jupyter notebooks (e.g., writing documentation near the top and running computations near the bottom). Some notebooks are really long with hundreds of cells...

(3) The underlying functionality that makes implementing collaboration possible also enables saving the exact edit history of the notebook. With CoCalc at least, Jonathan Lee added a history slider (like in Hackpad) that lets you easily browse the history of all edits to a Jupyter notebook. This can be very useful if you delete something, or want to know how your notebook got to its current state. It's also useful in case somebody really does "edit behind your back", since you can revert the changes.

(Regarding shortcomings of Colab: CoCalc is similar, but works in Safari, Firefox, mobile Safari, etc., and has support for all popular kernels, including Julia, Anaconda3, R, etc.)

Reply on news.ycombinator.com