Interactive Go programming with Jupyter

As someone who hasn't worked with Jupyter beyond a simple iPython shell, this comment sent me down a bit of a rabbit hole. If anyone's interested, here are some links.

Jupyter docs for making a kernel: http://jupyter-client.readthedocs.io/en/latest/kernels.html

List of jupyter kernels: https://github.com/jupyter/jupyter/wiki/Jupyter-kernels

Jupyter seems like a cleanly designed piece of software that is easy to integrate with. I like the use of ZeroMQ for the wire protocol.

It compiles a plugin for each Jupyter cell code with "go build -buildmode=plugin" then loads it.

It's easy to see with Linux "lsof -p LGO_PID": it lists, among others, the loaded shared libraries - there is one per Jupyter cell code.

My feeling is that the friction of learning a language has a large impact on the success and adoption of the language.

Something like this definitely reduces friction.

A comparison with gophernotes is included in the article. From the things they've chosen to compare, it looks like a step up!

Jupyter has uses outside of data science; I find it to be an extremely useful programming scratchpad for web development with Python and Node. It was renamed from IPython to Jupyter to reflect its broader scope, Python was the original kernel but there are many now:

https://github.com/jupyter/jupyter/wiki/Jupyter-kernels

> I don't get why you want to use this over an IDE. All you get is cached intermediate variables and being able to print them, am I right?

You get more than that from Jupyter. Especially with Python, but for Go also.

Jupyter is for exploration and prototyping. Think of Jupyter as being a massive improvement on the command line shell, rather than an IDE downgrade. If you're developing large structured programs, yes the IDE is definitely the way to go. But if you're doing interactive coding and you want to run the code line by line or in small groups, you'd use the shell, or better yet Jupyter. Jupyter is between an IDE and a shell.

Jupyter gives you inline images & plots. It's made with visual results and interactive plotting in mind. Most IDE's don't have anything like that. You can also compare Jupyter to Maple or Matlab or Mathematica.

Jupyter lets you add readable formatted markdown between code blocks, and you can export your notebook in presentation formats like PDF or HTML.

Notebooks show the results of the program run, so someone you share with can see what happened before they run the code.

And I don't know about the Go kernel, but with Python at least, Jupyter most often installs a sandbox environment that includes several powerful Python libraries. What this means in practice is that sharing notebooks is vastly easier for the recipient of a notebook.

Jupyter serves a different purpose than IDEs and text editors. The main use cases are experimentation and presentation.

If you want to try a library it can be more flexible than a REPL and more interactive than a program in its own file.

Jupyter is also used to produce documents that mix code, prose and visualizations. A typical use case is to write a small amount of code that processes some data and produce a chart, providing explanation formatted in Markdown along the way. These documents can be exported in a variety of formats, including printable documents, presentation slides and blog posts. There's a very vibrant and high-quality ecosystem of projects developed around Jupyter which ranges from making interactive documents to running distributed notebook servers.

In general you wouldn't develop a full software project in Jupyter.

Note that Jupyter is not the only project that provides this type of notebook interface. You'll also find Zeppelin coming from the Java community, Observable for JavaScript and the very powerful Nextjournal SasS product.

I love to use notebook style IDEs to prototype, test and benchmark "smaller" things. And with smaller I don't mean things that aren't complex or difficult, but things that don't need thousands of lines of code or ellaborate software architecture.

They are great to try new algorithms or evaluate their performance. If something is not quite right the first time, just change a line and evaluate that block again, without having to run another, potentially expensive block, again. You get to see the changes to your latest block immediately, without having to go through all the previous steps again.

They are also great if you want to put together some graphs or statistic displays.

I'm still hoping for something like Mathematica for javascript, including ways to print graphs, meshes, point clouds, etc. to you notebook.

@rufugee @rhizome31 @mschuetz

well in the context of python it makes some sense to use it for teaching and presentation purposes, even tho it's a bad habit to program in it in my opinion, as - it's a pain to write abstractions, as you have to remember to reevaluate the cell - you can't use the debugger - you can't jump to the definition of externally defined stuff, which is critical if you want to become a really good developer, as it's teaching you to read and understand code

@mschuetz there are decorators for everything, to time functions, to cache results, to cache results, to cache results for 1h and get fresh data after one hour, to cache results given certain parameters

but in the context of go I don't see the point, as nobody sane will ever do an experimental data analysis with it

go is excellent for high performance data processing, but you definitely don't want to use it for data analysis.. I'm not even sure if a go based dataframe implementation would be that much faster than pandas as it's more or less a wrapper for c functions

I believe it's intended for teaching and guides, not development. For example, I'm currently working through a data science course which uses Jupyter notebooks to allow interactive code to be embedded within the course materials, instead of being kept in separate files. The student can read the material and experiment with the code without switching to another window.

I just don't see Go ever becoming a good language for numerical computing. There was such a push with implementing [multidimensional arrays in Go](https://github.com/golang/go/issues/6282) but it looks like it's been shot down more than once.

When using Python and R interactively, the syntactic sugar and functions that allow for rapid analysis (subsetting, indexing, merging data objects etc etc) more than make up for the slowness of the language. If someone wanted to implement that workflow in Go for repeated analysis of very large datasets, I doubt they would need it to be interactive.

But it looks like Julia might become the goto numerical language that is compiled and fast.

This isn't really mature enough to be considered an equivalent to pandas, but that seems to be the aspiration: https://github.com/kniren/gota

If this were possible, it would require the blackest of black magic, and mapping anything other than primitive types across the void would be essentially impossible, e.g. anything useful like a DataFrame.

One could implement a kernel that forwards calls to other kernels depending on a prefix and explicit push-and-pull functions to move variables around. That last part would require an extension of the protocol or a parallel channel, but the forwarding kernel is actually not that difficult to implement.

In the simplest case, you would parse the text output of a previous cell for the other language.

That has the problem of sounding like "judgemental" in French ("jugeote")