An Introduction to Pluto
lwn.net
lwn.net
They have checks, hints, and interactive plots that update based on your answers. It's really satisfying to get this immediate feedback. The content is great, too, which helps.
Sure, Python is not going anywhere (it is a nice language after all) but it wasn't made with scientific computations in mind and it shows. I think sooner or later it will make way for Julia in that domain (and hopefully also some others).
Beyond that, Pluto has a huge amount of state that's hidden from you in that it tries to keep track of all the variables you define and where so that it can control which cells need to be re-run when you change something. This lets it avoid re-running all the cells when you make a slight adjustment.
However, yes I agree. As I originally said, they added state that makes it work more like it doesn't have state. Or in other words, they solved Jupyter's state problem by adding more state.
There is no hidden state in the editor that can change the values between runs.
> That would contradict the claim in the article that Pluto files are just regular julia files.
Not really, no. The files pluto generates are a subset of valid julia files. All Pluto files are valid julia files; not all valid julia files are valid Pluto files.
When loading Revise.jl in a Julia REPL session, it keeps track of all packages loaded afterwards and if the code is changed in one of the packages it automatically updates the functions provided by that package. As far as I know this does not work (yet) for structures that have been changed though.
The cool thing is now, that it seems as if one can also use Revise.jl in a Pluto notebook - when the code of a package loaded in a Pluto notebook is changed, then also the cells in that notebook are run again with the updated code.
Observable (https://observablehq.com/) is a spiritual successor to Mike Bostock's (d3) https://bl.ocks.org/ that's very useful for data visualization or just general JS-based data exploration.
Here is how we do things (in the context of machine learning as a team working together on a project using our platform[1]. This may not be relevant for solo/portfolio projects with few or no expected deliverables):
We use notebooks interactively just to get something we can schedule as a long-running notebook. We don't run training jobs in interactive mode: we schedule the notebook and it's pretty much fire and forget and no hidden state. It runs against a Docker image, and you can close your browser if you want at that point, we have both the computation and the output as it runs. Or you can keep tweaking the notebook, schedule that tweak, and keep on tweaking. We do that to decouple experiments from human time. Users can publish AppBooks, and then try different params using an automatically generated form from the notebooks so we can fiddle with params in a form without opening the notebook. We do that to be able to try things from a form, as in on our phone or allowing a domain expert to train models without the whole notebook experience overhead. This is particularly useful when we use features that the domain expert wants to be able to tweak. We just expose them and let them do their thing.
We do this to track experiments as we automatically detect models, parameters, and metrics without having to write tracking/saving code. All this allows us to have ready models for deployment, which we can do easily. We do that to allow people to deploy their own models instead of asking someone, and be able to use, and let others use the models simply by sending HTTP requests or using a form, instead of bothering with models and their dependencies.
For the collaboration aspect, we use near real-time collaboration to help someone troubleshoot or go over code with someone, and then commit changes. We automatically save checkpoints so people can go back and forth.
We mainly use Python and we built around Jupyter. We took a look at other notebooks but, in our experience, stylesheets aren't what held our machine learning projects back in the past and we decided to keep using Jupyter adding functionality as "natively" as possible. More embarrassingly, we didn't come from a front-end background but from an ML background, and built this around our workflow to enable us to execute projects.
- [0]: https://github.com/jupyter/nbdime
- [1]: https://iko.ai
The first sentence of the article, excellent. Many times, something is mentioned here on HN and even after clicking sound on a number of with pages, I still don't know what they are talking about. This look at Pluto is exactly what it says: a good introduction to a Jupyter system for the Julia programming language.
Maybe the title of this HN post could change to say that?
To clarify: it is not Jupyter system for Julia programming language. I mean you can use Julia in Jupyter, but this article is not about that.
It's about another, independent computational notebook system written from scratch in Julia for Julia language.