Dataflow, a self-hosted Observable notebook editor
observablehq.com
observablehq.com
I adore Observable notebooks, but the one thing that makes me hesitate in using them for everything is that the editor component itself is closed-source and only available on https://observablehq.com/
They're great open source ecosystem supporters - they released their runtime, their parser, their standard library and all sorts of other stuff through https://github.com/observablehq - but the editor itself is their proprietary sauce.
I totally support their decision on this - it's what they're building their business around, and I want them to be successful. But as a user it does give me pause.
This project from Alex Garcia looks like a fix for exactly that. Having more-than-one editor for their notebook format (and an open source option a that) resolves my hesitancy in leaning hard into their ecosystem.
I don't even see it as a competitor to ObservableHQ - the hosted Observable editor has collaboration features that don't even make sense for a local running version.
Plus, Dataflow has some great ideas of its own - in particular the live file attachments thing.
Just in case this is of interest to others, some other open source browser-based computational notebook tools include:
* Starboard https://starboard.gg/ * And of course there's always Jupyter, but it requires a server component
And this isn't the same thing, more of a javascript playground (open source alternative to codepen and the like), but see also Slingcode: https://slingcode.net/
But it's not clear that this new project is a fix for that: there's no license file or licensing notice on https://github.com/asg017/dataflow and no mention of licensing in https://observablehq.com/@asg017/introducing-dataflow. So, unfortunately, under Berne Convention copyright laws (which is most of them nowadays) the software is by default restricted by copyright, and looking at it may put you at legal risk, because access plus substantial similarity is deemed to prove copying.
Now, possibly that isn't their intention; https://github.com/asg017/dataflow/blob/main/package.json does say "license": "ISC", so maybe it's just an oversight. But I'd like to see a much clearer and more unambiguous statement of their intent to irrevocably commit Dataflow to an open source license before touching it.
Possibility 1: they are opposed to full automation?
Possibility 2: they don't think software licensing is a potential obstacle to full automation or sufficiently equitable access to the resulting abundance?
Possibility 3: they just have no idea what fully automated luxury gay space communism is, and lack the curiosity to look it up, and so they react without thinking?
Possibility 4: they hate luxury or gay people, so they object on principle to the vision of the future embodied in the phrase?
Possibility 5: they don't think human-factors improvements in software development environments rise to the level of importance implied by my thanks? (But if so, why are they reading this thread at all?)
I'm curious what could possibly motivate this kind of astoundingly hostile reaction.
And an Azure feature: https://docs.microsoft.com/en-us/azure/data-factory/control-...
And a Spring feature: https://spring.io/projects/spring-cloud-dataflow
To add to this, the name of this product is confusing given the context and usecase shown. I assume "dataflow" to the author means the ability to watch data being rendered on a page?
To "big data" folks (like myself), the term "dataflow" tends to represent the routing and processing of data streams along an information pipeline. Not anything to do with a visual representation of a dynamic notebook.
Well that's a bummer. And here I thought I was being very unique :-)
> created: October 15, 2012
nice :D
Still, much more significant is the fact that it seems to be such terrific software that this is a discussion worth having, because it's going to be very influential.
Related, does anyone have any recommendations of a (Postgres) SQL "notebook"? I don't really need any visualizations, more just a markdown integrated doc that allows me to lay out the different queries I use to answer a question.
[1] https://bookdown.org/yihui/rmarkdown/language-engines.html#s...
https://django-sql-dashboard.datasette.io/
In my own work I've been using it for the kind of things that I would normally use a Jupyter notebook for - gathering together research on problems I'm trying to solve.
Shameless plug, we are building a few tools for JS to narrow down this gap as well: - https://hal9.ai (Drag&Drop / IDE) - https://marketplace.visualstudio.com/items?itemName=Hal9.hal... (VSCode extension) - https://observablehq.com/@javierluraschi/running-nodejs-in-o... (ObservableHQ extension)
Would love to chat if you are interested in providing feedback, I'm in javier at hal9.ai. Cheers.
It’s feels like a markdown doc that runs SQL.
https://fluca1978.github.io/2021/01/18/PostgreSQLLiteratePro...
Disclaimer - I work there :)
It’s the same idea except that cells are spread out on a 2d canvas with tabs similar to excel.
I have a farm of jupyter kernels that I can run on demand, and would like to integrate a UI for these kernels on my React website.
I've had a look at the Jupyter default UI but it uses Luminos components which are basically not compatible with React.
Also had a look at nteract components, but their projects seem dead.
Anyone working on something similar? Clean react components to act as UI for the jupyter protocol.
Can other people chime in? Maybe i'm just working at the wrong place.
This thing - Dataflow - is an open source run-on-your-own-machine alternative to the official Observable hosted solution, taking advantage of the fact that Observable itself is JavaScript code with some special sauce that's available as open source runtime/parser libraries.
If someone's environment isn't working, having a button they can click to get a brand new working one in the cloud is an enormous time-saver.
What might be tripping you up is that in this demo the observable notebook isn't showing the code cells, only the outputs. The code is in the editor on the left and the output on the right is the result of running the code as an observable notebook. In some ways it is like a simple interactive web app.
https://observablehq.com/@kragen/plotting-with-a-slider is a really simple example, slightly tweaked from one of Mike Bostock's; you can drag the slider to change the parameters of the WebGL plot displayed below it. This gives you a much faster feedback loop than editing and recompiling in Overleaf. It's way more live than Overleaf.
Also, the document lives at an URL and you can link to it with different parameters. So instead of just being a static document, it's a program that takes its input from the URL, interacts with the user, and lets the user save their work as a URL and send it to somebody else, like https://observablehq.com/@kragen/plotting-with-a-slider?f=si... for example.
And then, as I said, if you want to do a linear regression in LaTeX, you are going to be a very sad panda very soon, because it's possible.
... Done. Stoked to dive in this weekend!