Show HN: Starboard – Fully in-browser literate notebooks like Jupyter Notebook
starboard.gg
starboard.gg
Cell-by-cell notebooks like Jupyter are great for prototyping, explaining and exploration, but their dependence on a Python server (with often undocumented dependencies) limits their ability to be shared and remixed. Now that browsers support dynamic imports, it has become possible to create a similar workflow entirely in the browser.
That motivated me to build Starboard Notebook, a tool I wished existed. It's:
* Run entirely in the browser, there is no server or setup, it's all static files.
* Web-native, so no widget system is necessary. There is nearly no magic, it's all web tech (HTML, CSS, JS).
* Stores as a plaintext file, it will play nicely with version control systems.
* Hackable: the sandbox that your code gets run in contains the editor itself, so you can metaprogram the editor itself (e.g. adding support for other languages such as Python through WASM).
* Open source (https://github.com/gzuidhof/starboard-notebook).
You can import any code that targets the browser directly (e.g. puts stuff on the window object), or that has exports in ES module format.
I'm happy to answer any questions!
I've been working on something similar on and off for a while, and I've found the space to be fascinating and somewhat puzzling. Most of the first questions I try to ask about these in browser notebooks don't have clean answers. In no particular order:
- Do you have any ideas about solving logic errors? As it stands right now a while(true) loop crashes the page and consecutive re-openings of the page for a short while (in chrome). What about things like document.body.innerHTML = "" (you actually seem to handle this pretty gracefully, I'll see if I can poke more holes in this another time). For what it's worth none of the online notebooks I've seen have a satisfactory solution to infinite loops, loop timeouts are too blunt an instrument and crashing the tab can take the browser minutes to recover.
- Is there a particular reason you use eval rather than the new Function constructor? From what I've read using new Function is much more performant, and other than different scoping a better choice than eval. Can't find the link at the moment, but it was wrapped up in the mdn[1] design docs for their codebox examples.
- I see you're using the first codemirror 6[2] beta release, how are you liking it? I really enjoy the interface so far.
- Do you have any favorite resources or inspirations about why you went about building a hackable offline local first notebook environment? I particularly like the experiments at Ink and Switch[3]. (many related hn submissions). As well as webstrates[4].
My attempt at this game is to break out of the notebook style single column layout and embrace an art board style canvas, which is a rather radical idea in that it is not obvious what that should look like or how basic interactions like hierarchies or execution order should look, but fun to explore. I also desperately want to prevent crashing as a result of logic errors and workflow footguns(like deleting DOM elements or overwriting storage), to that end I have a separate storage of scripts to rebuild a "safe boot" interface, but there is more thinking to be done here.
I look forward to poking around your code some more soon. Thanks for posting.
[1]: https://developer.mozilla.org/en-US/ [2]: https://codemirror.net/6/ [3]: https://www.inkandswitch.com/ [4]: https://www.webstrates.net/
Edited out a question about your motivations. I reread your comment and the about page and I realized my motivations are similar to yours. While observable doesn't have a few of your key points, it is a fantastic product. The reason I don't settle on it is that I'm interested in experimenting with the environment outside of a notebook-with-cells interface.
> [...] I also desperately want to prevent crashing as a result of logic errors
I could be wrong here, but I think the new Site Isolation stuff in Chromium[0] may mean that you can sandbox an iframe to a different origin, so that the main threads of the top frame and the iframe aren't synced. So the iframe where the code is running would freeze, but the main frame would still be responsive. Again, I may have misinterpreted this because I only read about it in passing, and it was a while ago.
[0] https://www.chromium.org/Home/chromium-security/site-isolati...
Web workers are interesting and relevant to this use case, however the almost total isolation doesn't lend itself to manipulating the DOM without the user explicitly writing a custom API to interpret the worker's messages. So, back to the same problem.
Thanks for the link.
I'll do some digging into chrome's sandbox[1]:
[1]: https://chromium.googlesource.com/chromium/src/+/master/docs...
I also want to leave the notebook style, focusing more on graph approach. Think higher order spread sheet without the grid.
One, somewhat tangential, thought I’ve had to handle the infinite loop issue though. Is to fragment the language somewhat like they suggest in the out the tarpit paper. Trying to avoid infinite loops by construction, only used at the highest orchestration layer to schedule execution blocks, where the runtime can keep track of things going out of control.
Not sure what it would take to extract a “total” subset of javascript though. But one idea would be to insert a trampoline breaking out of infinite loops often enough for error handling.
All good ideas, thanks for sharing.
- I think logic errors can't be solved or prevented when exposing vanilla HTML, CSS and JS. You can completely break the notebook if you want to. Like the sibling comment to this stated: the notebook is run in an iframe of a different origin so you (should!) not be able to break out of that iframe. But of course if you introduce some endless loop that crashes the browser (or maybe just the iframe depending on the browser), I can't stop that.
- No particular reason, but it was always a bit iffy getting the return value correctly. To support top level await your code actually gets wrapped into an async function, I don't know how nicely that plays with new Function.
In the future I want to support ES modules with import instead. Perhaps no magic will be required at all anymore and performance will probably be even better (https://2ality.com/2019/10/eval-via-import.html)
- Codemirror is very good and doesn't have a huge bundle size. I decided I wanted to support touchscreens. The monaco editor completely fails on my phone (if I type space it usually copies the word in front of it), whereas codemirror just works. The autocomplete and language support is much worse though, on a desktop I think I will always choose the Monaco editor.
- Probably not the most exiting answer, but for me it was just wanting to use Jupyter for things it was really bad at. There is Project Iodide which is very similar, but they made some different design decisions that move away from the code-output-code-output structure.
By putting the editor inside the sandbox things become much more straightforward and less "special" or "magic", that's another big difference from Project Iodide.
I think more and more fully client-side editors are becoming possible now due to dynamic import and webassembly, do share your project with me if you get the chance to work on it!
I'll have to look more into async stuff. Great eval article there. I've found these four lines to work for importing content[1]. I dove into realms[2] and the realms shim [3] a while ago after reading about Figma's engineering behind plugins [4] but I never figured out how I might use that for my system.
You might take a look at how the codemirror author built his book [5] and editor [6] in this repo [7]. He implements the loop detection I derided in my first comment. I shouldn't be so harsh, it's a good and simple idea. It's just not an answer, more of a hack. My interests align pretty well with the p5js editor audience[8], so some of the early issues there convey why the simple loop detection isn't ideal.
I'll drop you a link soon!
[1]: https://stackoverflow.com/a/5282268 [2]: https://github.com/tc39/proposal-realms [3]: https://github.com/Agoric/realms-shim [4]: https://www.figma.com/blog/how-we-built-the-figma-plugin-sys...
[5]: https://eloquentjavascript.net/ [6]: https://eloquentjavascript.net/code/ [7]: https://github.com/marijnh/Eloquent-JavaScript/tree/master/h...
[8]: https://github.com/processing/p5.js-web-editor/issues/174
And that drawback obsoletes the strategy it seems, unfortunately.
It's an interesting problem, how to provide full access and control as well as non-corruptible recovery systems.
One of the biggest problems I consistently run into is that printing to the screen is a slow operation, every learns this day one of any graphics work, but things like console.log or Node.appendChild are the primary way to destroy someone's computer for a short period of time because orders of magnitude more operations are queued up before they can complete. I wonder if painting operations could be limited to some max number per second to prevent that orders of magnitude queue problem. And, more importantly, how do you code that limit?
For node.appendChild you have the whole virtual DOM thing leveraged by modern JS frameworks. If you hit issues try using any framework that boasts having a virtual DOM.
The same strategy could be done for the console to some extent.
The built-in console object is over-writable so you can add a layer that would batch console calls (if you do many in a short time). Note that this will break the call address you have on the right-hand side. Also IE11 does strange things with the console object depending if devtools are open or not.
The solution above is doable but it's a hack. If you really have to call console a lot of times in a short time adding a custom log method would be better.
globalThis.__debugLog = [];
globalThis.__printLog = () => globalThis.__debugLog.forEach(i => console.log(i));
const log = (...args) => globalThis.__debugLog.push(args);
// then in code:
log(whatever);
// and once you want to check it, call printLog in the console
__printLog();
Again with this solution you don't know where the call happens but it's a starting point. I recently started using the pattern above to debug performance and memoization issues I had in heavily used code. The console wasn't readable anymore without it. Console.table is helpful for printing out bigger sets of debug data btw.For canvas setTimeout on requestAnimationFrame seems to do the trick https://stackoverflow.com/questions/19516307/limiting-framer... I don't to much with canvas besides using it indirectly with libraries.
That's an interesting perspective I hadn't thought of before, I'll see if that can inspire some design pattern. Thanks for another good prompt!
Note that it does not mention promises and async/await but it does include it. Promises are essentially syntactic sugar for a setTimeout calling success and failure methods. This also means that promise code will be executed in parallel but on one thread and one heap, so each promise will fight with all other for priority. This is generally fine for async stuff like fetching resources but it would completely destroy performance if it would do anything heavy, like, say unzipping zips with jszip.
Historically the event-driven model of JS enabled Node.js's API to be callback-driven. This lead to what is called "callback hell" and eventually to a proper introduction of promises and async/await to the language.
From the article:
> "Run to completion"...
> A downside of this model is that if a message takes too long to complete, the web application is unable to process user interactions like click or scroll. The browser mitigates this with the "a script is taking too long to run" dialog. A good practice to follow is to make message processing short and if possible cut down one message into several messages.
Good advice. And this is what happens when you `while(1) {console.log("loop")}`. Though I've never see a "taking too long to run" dialog, I'm just eventually able to kill the process myself and sometimes it takes a few minutes while my fans spin.
Observable's html templates were inspired by lit-html, but they ended up implementing their own library. I'm glad the lit-* stuff is useful to you, and really excited to see it available in cells. I'll have to try this out!
Also, have tried PureScript? e.g. [1]
One big difference to the projects you listed is that lit-html doesn't require a compiler. It's plain JS, and is fast because it uses template literals and built-in HTML parsing/cloning to associate expressions with DOM.
Without lit-html Starboard would be nowhere near as flexible or elegant. Starboard has to be completely customizable at runtime, that would probably be difficult to achieve with anything that has a build step, with lit-html it's "just Javascript" all the way down.
If you usually use a framework like React, Vue or Svelte it is still worth trying out lit-html, they are not mutually exclusive, it's simple to learn and deceptively powerful. It's great to have it on your toolbelt.
- did I understand correctly that notebooks would be hosted (at least stored and served) on the Starboard website? (you mention it's kind of a github, in the About section) What does this imply for private notebooks? Are they possible?
- any example on how to run python there with pyodide? I didn't understand by quickly looking if that integration is part of future plans or it's already a possibility.
- I read that not all features are ready at the moment, for example exporting to a proper website. Any forecast on when will features like this one ready?
Thanks!
Answering your questions:
- If you log in and create a public notebook, it's stored on Starboard's server, then you can just share it by link. If you create an offline notebook it's stored in your browser's LocalStorage. Right now you can't create a notebook yet that only you can see or can be shared with a private link, to do!
But of course you don't have to use the Starboard website, you can just save it as a text file on your disk. In the future I want there to be a CLI tool that serves that notebook for you on localhost.
- No examples yet, it's future plans for now (but I hope to land that in a week or two!)
- No forecast.. It's just myself working on it, so you'll have to bear with me. I am working full-time on this.
One thing is I can't really find docs - I want to add RxJS on the page - is this possible?
Edit: Oh nevermind, I see it in the screenshot on the Github page with the import
Edit 2: Oh actually it's not working:
TypeError: Cannot set property 'rxjs' of undefined
If I try import { interval } from '...' I get: SyntaxError: 'import' and 'export' may only appear at the top level (1:14)E.G. const foo = await import('foo.js');
It looks like the reason it doesn't work here is because it relies on "this" being the window object. To make top-level await possible your code gets wrapped into a function, and in there "this" no longer points to the window. I am sure a workaround is possible, I'll make a ticket on github.
Can it be combined with https://svelte.dev? Would be great for reactivity and cheaply creating visualisations.
[1] https://github.com/ryansolid/solid [2] https://indepth.dev/exploring-the-state-of-reactivity-patter...
But “wide”, for example, is not internet owned so you could start something scrappy called “wide” with the hope that one day you can internet own it.
Starboard is definitely not internet owned so if you do decide to drop it at least know that there is no reason to do so because of external optics.
Also note, there is Probably no other brand that you could choose that didn’t already have a lot of folks using that brand already.
In the end the market will choose the winner!
All that said, this version of starboard here does look pretty cool. I don’t mean to upset anyone. But I did want to explain this point of view on brands for indie founders.
BTW What’s your product all about?
Of course I have lot's of other questions like "How you gonna make money?" but this is probably not the time or the place ;)
Pretty cool. As someone involved in building a machine learning platform which also allows users to leverage Jupyter notebooks, I feel your pain. We had to add many features to make working on non trivial projects more feasible. For example near-real-time collaboration so people could pair-program on one notebook.
One other problem you mention is the kernel in the backend. The problem with that is often found in issues where a user launches a long-running notebook, closes their tab/browser. The computation happens but the results don't find their way, which means users must have code for side effects and artifacts in the notebook.
We've added a functionality to run notebooks asynchronously to address that problem, which also addresses the problem of multiple users competing for limited resources (tragedy of the commons) and kernels dying. Now users can run their notebooks asynchronously: they're scheduled, run, and saved seamlessly.
Jupyter is a useful tool which has these problems that aren't trivial to solve. For example, for the near-real-time collaboration, the Jupyter team is revamping their API with Lumino (https://github.com/jupyterlab/lumino), which was PhosphorJS but the development stopped when the main contributor had retired.
Are you building a product that you plan to release? Or are these contributions that are available as open source components?
In any case, I'd like to know more, or to have some information on relevant topics that might be useful to understand and find workarounds for these issues!
>and resuming running notebooks whose frontend was closed.
The platform schedules the notebooks, runs them, and saves the results with the output[0]. You can still check the progress, status, and position in the queue of the notebook if you wish. You still get the results after it's done even if you had closed the browser.
>Are you building a product that you plan to release? Or are these contributions that are available as open source components?
Well... We're a machine learning shop, present in Algiers, Algeria and Paris, France, that's been building custom turn-key products for large organizations for quite some time. As you know, non-trivial projects with real stakes are a bit more "challenging" than portfolio projects. Tracking rationale of projects, tracking metrics/params/models, shared access to data, deploying and managing models, challenges in collaboration, setting an environment and keeping things from breaking, or needing a beefy machine.
We started building the platform to allow our teammates to remotely work on projects involving machine learning, as commuting is hard and having contributors in different countries can be challenging. We also wanted to simplify access to a larger talent pool from other countries, which means we had to have a way to work together. We also wanted self-service: our data scientists had to sollicit the help of other engineers to either use some language specific feature, or to deploy their notebooks. We wanted to allow them to do so themselves in one click so they don't wait, and other engineers can work on other things. We wanted the results to get as soon as possible to the domain expert, and allo that domain expert to be able to train models themselves with no code experience, by exposing parameters that are relevant to some sector, and allow them too to deploy a model. In many cases, a domain expert would chime in to tell us that some variable that was dismissed was actually critical at some point in a decision-making process. So we wanted that, too.
We also wanted developers to interact with the models with HTTP requests period. We kind of suffered from having too many simultaneous projects with different stacks.
We needed a way to execute projects efficiently and effectively, but we didn't like other products. We also wanted extensibility. As I said, we built whole products, not just the models, so we needed a way to plug functionality in.
We're actively developing this.
>In any case, I'd like to know more, or to have some information on relevant topics that might be useful to understand and find workarounds for these issues!
Sure, shoot us an email at the site below or using the info in my bio.
It's half abandoned now. one issue of js based notebook is the missing of a file system. data scientists load huge data into notebooks, this is hard to do on the web. There is the new web filesystem api, but that only helps load local files.
It's not a silver bullet. For larger data you can't afford to download it every time. That tends to balance itself out though, because you don't really want to store huge data on your laptop anyway, so you're going to have to find a different solution for that anyway (such as using Jupyter). You can accomplish quite a bit with streaming + sampling, which is the crux of how all our apps work in iobio.
The jyve project issues discuss various ideas for repacking Python packages beyond the set already included with Pyodide and supporting loading modules from remote sources.
https://developer.mozilla.org/en-US/docs/Web/Security/Subres... : "Subresource Integrity (SRI) is a security feature that enables browsers to verify that resources they fetch (for example, from a CDN) are delivered without unexpected manipulation. It works by allowing you to provide a cryptographic hash that a fetched resource must match."
There's a new Native Filesystem API: "The new Native File System API allows web apps to read or save changes directly to files and folders on the user's device." https://web.dev/native-file-system/
We'll need a way to grant specific URLs specific, limited amounts of storage.
https://github.com/iodide-project/pyodide :
> The Python scientific stack, compiled to WebAssembly
> [...] Pyodide brings the Python 3.8 runtime to the browser via WebAssembly, along with the Python scientific stack including NumPy, Pandas, Matplotlib, parts of SciPy, and NetworkX. The packages directory lists over 35 packages which are currently available.
> Pyodide provides transparent conversion of objects between Javascript and Python. When used inside a browser, Python has full access to the Web APIs.
https://github.com/deathbeds/jyve/issues/46 :
> Would miniforge and conda-forge build a WASM architecture target?
> Emscripten or WASI?
So, without a kernel that can do heavily lifting, you're limited to basically what you get when have the browser console open; ie. html + js.
Is that the idea? A better / structured browser console?
Or am I missing something?
I mean, I completely appreciate the benefit of notebooks; but if you're limited to running code inside your browser window sandbox... how do you do anything useful with it?
How do you import a dataset?
> const data = await fetch(`https://api.exchangerate.host/timeseries?start_date=2020-01-...
Is great for trivial public datasets, but CORS is going to forbid you from loading any personal/private data right?
unpkg can give you public packages, but if you have any personal code, how do you use it?
It seems like you basically end up with an empty sandbox.
I am really curious what was the author's thinking process here with regards to solving the problem, what in fact was even the problem. Perhaps the author could share their thoughts.
Edit: grammar
I'm looking at this and comparing it to the situation I find myself in with looking for a syslog collector. I want something that can collect all my syslog and present them in a web UI in a sensible way.
My options appear to be limited to commercial solutions, ELK (Elastic, Logsearch, Kibana), or text files. There's no middle ground. I have ~20 VMs + containers, and no way to view all of their logs in a central sensible manner unless I roll my own (or buy more hardware to be able to run the ELK cluster).
I feel like I must be missing something here; you want to aggregate your syslog data; you have a bunch of files and a web browser.
Now what?
You beat your keyboard against the monitor while your browser sandbox steadfastly refuses to load your local file system files? [1]
You write your own server that serves data and the open source core? [2]
Yay, you now have the dubious honour of having re-invented jupyter notebooks, only, you had to roll it yourself, and it's insecure and really not very good compared to an existing mature product.
What can you actually do with this, that you can't do by right clicking on your browser window and picking 'inspect', and then clicking on 'console'?
You can certainly make interactive blog posts; absolutely, I 100% acknowledge that as a blog plugin, when you have already got a server to host your content, this would be pretty awesome.
If you do want to run your own databricks / notebook SAS, then I also see it as being a really great kickstart for you, absolutely.
However, out of the box? I'm... like, I think this is interesting, but I don't see how I could actually use it.
[1] - https://stackoverflow.com/questions/50007055/fetch-request-t...
As for interacting with local files: I imagine a CLI tool that spawns a webserver that will serve local notebook files and perhaps the files in folders you point it at. Or: you generate a notebook file with the data baked into it (as a JSON blob or something).
As for security: I think the browser sandbox is a lot more safe than running Python (Jupyter) with a lot of dependencies you will have to verify.
And finally: you need to be able to get the data into the browser. For a company the most 'scalable' would probably be self-hosting Starboard and (setting up CORS to) allow reading from various datasources from your internal network, or otherwise authenticating the requests from notebooks.
Another thing that has worked well for me in the past with other web tools: support drag and drop. You drop in the file you want to visualize.
In the end Starboard notebook is a tool for creating a small website that you can instantly share, without needing a backing Python server always on, and without the need of any build tooling (webpack, what have you). It's good for small web tools, articles and tutorials. I hope this makes it a little bit more clear!
The ability to basically drag and drop a file (eg. csv) onto the notebook to vendor it as part of the static notebook that other people could then access would be the bridge that I was looking for and couldn't find.
F12 or Ctrl+Shift+i and Ctrl+c Ctrl+v and you're executing code in the context of the post ;).
It's not a notebook but does the job well enough.
I wonder how much value there is to this particular use case because it lives somewhere between devtools and an online IDE. Also arguably testing out a library in a codepen is more comfortable than an inline which you potentially can't save.
Then what do you do when someone is demonstrating how to reset a page, whether that be document.body.innerHTML = "" or document.body.removeChild(main). They could be run in sandboxes, like starboard here in iframes. Or a powerful history tool can be attached to the page. Every program takes an environment as a parameter, maybe we can make it as easy to shuffle around them as playing with cards.
To really simplify it down, I'm exploring the space of digital workspace + shell. What do you get with a compute environment in OneNote or Figma or Sketchup. What do you get if you can position things in a freeform space rather than a filesystem and text based browser? Can I build a Sketchpad with code editing, and take advantage of viewport macros to build a space of my project and explorations?
Makespace.fun + jsfiddle/observable seems like an environment that I'd like to work in with myself, my collaborators and arbitrary content. To play with things like history and alternatives, and associations of material and ideas.
I have a good example of success of programmatic control of content and auto layouts working with my tiling window manager. Being able to explicitly describe compositions and demonstrating computational content seem to be targets not yet hit by the pop culture of personal computing.
I might be just blowing hot air with all this, but it is fun to play around with.
Observable cells are basically Javascript, with a small handful of additional conveniences (most importantly: they can reference other cells reactively) which make a huge improvement in the notebook context, and make Observable significantly more pleasant than any other notebook environment (Jupyter, Mathematica, etc.).
Javascript libraries and code snippets can be used as-is in an Observable notebook, and anything in Observable cells can be straightforwardly ported to vanilla JS at the expense of a bit of effort and slightly uglier code. (Or the Observable runtime and standard library can be used directly externally.)
If I write a macro in Common Lisp that packages my code in a lambda, establishes a bit of dynamic context around its execution, and perhaps walks the code itself to modify some of its contents, this doesn't make uses of that macro "not Lisp". Yet that's what's happening here, just with less elegant machinery as JS doesn't have proper macros.
* Observable has quite some magic, it's not just Javascript (there is a compile step to make it possible, see the other comment). With Starboard the plan is to make it trivial to export as a static website, in the end Starboard is just away to make a simple website without too much magic that you can instantly share.
* The reactive system they have there is powerful, sometimes probably a bit too powerful. Starboard is just plain javascript (for better or for worse). You can run cells out of order and have complete freedom. Any restrictions on the order that cells should be executed and re-evaluated will have to come from external plugins.
* Observable is a walled garden, you can't export the notebooks into say a text file and use it independently (as far as I know). They own the ecosystem and you can't really move away from it (and fair enough, it's a commercial product!). Starboard notebook is fully open source and it is built with customization, vanilla-ness and portability as main features.
I'm the founder of Deepnote (https://deepnote.com/). We're working on a data science notebook (primarily focused on Python), happy to share what we've learned. I couldn't find any contact info on the website, feel free to message me at jakub@deepnote.com.
Have you looked at nbconvert with the ExecutePreprocessor? I use it (with a light wrapper) from cron for sending automatic email reports and generating static dashboards. It might do what you're looking for.
Also, I hadn't realized there was a category of 'Open Startups', that's pretty cool.
I don't have to tailor ads to the visitor or something, I just want accurate enough stats in aggregate.
Good job!
You planning to seek funding to work on this full time?
I am not sure what makes the most sense in terms of monetization in the longer term. I think it's a difficult for many open source projects.
check TiddlyWiki
I'm honestly just happy that Literate Programming is being hacked on, shared, discussed, and criticized. Living documents just seem so much closer to the promise of hypertext and the original web than what we have now.