I'm waiting impatiently for the coming revolution of non-web collaborative internet-connected rich client applications.
Of course, it's not web-based and it requires some basic knowledge of Emacs, but functionality-wise is so much better that it ends up being frustrating to use Jupyter when collaborating with other people. There are some workarounds (EIN, and ob-ipython modes help, but it's not quite the same)
1. Plain text format, git and git diffs work
2. You can combine many languages in a single document, and every code block can be part of a separate session, as an analogy to Jupyter Notebooks, you can have multiple kernels backing a single notebook and you can decide what kernel you want the current code block to run in.
3. You can edit a code block in the major mode for that language, i.e. you get all the features of Emacs while editing code: documentation, auto-complete, snippets and anything Emacs can do, and Emacs can do a lot :)
4. You can have internal and external links to any part of the document (or any other org-mode file) within the editor which get exported as links in the HTML file too. Want to refer to a code block you used before, just name it and drop a link. Extremely useful in binding the whole document together.
5. Literate Programming support -- You can decide the order the concepts are introduced in according to the human reader, not according to the execution order the machine demands it to be in:
#+NAME: named_code_block :eval no
function_not_defined_yet()
#NAME: complete_code_block
def function_not_defined_yet():
print("nice function innit?")
<<named_code_block>>
6. With the internal and external links and <<named_code_block>> (NOWEB syntax) the org-mode file is closer to being a hypertext file than Jupyter Notebook even though Jupyter Notebook is the one running in a browser.
I have covered only the major features of org-babel, I haven't even covered all the features. I love Jupyter Notebooks too, but org-babel is something else. I am currently working on a toy ray tracer in Clojure in literate programming style and loving every moment :)
I'd add that you can also benefit from other aspects of org, such as project management functionality, outline editing, table editing, tables with formulas, direct git integration via Magit, etc., etc.
I agree that getting people on Emacs is a non-minor issue, but using something like Spacemacs with CUA-mode enabled could go a long way towards acceptance.
Just to give an example of what can be done using org-mode, this is the project I am using to grok Literate Programming -- http://ray_tracer.surge.sh/
The whole thing is generated using one org-mode file -- https://gitlab.com/snippets/1710454
and this org-mode file is the one I work in, it will eventually generate the source code in separate files too, once I have finished the project.
This is what my setup looks like -- https://i.imgur.com/mqi8vPR.png
Anyways I can't convince you to use it, but hopefully I can convince you to give it a try, it isn't easy but it is worth it :)
You can do this with RMarkdown as well, despite the name, and run notebooks in RStudio. Much better experience than Jupyter in a browser.
Except engine='R' (default), all chunks are executed in separate sessions, so the variables cannot be directly shared.
This feels like the argument against Slack in favor of IRC. I don't doubt it's true for you that org-babel is better, but there isn't really even that much comparison between Jupyter and org-babel, they are very different things.
One of the more compelling reasons to use Jupyter is inline images and plots. Another one of the more compelling reasons to use it is it basically comes bundled with everything you need, python with it's own virtual environment that doesn't modify your system python, and a bunch of amazing libraries, numpy, scipy, matplotlib. Not needing to know emacs helps too, but that's really low on the list.
> Jupyter uses a rather obtuse js on format which is not practical to work on directly.
This is a wierd thing to say. Jupyter has download as .py right in the file menu. Not to mention download as Markdown, HTML, LaTeX or PDF, if you want to present or share instead of export the python.
You can do those in org, since forever. I meant that the files themselves are plain text, which is good for sharing, version control, etc. ipynb files embed images with Base64 encoding, which makes seeing diffs a major PITA.
>This is a wierd thing to say. Jupyter has download as .py right in the file menu. Not to mention download as Markdown, HTML, LaTeX or PDF, if you want to present or share instead of export the python.
You can export org to those formats and more, but the main part is that you can organize your code (e.g, tangling to different files, etc.), and use other languages than python (or whatever other kernel you're running in Jupyter).
Plus, you get to use a proper text editor instead of editing on a browser textarea.
It sounds like org mode does not store images inline, since you don't like the b64 images in Jupyter? I imagine that could go both ways depending on what you want. Even for diffing, you might like to see that the image changed (though I generally agree that diffing images is not ideal). If you have to package images separately when sharing, then wouldn't inline images score a point for easier sharing?
> use other languages that python
Agreed, and that is powerful, but this also goes to my point. Jupyter is made for python and org mode isn't. Jupyter comes with python and libraries, where with Emacs you're on your own to figure out how to install and use different languages, and setup your environment so Emacs can see them. You're on your own to bring in images and plots, where Jupyter already understands commands that output images.
> Plus, you get to use a proper text editor instead of editing on a browser textarea.
Yes, but the editor is Emacs. :) For a lot of people the browser textarea is a bonus because it's the same thing you use everywhere. It's uncomplicated and non-technical and utterly consistent, even if lacking any power.
I guess that depends on who you're sharing with and how. If you're using git, you can just add them to your repo (only they won't clobber your diffs). If you're just publishing, you can export to HTML and put it somewhere online.
>Agreed, and that is powerful, but this also goes to my point. Jupyter is made for python and org mode isn't. Jupyter comes with python and libraries, where with Emacs you're on your own to figure out how to install and use different languages, and setup your environment so Emacs can see them
Well, that requires a minimum of computer literacy, but rocket science, it ain't. I agree that Emacs is not for everyone, but if you're doing data analysis (and presumably something with the results), it's certainly not out of reach, and good tooling is worth learning.
I agree that Jupyter is "easier", but it's not simpler. And also, it's limiting in what it can do. So yes, you might get up and running a bit faster, but you're setting yourself for frustration down the line.
> Yes, but the editor is Emacs.
Precisely, it's vastly better than the alternative. If you don't like the bindings, you can use Evil (this is my choice, despite never having been a heavy vi user), or CUA mode, and get a more familiar experience for people used to Windows.
Adding git as a dependency to sharing is too much in general, that would prevent sharing.
I'm fully aware of that, my point was that functionality- and ergonomics-wise, you're better off using Emacs/org. I agree that it's not for everyone, but for people working on data science, it's a much more powerful tool.
>Adding git as a dependency to sharing is too much in general, that would prevent sharing.
I think we probably have vastly different target audiences in mind.
I have a shortcut that lets me edit any browser text area in MacVim, which works out really well in the Jupyter notebook.
What I wonder is why are Jupyter notebooks so popular? I've tried them, but perhaps I don't understand the workflow in the way people use them. Are they intended to be works in progress where one explores ideas and looks at results? This is how I've usually used them and Mathematica's notebooks as well.
For me though, my Jupyter or Mathematica notebooks end up with lots of false steps and dead-ends. To prepare a presentable form of the results I have to go back and edit the notebook to such a degree that it doesn't feel like working in a notebook as much as it is working in a complex document with a really weak set of editing tools. To be fair, I haven't used Jupyter or Mathematica for a couple of years and things may have changed. Maybe HN readers can help be understand how to more effectively use these tools.
I often wonder the same. I'm now kinda forced to use them, since they're the tool of choice in my team, but it's a major pain. I think that they're "easy" (in the way Rich Hickey describes "easy" vs "simple") to get started with (just as python, btw), but their lack of flexibility comes back to bite you sooner or later.
For me, I've resorted to using a mix of EIN and ob-ipython, and then copy/pasting as needed to keep the stuff in synch with Jupyter. I'm now considering writing a simple ox-jupyter exporter, and maybe some import functionality, to avoid dealing with Jupyter directly.
John Kitchin has written a basic exporter (http://kitchingroup.cheme.cmu.edu/blog/2017/01/21/Exporting-...), but it doesn't support inline images (it should be simple to fix this), and it doesn't really adhere to the ox- style of exports, so I think I'll probably do my own instead of adapting his (it'll be a good learning experience, for sure).
How easy is it to share a Jupyter notebook with someone else, versus sharing an org-mode notebook? Think about the recipients rather than how hard it is for you. Assume the person you share with needs to be able to re-run your notebook. Assume the person you share with isn't using the same operating system that you are. Be honest and think about all the steps involved.
With Jupyter, I send a single install link to Anaconda. With Emacs + org mode + python + numpy + scipy + matplotlib, I might not even know where to begin if the recipient is using Windows or Mac. The recipient needs to install emacs, and they need to know how to use Emacs. They have to know how to install python packages (and possibly a package manager), and ideally be able to use virtualenv too.
If I need different libraries than what Anaconda comes with, I can put the commands in the notebook, and I won't mess up the recipient's system version of Python. With org-mode, they'd need to figure it out on their own.
If I choose to use a different language than Python in my org-node notebook, the recipient will have to install that programming language and environment themselves before they can use your notebook.
It's the simplicity and bundling of dependencies that make Jupyter popular. All the flexibility and power you get with org mode comes at a high price.
Well, now you're talking about a whole environment. Use docker, or VMs or whatever. There's plenty of options for doing this (we actually use Docker for Jupyter, to make sure we work off a consistent install, so I guess sharing can be non-trivial if you walk a bit off the beaten path, anyway)
> If I choose to use a different language than Python in my org-node notebook, the recipient will have to install that programming language and environment themselves before they can use your notebook.
Sure, and if you need to use another language from your Jupyter notebook, you can't. I don't get your point. Yes, if you need to use a tool, you need to install it. It requires a minimum of computer literacy.
> I don’t get your point.
My point was Jupyter inflicts far less technical burden and debt on other people. There’s a huge downside to custom environments. If that’s not obvious, that explains why it’s hard to understand jupyter’s popularity.
> it requires a minimum of computer literacy
I’m sure you didn’t mean it that way, but this borders on white tower programmer centric dismissiveness.
Personally I know how to install all the things an org mode notebook might require, but I don’t want to. It takes too much of my time and energy. Expecting other people to adopt your choice of tools and environment by arguing that your tools are basic literacy is presumptuous. Use org mode, I’m wildly in favor of people using tools that they like, and you like it. Just don’t claim it’s objectively better on all axes when it’s not.
You're again comparing apples to oranges. If you want to use Jupyter, you need to install a whole lot of stuff (easier to do if you're using Anaconda, agreed). It's "not simple" enough that there are public docker images for it. My point is that when tooling is an issue, you can do the exact same thing with an org-based setup.
>My point was Jupyter inflicts far less technical burden and debt on other people. There’s a huge downside to custom environments. If that’s not obvious, that explains why it’s hard to understand jupyter’s popularity.
I agree that there's difficulty in achieving reproducible environments, and that's why there's tools that handle that for you. I think that much of Jupyter's popularity comes from the fact that it's easily available, and based on Python, which let's non-programmers get started moderately quickly. I just don't think that it's the best choice. This is not to shit on Jupyter, I've mentioned a bunch of advantages of using org (which by the way, you haven't really addressed, other than focusing on the setup aspects)
> I’m sure you didn’t mean it that way, but this borders on white tower programmer centric dismissiveness.
It's not, really. My point is that when you're doing presumably advanced analysis on presumably complex data, you (should!) certainly have the capacity to deal with tooling.
Not entirely true, BTW.
https://blog.dominodatalab.com/lesser-known-ways-of-using-no...
If they want to actually work on the notebook and keep their changes, they'll probably need to install it (or switch to a platform like Cocalc). But Binder is enough to let them view, run it and make experimental changes, without installing anything or creating any new account.
Seen that way, sharability is just a nice bonus. Yeah, I'd never try to turn anything directly from a jupyter notebook into a serious way of sharing research results---but for a quick and dirty "here's what I did" on a blog or email or something, or to show students, it does the job.
Kind of odd, but repl on steroids is really just how dim the modern view of the old lisp machines is. Most of our neat repl tricks were just taken for granted on some of those machines.
That being said, it's still pretty sweet!
[1] https://github.com/syl20bnr/spacemacs/tree/master/layers/%2B...
I am really interested in the specific advantages MathCAD had/has, and the problems you see with cells and being in a web browser.
> the coming revolution of non-web collaborative internet-connected rich client applications.
why? how is the web not up to this?
My end goal is a document, and for that I either want a programmer-grade text editor, or a full-featured word processor. Filling in a web form is a terrible way to write a document. My tool of choice at the moment is RStudio - all the interactivity you could possibly ask for, and very easy to edit complex files then render them to a great-looking PDF with the output of my code interspersed with paragraphs of text, footnotes and TOC and all the rest.
MathCAD was later bought by PTC, and further went to shit. It amuses me how some companies can self destruct.
That would be a couple of years after I last used it. What a shame, at least I have those memories. Really loved how low-friction working with it was, and how trivial it was to generate high quality documents or reports, there was no "write it up" stage, you were done basically as soon as you had done the actual work. Then just print and go! Still can't really do that with Jupyter.
Started using the DOS version in 1990 in college.
Bought version 3.1 in ~1992, Windows version running it under OS/2. Could afford to buy it then, even as a student. Ah, OS/2 was also good.
It was < $100 for each version upgrade until 2000, when they switched to maintenance based licensing, soon bumping the price to $495.
I had a home license which I dropped. I still had a work license, but then some sales twerp said I was using a pirated license at work, and wanted to speak to my IT department. This is a >100k employee company, so serious consequences for software piracy. I faxed them my valid license certificate, and told them to stuff it, dropping my renewal, and buying Mathematica.
Now I see it's Mathcad Prime, a subscription based license, for $800 year. They were purchsed by PTC several years ago, which only made things worse.
This is textbook case of screwing over your user base, cha$ing corporate accounts. I don't know anyone who uses it now. Many people used it back in the day.
Sad thing is I don't even use Mathematica much. Too painfull most of the time. I don't like the interface, but it's better than browser based Jupyter. Just forget trying to install Jupyter and dependancies on a corporate managed Windows computer. I just use papaer and calculator. I swear things have regressed from 20 years ago. At least I can still buy a new RPN calculator.