However, this article doesn't mention a newer form of literate programming that has gone mainstream. Notebooks! Whether it's jupyter, colab, whatever the Julia one is called, etc. People use literate programming all the time for data science and machine learning.
(1) No linking from on cell to another. A sequential execution is assumed or otherwise you need to manually run cells in different order. There might be extensions to help with that. It is not out of the box though.
(2) The documentation language is markdown This does not allow for great technical documents with arbitrary links to arbitrary other parts in the document or inside another notebook. ReStructuredText would have that using link sources and link sinks. org mode can also link to parts in other documents.
(3) I cannot include other notebooks to have it all exported as one whole document or notebook as I can in org mode. In org mode I can even specify at what level some heading should be imported.
(4) The editor in notebooks is not that great compared to any IDE or normal code editor or Emacs.
(5) In a typical JupyterLab environment there is only one programming language per notebook. I know there are other kinds JupyterLab environments, which allow for multiple langs inside the same notebook.
(6) The base format of notebooms is JSON instead of plain text like in org mode. This means diffs are harder to read and version control. Again you need extra tooling for dealing with it well (which does exist).
The literate programming stuff of notebooks has all long been there before notebooks came into existence. It is not like that was the advent of a new form of literate programming. In JupyterLab environments it would take lots of fumbling and tweaking to get something comparable to org + org babel.
As a casual rather than power user, I especially dislike the JSON format of jupyter notebooks.
Maybe I should give it another go with Spacemacs because I like the idea of Org mode. That might make the learning curve gentler. I hadn't considered that possibility at the time.
I don't understand why Emacs's features aren't exposed via context menus. You can always switch to the keyboard later as you get good.
Being emacs, context-menus will also be configurable.
https://github.com/emacs-mirror/emacs/blob/emacs-28/etc/NEWS... https://ruzkuku.com/texts/emacs-mouse.html
For base emacs, the learning curve is not as steep as vim. I wonder if your problem was trying to do everything at once?
Regardless, emacs really is more of a long-term investment. It will take you years to be fully proficient. Personally, I have gone away from using emacs (including org-mode and vim emulation), because it took too much time to keep the configuration shiny.
My main issue with org-mode is that it's only really a first class citizen within emacs. I've not found any vim/vscode/etc. plugins that emulates the experience to satisfactory degree, and that lock-in means I've given up on org-mode.
Now I use org mode almost every day and definitely for all kinds of documentation and notes. It might not be perfect out of the box for formula rendering, but even that can be arranged/configured, so that you can switch to rendered formulas inside your buffer and I have done so in the past.
I have not tried to use Doom Emacs or Spacemacs or whatever flavour. I started with standard Emacs and went with that and still use that. If Doom Emacs or Spacemacs introduce too many new concepts or make the learning curve too steep, perhaps trying standard Emacs will be a better experience.
Not sure what use context menus would have in my personal workflow. Perhaps if they could be opened via keyboard shortcut and then I could select menu items quickly, it would be useful.
Simply run M-x menu-bar-mode, or add (menu-bar-mode 1) into your init.el or config.el
Same syntax for tool-bar-mode. Context menu is ootb for me with doom emacs, so that should just work.
It’s hard to imagine experienced Emacsers wanting to lead a project that solves a problem they don’t have, but the community is very friendly so whoever took it on would get plenty of help.
Mainly I think it is a matter of efficiency...on the implementation side. I mean the hard part about learning something like org-mode (or emacs itself) is conceptual not key bindings.
It's this way because org-mode (or emacs itself) is a powerful tool that users use for decades. The tooling is intended to reward expertise and experience. No amount of context menus will change the fact that the tools are deep and conceptual mastery will take more than a few hours (never mind the thirty seconds the typical app has to engage the user).
Org-mode was written for it's author's use. It was written to get things done. Easy onboarding and massive investment in graphical UI for non-experts wouldn't do that. It requires work to learn. That's true of anything hard.
It's not that I don't sympathize...though it probably can be interpreted that way. It's that using emacs (and org-mode) is analogous to playing the piano, not playing the stereo.
Finally, I while I get the appeal of things like spacemacs, I think it's mostly a distraction. The documentation and tooling is by it's nature derivative...it is by it's nature extra work. Even worse it's extra work that's driven by ideology. Worser still, it's extra work driven by the ideology of Vim versus Emacs. And worst of all, it's rational is emacs-is-too-hard-to-learn.
Emacs is hard to learn, but not too hard. It's harder to learn than a sixteen week college course. But that's most things in the working world.
- cells are linked through a dependency graph, so the output you see is independent of the order in which the cells appear;
- each notebook is backed by a plain Julia module, so you can use version control, edit in a real editor instead of only in the browser, and import other notebooks;
- You can create interaction² using HTML inputs, Julia wrappers over those, or go crazy with your own custom JavaScript controls;
- It’s easy to embed LaTeX into the MarkDown, and you can use HTML, so any hyperlinking works.
1 ‘An introduction to pluto’. LWN. Available from: https://lwn.net/Articles/835930/
2 ‘PlutoUI demonstration’. Available from: https://pluton.lee-phillips.org/sliders/uiDemo.htmlI wish it had a sequential mode.
2) I can't understand the desire for RST over Markdown. Between simplicity and adoption, Markdown beats almost all mark-up languages -- maybe even html.
3) You can use the `run` command to link notebook executions and use Markdown linking to link the documentation.
4) This isn't true. I use vim in Jupyter. There's plugins for emacs too. Also, many editors like VSCode can edit notebooks directly.
5) This is a weird complaint. I can imagine a few corner cases where mixing languages in a notebook would be nice...but the concept of a multi-language Jupyter kernel doesn't make much sense from a practical standpoint.
6) There are diff tools such as `nbdime`. Between those and a few git commit hooks to leave your notebooks in a consistent execution state, merging versions notebooks isn't any worse than other files.
In fact, the execution graph logic is handled by the user. When it gets too complex, it's a sign that the notebook itself is too complex, and it's time to move a lot of code into libraries. Notebooks /should/ be conceptually simple.
Then again, I bounced off emacs after deciding the complexity didn't actually justify the endpoint: I was already able to write and maintain code with much simpler tools, so why bother? I can use the party of my brain that would otherwise be used for key bindings for other things. So, mileage varies.
- python3 (OK, mostly a given, though not on Windows)
- pip and depending on your requirements you might need Poetry or Pipenv or whatever else
- Jupyter ecosystem (JupyterLab, Jupyter/notebook server and all their dependencies)
- some extensions for JupyterLab. Those could be frontend and backend. If frontend, then you will also need to have NodeJS on your machine, to have NPM, to have tsc to be able to build JupyterLab with the extension installed.
Quite a few parts there as well.
Cognitive load when using the notebook is very low compared to emacs. As has been remarked on often, emacs is probably fine if you've been using it for thirty years, but having to keep track of different key strokes for copy paste depending on what window you're in (emacs vs the rest of the world) is pointless cognitive load. The idioms are so sprawling and out of sync with everything else that the editor is an obstacle in the way to doing what I want.
And while (historically) that perspective may just paint me as a noob, I'll just say that no amount of memorizing editor functionality or micro optimizing configurations will ever help me prove a theorem. If there's another tool that gets the job done and doesn't want to fight me before/while I use it, then it's the better tool.
(1) This is better to me, because it makes notebooks more consistent and easier to jump into. I read english prose sequentially, why not code?
(2) You can do this in all environments I've used with <a> tags.
(3) this would be nice, but I'm not sure it's worth the tradeoff because the editing environment wouldn't correspond as well to the exported document.
(4) true, but I have a vim plugin that make it better
(5) also true, but this is a good thing for simplicity. I don't want to have to keep track of the lexical scope of multiple language environments
(6) There are tools for this, but yeah it's not great. The main issue is saving outputs
It sounds like the goals of notebooks vs org mode are slightly different, at least the way I use notebooks.
For me, notebooks are a tool for sacrificing control and programming flexibility in exchange for easier collaboration and visualization. It's harder to write structured code, but it's much easier to share with a large number of semi-technical people.
As a counterexample, some things are not important but have to be mentioned. In English, these things end up in the appendix. In Pluto notebooks, these things can be moved to the bottom.
Another benefit of less strict ordering is when debugging. Sometimes cell A fails because of cell B. It can be useful for debugging to temporarily put these cells next to each other
In a top to bottom typical notebook, you will have to run the notebook from the top. There is a menu item for that, of course. However, what if you also have some cells in there, which do not have to be run? They will waste time unnecessarily or even worse, will change the result. So you will need to take a look at each cell to make sure you really can run everything sequentially. Either the notebook has to be created to make sure, that running top to bottom always works and gives the same result, or you will have to look at everything. When code segments can define their dependence on other code segments, they can be designed to that it does not matter which one you run, it will always give the intended result.
I love notebooks, made VS Code extensions for Go (gobook) and for Rust (rustnote). Give them a try if anyone is interested, still lots of features to add, but the source code is standard markdown, so the code results can be saved straight to Github and they render there.
(also obserablejs) for a different take on it.
I have a feeling notebooks are not as popular for javascript since it's quite limiting relative to what a code sandbox can do.
That aside, I completely agree - Observable has brought back a lot of the joy of programming I had when I first started. Congratulations to Bostock and the team.
With "literate Programming", it is one product, the program! Ideally, it is just readable code, code that human can read literally. Since human mind works differently from computer, so the literate programming environment need transform the literate code into a form that compiler can optimize. I believe this idea is inside Knuth's LP. But early attempt is always limited by technology. For example, Knuth's system is based on token parser. Token is not the unit of comprehension for human. Thus he can't do much with the Pascal part other than "tangling" the code as is. Then he had to supplement his goal with English or "document". But if we start to develop the framework beyond "tangle", we'll see a new page in literate programming.
* The "Notebooks" (the noun) implies we are emphasizing the product as documents. However, nothing prevent people to use "Notebooks" to program. Then immediately one will see how limited it is. "Notebooks" does not "tangle" code. Really, there is not much you can do with the code part. For example, try use the "Notebook" form on a C program.
The Julia one is called jupyter.
Jupyter stands for Julia, Python, TeX and R.
1 ‘An introduction to pluto’. LWN. Available from: https://lwn.net/Articles/835930/The focus on a notebook is the report, not the code. In practice this means that the report structure dictates code structure, not the other way round.
By contrast, the focus of literate programming is the code. Any "reporting" done is strictly to enhance the code, and not the other way round.
I've seen far too many codebases that should have been "code generating a report" reverse that logic and implemented as notebooks instead, in the expectation that the underlying code can simply be exported from the report. Good luck if you need to work on someone's code of that kind. It's its own special kind of hell.
Notebooks are used for small snippets, many of them trivial. The comments usually tell you what the buggy snippet is supposed to to. They are more like Powerpoint.
AFAIK, there are some types of notebooks that allow the use of chunks, but I've never seen people writing in those.
int main(int argc, char **argv) {
<<main-parse-argv>>
// ...
}ipynb files are shit, but Jupyter doesn't have to be. For example, VS Code supports Jupyter without ipynb. This way you can end up with files which, unlike ipynb, are valid in the target language (e.g. python, julia) AND play nice with git, but are still interactive thanks to Jupyter.
In fact, the above realization - that there are IDEs that support Jupyter without ipynb - has been the biggest boost to my productivity in 2021. This came in the form of being able to version my interactive jupyter files.
https://code.visualstudio.com/docs/python/jupyter-support-py