Notebook Is A Better Readme | Hacker News Reader

Notebook Is A Better Readme

matyunya-readme.ellx.app

102 pointsmatyunya76 comments

Somewhat of a tangential comment: I love that README files have become useful again, say, in the last 10 years. I think GitHub and the proliferation of open source projects may have had something to do with that.

My memory is a bit fuzzy, but back in the late 1980s and most of 1990s, the README file was useful,and it was the first thing that I opened when I downloaded a software package. Then sometime in the 2000s, the corporate lawyers took over the README file and it just became a wall of CAPITALIZED legal text that contained zero useful information. I stopped opening the README files. Then sometime in the 2010s, the README files became useful again, and I started to read them again. Not exactly sure when that happened.

> I love that README files have become useful again, say, in the last 10 years.

They have always been extremely useful, actually. It is always the first thing you open when you download a package (well, that or INSTALL). You're right that the README's of, say, MS-Windows or such software is less useful, but I don't know that this situation has improved recently. Perhaps it's just your use of FOSS? :-)

No, I don’t think so. Readme pages should be the lowest common denominator to information on the project no matter where you come across it.

I think markdown is probably as fancy as it can get and still be readable in the terminal after I clone the repo, or any ide, or the browser.

I think of it as a more formatted man page. Would I want a man page to be interactive? Or only work in a browser.

I also think if GitHub allowed more flexible markdown (like allowing html and js) it would turn more into a MySpace chaos. I don’t think authors would limit themselves to nice functions only once you let the JavaScript and CSS cat out of the bag.

I think your example is neat but that’s what pages is for. Use pages and add a link at the top of your readme. Just because GitHub is the first thing your users see doesn’t mean it has to be. Make a project web site, direct users there. Use GitHub pages for that if you don’t have a site elsewhere.

This. Readme files should be readable by anyone. That's one of the promises in the filename (it is readable ;) ). There is a reason readme files are not PDF files.

In the examples provided, I find the original text quite readable in text-only format. It just has additional richness if interpreted in notebook form. (I'm sure that someone will take the examples quite a bit farther and make portions of the documentation less accessible in plain utf-8 format, but major portions will still be entirely usable I think.)

Are man pages not readable because the original source form is troff?

Readme files aren’t readme files anymore. Now they’re collaborative team documents used for all sorts of purposes, such as onramping new people or explaining bureaucratic structure.

The times should reflect this evolving need.

Those things are deserving of being explained, but do they really need to be in a readme? Can't we just link to them?

They can be both. I actually like markdown for collaborative team documents because they are so simple. I’d rather people focus on the content rather than trying to make the perfect PowerPoint or format a word doc or something.

I also like that the on-ramp document can get PRs from new hires who find some improvement, etc etc.

> The times should reflect this evolving need.

There's no need since, these needs are already addressed in their own files for decades IMHO.

Readme files are a general map most of the time. Other relevant files are "Install", "Contributing", etc. Your README should be concise and refer to another text (or markdown) files in the repository.

Of course people adapt and evolve but, I don't understand the attitude of "We've just invented this". No, we didn't. README.1st, CONTRIBUTING files are as old as computing now.

I'd say that after the launch of Github wikis/Pages, using READMEs as anything but a quick project overview is a regression rather than an evolution.

Sure, you can put a bunch of quick links in there to get people started, but as a potential user interested in using a project, I'm not interested in your complete overview of the source code architecture or the necessary onboarding steps.

Put a quick link in the README to guide people to the right place, but don't put all of your project documentation in there unless your project is particularly small. We've grown beyond storing a bunch of texts files in a SVN respository, every major source control system also has a way to document your project.

If you choose to keep your documentation inside of your source control, the rise of Markdown viewers that support navigating links can help you achieve the same benefits by just linking to docs/project-overview.md in the top of your README.

Ok but then you get README files that are useless on their own, containing basically only a brief descriptions and several links to github pages / docs website. What good does it make to be able to read that on your terminal?

I don’t think thats required and based on viewing probably dozens of readmes a week that’s pretty rare. It varies a lot by project but I find readmes pretty useful even if they do point to a readthedocs site for more info.

I think it’s unusual to have a readme with as much info as a man page, but readme is for the source right. The documentation is usually organized depending on the type of project.

>Or only work in a browser.

Shouldn't everything only work in a web browser? I'm not sure why documentation hasn't been migrated to containerized microservices deployed on K8s with a React frontend yet. This stuff is all sooo antiquated. /s

Among the other issues mentioned, I also found basic scrolling on that page extremely slow on mobile.

Should be fixed now. Thanks for spotting this

Divergent goals, divergent choices.

I think the disagreement largely boiks down to semantics of the word "readme." Both versions can coexist, probably will. They're just going argue about which one is the "real" readme... the old school or the one that the author wants you to read.

> Would I want a man page to be interactive? Or only work in a browser.

That's what Stallman wanted for GNU. The results speak for themselves.

Make a blog post. Social media is for cats and shallow memes.

I too prefer to subjectively dictate based upon prior experience.

How someone organizes a project is up to them, and the folks interested in it?

I think READMEs should be kept as simple as possible. They should remain compatible/functional with terminal output, and they shouldn't have dependencies.

But I also think many READMEs could be improved with the inclusion of a figure or two.

It really helps if the artefact has some kind of visual output, and if there isn't, some kind of diagram could be useful.

What we should do is turn readmes into readmes and not into documentation. Github-like readmes are awful for that purpose. They have no TOC, heading styles are often misleading after two levels, and there is no indenting that could help. For contrast, GTK documentation is great, and so are G docs in general. It usually has toc, intro, description, examples and then detailed and cross-referenced “what you pass and what you get back, and how errors are structured” and formatting to grasp it quickly. Compared to that, usual github docs are just unreadable crap with a little knowledge spread on paragraphs of spoken text.

https://developer.gnome.org/gtk3/stable/GtkWindow.html

> Github-like readmes are awful for that purpose. They have no TOC

I add TOCs in README.md's as they get longer, and I suggest everyone consider doing the same.

Example:

https://github.com/eyalroz/tpch-dbgen

Oh god please no. The last thing we need is more dependance on web browsers.

Web browsers are an operating system these days. Having more dependence on them is like having more dependence on Linux/Windows/OSX. It's a pipedream to think they will remain consistent and/or shrink in use case.

Like it or not, this is here to stay. What we should fight against, imo, is not browsers as an idea - but the lack of diversity. We're in a reality where we all but only two/three browsers - it would be as if we only had Windows and OSX. The answer wouldn't be to get rid of Windows or OSX, but rather create a rich ecosystem of complementing operating systems - ala Linux.

> Like it or not, this is here to stay

Like it or not, the web browser game is one monopoly after another. First it was internet explorer, now it's chrome.

It's been happening again and again, the only way not to lose is not to play the game.

Thus, relying less on the browser is a good thing.

So, will you forsake all propriety computer architectures too? And wireless tech?

IPython cell can run in terminal too. Jupyter notebook consist of JSON too, just waiting for someone who could combine all of these things in terminal.

Although I think README need to be static, I still think that having some images clearly describe what is this project for or some code example is better. English is not my first language, so I don't like to read a wall of English text. Short paragraph with code example and images are much more approachable for me. This is also a reason why I rarely use manpage, and use "--help" or just Google it instead.

A while ago I made a demo of an “interactive readme” for a fuzzy string matching library called fuzzyset: http://glench.github.io/fuzzyset.js/ui/

Really cool that this author is thinking about ways to make that kind of stuff a reality. To me it’s exciting to imagine a future where programs automatically generate UIs that explain what they do with the best techniques from pedagogy, graphic design, video game mechanics, etc.

There is already a better readme. It's called a man page. The original "online" documentation. The ability to have content generated dynamically, using source and system operators. Spreadsheet style abilities with a touch of awk.

Perhaps it needs to be rebranded with a vegetable name, for modern developers to take interest. Samphire, maybe.

Perhaps we need to adopt something like a "TRYME" as a standard way for people who want to sample a project or do a little interactive learning with it.

An examples/ subdirectory seems a semi-frequent open source solution to this.

I prefer my readmes in plain text formats, be it Markdown, AsciiDoc, or unstructured txt. This reminds me of two interesting projects, maid[0], a task runner that uses markdown files, and mandown[1], a markdown pager that improves reading markdown files in the terminal.

[0]: https://github.com/egoist/maid

[1]: https://github.com/Titor8115/mandown

I skimmed through the article, and seems similar to literate programming: https://github.com/gicentre/litvis

Would be grateful if anyone could explain the significant differences between the two.

Org mode is a better readme and a better notebook :)

Is there a way to enjoy all the pleasures of org mode without having emacs installed on my computer?

If there isn't, it seems that "org mode" is a very bad name and it should be called simply "emacs". Or, if you want to be too precise, "emacs, with some configuration tweaks".

emacs is my main editor but I haven't figured out org-mode. So I see the users as more a subset (possibly power-) group of emacs users. But yeah while there may be other interfaces to org-mode these days emacs is the primary one.

It's for organisation of information and it's an Emacs "mode".

It's as much "emacs with some configuration tweaks" as VS Code is "chromium, with slightly improved <textarea>".

W.r.t. the good question in a comment that definitely has more than just that, yes, there are partial ports of Org mode to vim, VS Cod{e,ium}, and AFAIK Sublime Text, but Org does make use of some facilities of Emacs that are not as easy to replicate on these editors. For exporting, Pandoc can help to some extent, but it's imperfect and not as configurable.

> VS Code is "chromium, with slightly improved <textarea>"

Ha! I love this, definitely going to say it in public.

Apart from this joke, I asked a well-intentioned question. I have seen people doing magic with org-mode, treating text documents as if they were notebooks, running snippets of code by selecting them, then having the output as text. I would like to do something similar in vim, if possible.

You can certainly enjoy org mode documents read only without emacs, it’s ultimately plain text.

If you want to edit org files I really don’t think there is a good substitute for emacs if you want all the features, however you can definitely do basic modifications with any text editor.

Okay, um, a readme file is just the lowest common denominator “what the heck is in this directory tree” file. Something to get your bearings from. Something that can then point to some more fancy documentation like a homepage.

But now I guess it’s a landing page with a bunch of tags at the start. That’s not a good development in my opinion.

For our notebook format at Nextjournal we use an edn appendix to store metadata like results and environment (docker image). The user-edited content is just plain markdown, while also being runnable reproducibly. See [1] and run it at [2].

[1]: https://github.com/nextjournal/notebook-format-demo/blob/mas...

[2]: https://github.nextjournal.com/nextjournal/notebook-format-d...

This is not a new concept. Let's take a look at TensorFlow website, the site is basically Colab notebooks. Colab being free and requiring just a gmail account allows people to easily just run TensorFlow examples within seconds. This a specific case for Python, (other projects such as Pytorch, Google Cloud documentation, Beam follow the same patterns). When your README wants to show users runnable code or quick starts in my experience that's what user ultimately and if you can achieve this, people will actually use the README. From maintenance perspective, your README notebook may not be runnable in all Notebook environments which may be ok if user at least can render the content.

An alternative would be to leave readmes alone, but then to provide a nice, concise, universal documentation format.

The way github structures their doc trees is also not nice for easy navigation, which is required often for docs when they are structured that way.

I think this is a commendable idea. It may be that conflating it with README is not the right idea, but I can definitely see value in having a playground/ sandbox. Most people commenting here are probably comfortable in code and getting things up and running locally, but that isn’t everyone or even most people. Being able to see the library do what you want gives newer users far more incentive to stick with the possibly painful process of making it work for them in development.

I think it's funny that since I had js disabled all I saw was "Error - blah blah blah" and I thought this was arguing AGAINST using a notebook as a readme.

Looking at umatrix all the js is coming from a CDN too. I think it show the obvious benefit of readmes compared to a notebook which might be useless in 10 years due to link rot or the CDN going under.

Yeah, how about no. READMEs should be viewable in anything and NOT run any code. There is a time and a place for documentation which exhibits the features of the thing being documented inline with the text, but the README is not it. Tell me what the thing is and does and where to get more info.

One thing that would have helped me as a reader is to cite the name Ellx by adding the bracket text in this sentence:

> Now, I've been working on [Ellx, ]a tool

The first reference to the name that I saw was in relation to a scope convention and I had to go search for the anchor to that reference.

Thank you, I added reference to the docs:

https://docs.ellx.app/#namespaces

It seems like the framing around Readme sort-of backfired. It doesn't strike me as the core of this project, which is really a different take on the colab/jubyter paradigm.

I like the idea of spreadsheets as a first-class citizen in such a concept.

Complicating READMEs may not be the best idea. This is under the assumption that README will be rendered in a full featured web browser which may not always be the case.

That said, I definitely see how this can become invaluable in documentation for DevOps tools.

Scrolling this page on mobile is terrible and I think that's a pretty compelling argument against it.

I would love if every package had a LINK to a live notebook to test things, but it should not replace the README.

Thank you for pointing out the lag, it should be gone now.

This 'article' appears to be a new-ish form of GPT* generated SEO blogspam.

Scrolling on that site lags so badly on Chrome on Android (Pixel 4a).

Thank you for pointing out! Should be gone now.

Is Ellx open source?

Ellx founder here. It will be. It started as a side project which quickly grew into something very versatile, so we are still in search for a product-market fit and a community that would benefit the most from it. Testing hypotheses. Open source comes with a significant maintenance burden that we just don't have resources right now to carry. Would you like to help?

No.

And, by the way, stop with the markdown README crazyness.

README files are supposed to be readable as plain text and hard-wrapped at 80 columns. I have found some "readme.md" monstrosities with github-specific markdown that were unreadable outside the github website. This is akin to presenting your readme file as a flash ainmation.

Markdown is still just plain text without any "crazyness".

Stuff like "## Header" isn't any more "crazy" than the ASCII art used in many Readme's of old and on top of that doesn't rely on specific monospace fonts or fixed width. And more advanced stuff like hyperlinks or tables are hard to do "right" in text anyway, since it's basically meta data without an objective "right" or "wrong" way of representing it.

I also truly believe that it's time to finally let go of the 80-columns obsession. It's not 1975 anymore and even the ancient VT100 from 1978 had a 132-column mode. It's just an indefensible relic that held developers hostage for way too long.

Still want 80 columns on your 4k 27" 10bpp HDR monitor? Fine, but don't insist on others bending over backwards just not anger the grey beards on top of Mount Teletype... Text editors are perfectly capable of introducing proper line breaks even at word boundaries if need be.

It's really just become l'art pour l'art to force this arbitrary (by today's [and by today I mean the past 30 years!] standards) restriction on developers with no benefit to readability.

I'm not against markdown per-se, but against markdown readme files that use so much markup (heh) that become unreadable as plain text, no matter your screen size.

> Still want 80 columns on your 4k 27" 10bpp HDR monitor?

This is offtopic, but a strong YES. There's good reason why on printed books you don't ever see more than about 70 characters per line of text. Long lines are just unreadable, regardless of your font/window/screen size.

> There's good reason why on printed books you don't ever see more than about 70 characters per line of text.

That reason is called format not readability, though.

Newspapers (if you still remember these) go way beyond this limit and are still readable.

Don't newspapers use columns, and thus go way below this limit?

They go beyond the limit by displaying text in columns. Imagine how difficult it would be to read a broadsheet newspaper if the text ran from one edge of the page to the other. As lines get longer, it is increasingly difficult to find the starting point of the next line and reading speed and comprehension are reduced.

https://www.sciencedirect.com/science/article/abs/pii/S10715...

Just for shits and giggles I grabbed a random book from the shelf and counted characters per column.

The book I grabbed was "Spoken Language Processing", Huang Acero, Hon (ISBN 0-13-022616-5) and I counted 88 characters per line. The book is still very readable, despite exceeding the claimed 70-odd character limit for books/columns. Another book sitting next to it (a numerical mathematics textbook) had 82 characters per line.

Granted, the limit of no more than 70 characters per line applies if whitespace and punctuation aren't counted, but that's never done with hard limits in digital documents either, so...

As others say, it's the opposite. Newspapers and magazines usually have a character limit of well under 80 characters per row of text.

Every so often I open a magazine that doesn't abide by this, and has text that spans across the whole page, and they're nearly unreadable.

I have a 4k 43" monitor. Now I can easily see 8 separate chunks of code with my monitor split into 4 columns and 2 rows. Or I can see 2 columns and have the app on the left half of the screen with a debugger. I break the 80-char rule frequently, but having code be limited in width is still useful.

I find soft-wrapping code pretty tedious, so even with huge screens IMHO indeed breaking around 80-100 (accounting also for indentation) is nice.

BTW I recently go a 24" screen (I was all about laptops, turns out I was torturing myself...) and I can easily fit 3 columns in Emacs, with a large-ish point size (238 chars across the monitor, DejaVu Sans Mono). If it's not a preference thing, and assuming (corrected-to-)fine eyesight, shouldn't 43" be capable of like 8x2 easily?

No. It might be useful for some but it would make it much harder for others

I question Unicode as well.

A readme file should work as expected in simple text editor. Like an early version of VI from 30 years ago.

Anything fancy should be able in the documentation hierarchy for the project

Sorry, but utf-8 is necessary. The world is not USA and some of us exist outside of it, where ascii is gross disregard to conventions. I can agree on plain text readmes, but there is reason that the world moved away from english only encodings and if you want to stay thirty years in the past, this is your choice.

This. Unicode is the new ASCII. One should avoid, of course, the fancy corners of Unicode (such as the private use area, emojis, hieroglyphs, etc.) unless specific to the use case. But the use of well established Unicode code pages in READMEs should be no problem for temporary text editors at all.

I am unsure why the hostility is needed. I am also unclear on the incorrect assumptions you made about me.

I do not live in America and the country I live in has it's own set of special characters to deal with.

I can perhaps agree that American cultural imperialism in computer science as with many other fields

This post was primarily directed at GitHub. It has users from all over the world.

Having a common Lingua franca for the various public rep is inclusive and productive

I want here there to be a low threshold to use and contribute to my projects.

Lastly, Github is an American company located in the US. With repositories primarily using American created and optimized for Americans keyboard layout programming languages.

Certainly, some programming languages originated in other countries. As far as I can recall they all stick to English for keywords.

A couple of decades ago there was much more variety, and it was more common for countries to have native originated programming languages that did not use English.

Linux started in Finland but it kept the US created standard UNIX names and conventions.

Most if not all documentation is in English many have now been translated to other languages.

With all that in mind. Sticking to the most compatible common denominator is the best standard on Gigahub.

You said that you question utf-8. I do not. That's it.

It is not about imperialism, but computer science is not all about problems that could be expressed in english either. What I'm supposed to do if I need to write a program for manipulating cyrillic texts? Transcribe everything in latin? Be serious and imagine that though english is useful, it is not sufficient.

The contexts remains ReadMe files on Github. Remember that.

If you wish to write a program to do Y then write the code your tests etc using whatever encoding makes you happy.

Nowhere have I said no Unicodein any repositories which is the strawman you are reachinging for.

You can still write Readme files without Unicode so that the great majority of user of public Github can see what your project is about.

Now there are many close editors today that do not support full unicode

Presumably advocating unicode should not discriminate against languages with Arabic.

Most code editors can not do that right out of the box. More and more can.

> the great majority of use

The great majority of users has editors that can handle unicode. What editor common today doesn't do UTF-8?

Let's imagine that I'm writing a cyrillic program. The users are most likely not english, the stakeholders are not english, the examples that the program works with are not ascii. Yet, you insist that in the readme I'm not using anything cyrillic related and use ascii instead.

If I happen to find an arabic readme, then I'll know that either I'm not the target audience, or that the author does not know english and might loose my contributions. Neither of those gives me the right to open a PR and insist that the readme is adjusted for my use case only and degrades the user experience of others.

What I strongly disagree with you is that from the principal of "be conservative in what you send and liberal in what you accept" you choose to take only the first part and forget the latter. Ascii is nice when it is applicable for the use case, but it is not universally applicable and assuming so should be a personal mistake, not tax on everyone. Maybe the disagreement between us is in how we define target audience. You imagine that if it is published in Github, then it must be readable to anyone there. I think that if the project is ment to help certain type of people, then it it must be optimized for them, not for people who would never bother using it anyways.

> I do not live in America and the country I live in has it's own set of special characters to deal with.

Yes. I’ve noticed that programmers from all over the world are narrow-minded and view things like international, inclusive standards to be “bloat”, preferring a monoculture since that would make things simpler (technically simpler, which is all that matters if you have no sense of aesthetics or culture).

Reply on news.ycombinator.com