Show HN: manpages-tldr – short manpages with examples | Hacker News Reader

Show HN: manpages-tldr – short manpages with examples

github.com

48 pointsJIghtuse31 comments

My small project similar to bropages and tldr, but (finally!) using manpages only. Manpages generated from Markdown files by Pandoc.

Open articleView on HN

Note that:

• Manpages are short. The average length is around 4 pages printed, much of which is pro-forma. Some run longer. Shells in particular (bash(1) runs around 98 pages piped through 'pr').

• Manpages include an EXAMPLES section, though this isn't always filled. That should be considered a bug.

• Manpages suffer from various projects attempting to deprecate them in favor of other forms of documentation. I particularly fault the GNU project's attempt to shoehorn info in place of man (the availability of man to Web utilities such as Debian's dwww largely wholly supplants this: info and the WWW were born within months of each other, Stallman and the FSF should have conceded the battle decades ago). Red Hat and the GNOME project have also been notably lax in providing manpages for utilities and programs. The Debian project considers omission of a manpage for an executable to be a bug, though not a release-critical one (I wish this severity would be increased).

• Fixing the deficiencies of existing manpages (and beating upstreams over the head with these until they're accepted) would be preferable to creating numerous externally maintained nonstandard variants. http://xkcd.com/927/

Thank you. I would do my best to improve real manpages. As I say in comments here, tldr pages aims to provide quick access to examples and must not replace real manpages.

How do you think, would I need to change license of a project if I want to push some examples to manpages?

If you're the author of the pages, you can set the licensing terms as you wish.

Generally, an MIT/BSD style license tends to be compatible with most other licenses. If you're the sole author, licensing the manpages under the same terms as the specific upstream works would work.

You might specify your own terms (say: a CC license), with a proviso "or, as an alternative, under the licensing terms of the original project for which this manual page applies".

IANAL.

I think stating that manpages are short is a little inaccurate. The average may very well be 4 pages, but the standard deviation is probably high (bash, gcc, ssh among others have long man pages). A few commands nowadays also have multiple man pages because people don't want to search through all the crap that's in a normal manpage (ie git, gittutorial, gittutorial-2, Everyday Git, ...).

People likely aren't going to use manpages-tldr for 4 page man pages. They may use it for gcc which is 17549 lines (without an examples section and examples are littered throughout the page, meaning you have to already know what you're looking for).

According to Wikipedia, man pages have been around since 1971[1]. Man pages were designed before a paradigm shift from systems oriented to user oriented design and are usually written in a manner that makes them easy to write, not easy to read. If you think about the first time you looked at man pages, they were likely difficult to look through and you just got used to it; that's what happened to me.

I would guess that most people would like examples of basic usage. The examples section of man pages is normally near the end but it's location isn't really standardized. I normally end up searching "/^[[:blank:]]*EXAMPLE" and if I don't find it I google it.

This doesn't mean that man pages are not useful when you know what you're looking for. They can be particularly useful for c code to supplement the headers themselves.

I think forcing man pages to be something that they are not isn't much better than having a bunch of nonstandard variants. I am not saying that the examples shouldn't be added, but man pages are very technical when most people probably want a layman example before the technical details.

[1] http://en.wikipedia.org/wiki/Man_page#History

I think stating that manpages are short is a little inaccurate.

You're arguing against the data. Care to continue doing so?

There are a few outliers, but really, the average page length is pretty low.

I'm running this right now, and what I'm getting is (5831 pages processed):

    mean:  3.9 pages
    median:  2 pages
    standard deviation:  11.06 pages

If you'd prefer precentiles:

    5th %ile:  1
    25th %ile: 1
    75th %ile: 3
    95th %ile: 12

Max is 476 pages (zshall(1)).

This is on a Debian GNU/Linux system with 3479 packages installed. I checked only English manpages under /usr/share/man (there are another dozen or so pages for openjdk under /usr/lib/jvm, max length 29 pages).

Obviously, mileage may vary.

Code:

    cd /usr/share/man
    time for f in $(find man[0-9] -type f -name \*.[0-9].gz); do echo -e "$f\t\c"; man $f 2>&1 | pr | grep '      Page [0-9][0-9]* *$' | tail -1 | sed 's/^.*\(Page \)//'; done

Takes about 4.5 minutes to run on my system.

Compute statistical moments with your preferred analysis tool (I've got one I wrote for the purpose).

There are 68 manpages 37 pages or longer in length (2 standard deviations over mean). I see several shells (zshall, bash, tcsh), many perl utilities, and a few complex tools (mutt, openvpn, wireshark, busybox) among them. Pretty small count, actually.

Man pages were designed before a paradigm shift from systems oriented to user oriented design and are usually written in a manner that makes them easy to write, not easy to read.

While understanding a manpage's format is useful, ANY documentation which is "easy to write, not easy to read" is a bug.

As for alternatives to manual pages, I find the Linux Documentation Project's HOWTOs to be an excellent adjunct. GNU info pages, not so much.

I meant generalizing all manpages as short is inaccurate. The original statement was that the average is 4 pages which tells you almost nothing without some idea of the spread. You have provided that, thank you.

5831 is the number of manpages processed, or the number of total pages inside them processsed?

My system has 18,590 manpages. Running:

  cd /usr/share/man
  find ./man[0-9] -type f -exec zgrep '^[[:blank:]]*EXAMPLE' {} \;|wc -l

yields 17. While I admit that this may not be the best regular expression that would best match the pattern all the time, I think there is generally not an examples section. This may be because a lot of the documentation is for code that does not need examples.

  for i in $(seq 1 9);do echo $i $(ls man$i|wc -l);done
  1 2554
  2 454
  3 16014
  4 45
  5 346
  6 49
  7 284
  8 821

I don't have any man 9 pages, and cropped the error from that part.

While it may be true that documentation that is "easy to write, not easy to read" is a bug, I think it is very unlikely that the average person would file a bug report for documentation.

I did not know about The Linux Documentation Project's HOWTOs, very neat.

EDIT: I also wanted to point out that depending on what section you are looking at manpages for, you have different statistics.

  man1 
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  1.000   1.000   2.000   5.519   4.000 470.000

  man2 
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  1.000   2.000   2.000   3.018   3.000  31.000 

  man3 
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  1.000   1.000   2.000   3.067   3.000 113.000 

  man4 
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   1.00    1.00    2.00    3.95    5.00   21.00 

  man5 
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  1.000   2.000   3.000   5.283   5.000 149.000 

  man6 
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  1.000   1.000   2.000   2.306   2.000  15.000 

  man7 
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  1.000   3.000   3.000   5.808   6.000  69.000 

  man8 
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   1.00    1.00    2.00    3.06    3.00   77.00

5831 is the number of manpages processed, or the number of total pages inside them processsed?

I'm not sure I follow your question. A man file contains a single manpage. My 'find' command explicitly specified objects of type file rather than symlinks to avoid double-counting manfiles with multiple symlinks (I find 2527 symlinks under /usr/share/man/man). That excludes hard links -- turns out that agetty.8 and getty.8 are hard linked:

    $ find man* -type f ! -links 1
    man8/agetty.8.gz
    man8/getty.8.gz

My system has 18,590 manpages.*

Mind if I ask what OS you're running? How many installed packages? And have you de-duplicated the manpages?

I think it is very unlikely that the average person would file a bug report for documentation.

I've filed such bugs. It's difficult to search across all packages, but there are a few filed for bash under Debian:

http://bugs.debian.org/cgi-bin/pkgreport.cgi?include=subject...

Re: different manpage section have different statistics: Well, yes. As I said, YMMV, however the mean is still generally within 1-2 pages of the 4-page mean I'd first described.

Note that nowhere did I say that all manpages have EXAMPLE sections. Many (most?) don't. Hrm ...

Of 3066 manpages in man1, I find 519 contain lines beginning with "EXAMPLE" in formatted output. That's roughly 17%.

My bigger point is that addressing this deficiency rather than creating de novo documentation projects would be highly preferable. Much less glory, sadly.

I found the term page ambiguous just on the very first use, whether you meant printed pages, or manpages.

I am running archlinux which would normally be light, but unfortunately I work with a lot of different programming languages and prefer different tools for each. I did not deduplicate any of the manpages. The number of packages that I have installed, listed by `pacman -Q|wc -l` is 1241.

I agree that it would be nice to have a single place to go to get all of the details, but I think manpages suffer from more than just examples. I think that someone just starting *nix is likely to look online. While you can search manpages with apropos or man -k it doesn't search the whole document and people just starting don't know the terminal commands to search more.

I would rather have 2 commands and have all of the information than just manpages that don't get frequently updated.

I found the term page ambiguous just on the very first use, whether you meant printed pages, or manpages.

"man page" == a given 'Nix feature's manual entry. I've tried to use "manpage" where this is being presented.

"Pages" without the "man" modifier indicates the number of printed pages produced (defaulting to one page per 56 lines of output using 'pr'). The page count is actually reduced somewhat when printing postscript: 72 vs. 98 pages for bash(1).

While you can search manpages with apropos or man -k...

Again: this is where tools such as Debian's dwww (also available on Debian derivatives such as Ubuntu) come in very handy: the manpages are presented as Web pages on the localhost webserver, and with an installed search engine ("swish++"), the full text of all documentation is indexed.

And just to be clear: Debian provides copious amounts of documentation packages: manpages, info pages, the Linux Documentation Project (in HTML or ASCII text: doc-linux-html / doc-linux-text), RFCs, and multiple guides and such. With dwww, all of this is presented locally as web documents, indexed, and searchable. Seems there's little reason Wikipedia couldn't be included as well, though you'd want to sync that regularly.

Why can't people just improve manpages so that they include a readable abstract and examples instead of coming up with projects with cute names?

So you think they should be "written as part of real manpages?"

From the readme:

> This project aims to make fast-updated, practical and precise manpages with examples for everyday usage. Its idea based on similar projects, like bropages and tldr, but written as part of real manpages (mostly converted from Markdown with pandoc).

And if OP is reading this there's a couple of typos there.

Normally I don't care, but this is a do umentation project so it's important.

> Its idea based

Should be

> It's an idea based

What I meant is to submit patches to the upstream manpages, so that everyone benefits.

Thanks. English is not my native language. Of course, this can not be an excuse.

No worries! I hope I was polite.

It is possible, but it takes a long way to approve and distribute manpages after changes. I like Michael Kerrisk for all his work. Manpages are great and I'm not trying to replace it, they have much more useful information.

Another way to think about this is, why can't manpages be written more concisely, so that the most common options and syntax are on the first page/"above the fold"?

Here is the 7th edition UNIX manual for comparison: http://cm.bell-labs.com/7thEdMan/bswv7.html

The printed output of the BSD 3.4 manuals for the first UNIX system I used (other than for playing BSD games as a kid) always had a certain fascination for me. Several large 3-ring binders comprising the sections of the manual, Postscript printed from their troff sources.

Note that you can still do this with manpages:

    man -Tps bash > /tmp/bash.ps

Or if you prefer PDF output:

    man -Tps bash | ps2pdf - > /tmp/bash.pdf

... not that this addresses the content of the manpages, as you note. But at least you can decide on the format.

Yes, as in that case

* the survival of OP's labour is more likely; and

* one might use something like `eg`[0] to directly jump to the 'Examples' section.

[0] I gave a possible implementation below (https://news.ycombinator.com/item?id=7167835).

Elsewhere[0] I mentioned a shell alias called `eg` that jumps to the 'Examples' section in traditional man-pages.

I made great use of it with the excellently structured yet rather verbose git man-pages.

Anyways, here's the alias:

    alias eg='man --pager='\''less -p "^[A-Z]* ?EXAMPLES"'\'''

Try it out:

    eg git pull

(With older shells you might have to say `eg git-pull`).

Now as concerns the fine project submitted here, I'd suggest to try getting as much of those tldr's into actual man-pages. This will at least make the long-term survival of OP's labour more probable.

[0] https://news.ycombinator.com/item?id=7167815

This was on here not long ago: http://bropages.org/

Wow. It took me ten seconds to click the link and read the description where the difference is highlighted. Can't you spare ten seconds?

I can : "This was on here not long ago: http://bropages.org/ but don't check it if you don't have ten seconds spare."

Welcome.

hah damn :( My bad, apologies! I skimmed over the project and saw that it seemed similar so thought i'd throw that up there for people interested in it!

it would've been great if all these (3 so far?) tldr man attempts would've also had a web interface, so local install would be optional

Well, pandoc can easily generate html, which can be styled as you wish. I make a list of tldr-manpages on my homepage: http://begorov.me/man-tldr/

It would be nice to have an example that is viewable from the markdown/on github. I think it would make people more likely to build it.

Do you mean screenshot of opened manpage?

Yeah, that's one way. Or a similar text formatting on html/md on the readme. If you look at the bropages one, it very concretely shows you what it is. Any way of doing this makes it easier for people to tell what exactly they're getting.

This is the man I've been looking for my whole life.

hf: if you see this, I think you are Hellbanned. All your posts come up as 'dead'.

Reply on news.ycombinator.com