Actually using Ed | Hacker News Reader

Actually using Ed

blog.sanctum.geek.nz

139 pointskryptiskt89 comments

I'm glad ed is there. I'm glad articles like this are there. But there is no way I'll have retained that article in the moment my terminal has borked and I actually need ed. I wish that something like this available in the man pages, or something like them, or at least referred to in them.

Because maybe I'm not clever enough but I don't find the man page articles to be very good introductions. As reminders of syntax and options and functionality they're great, but I don't find them useful as orientations to how the various utilities work. They are short on examples and demonstrations, which are often the quickest orientations to the basic operation of a tool.

The info tool is sometimes better but not so frequently I'm accustomed to using it. On Ubuntu 'info ed' actually looks pretty good from an example standpoint. But 'info awk' looks like a copy of the man page. And 'info vi' offers a discussion of the emacs vi emulator, about which vi enthusiasts can only mutter "well played".

It would be nice if there were a 'tutorial' command (or 'dontpanic' or 'towel') which gave a brief orientation to how these tools actually worked, with examples and demos, and then some lead into resources on the host (man pages, info, help available within the particular tool) and on the web and elsewhere.

On many systems you should be able to unbork the terminal with 'reset' or 'tput reset'. Then you do not need ed ;)

That's only going to fix a few causes of borked-ness. If the remote system doesn't have a termcap for your terminal, reset doesn't do much. In that case, setting TERM=vt100 will probably help matters, unless you're on a really obscure terminal of some sort.

I think my underlying point is that it's fine to learn ed, but if the problem is that you are borking your terminal then there are actually ways around that. These days it should almost never happen that you are sitting around tolerating a broken terminal. This is not an efficient use of time. The actual modern use cases for ed are not very common. If you want to learn it just for kicks or because you think it's cool then that is another thing.

That last happened to me fairly recently, when a server was unable to mount /usr (where the termcap stuff lives) and I had to use ed to edit the RAID config files so that I could mount it. Fun times.

Ouch, more extreme than I was thinking. I've encountered systems that don't like screen or urxvt or what have you, but they still had vt100 and that's a useful subset of the termcap of most modern terminal emulators. If there's no /usr at all, that's certainly a stronger argument for ed!

I've used ed and other commands on headless systems more than once, back when networks weren't common. One has to be careful of typing and having a bell can be useful for reassurance; grep newthing foo && echo ^G.

Linux man pages are notoriously verbose, and often obtuse and incomplete, this is in great part due to the extra complexity of GNU tools and their preference for the infamous info documentation system.

OpenBSD has probably the best Unix manual pages:

http://www.openbsd.org/cgi-bin/man.cgi?query=ed

And Plan 9 has a manual that is a pleasure to read, this is helped in part because the commands themselves have been cleaned up and simplified in many cases:

http://man.cat-v.org/p9p/1/ed

I dream of a system with features such as iPython live doc, or Bret Victor's tangle. It seems an approachable goal nowadays.

I think instead of `verbose' you meant the opposite `terse.'

They're either the output of --help turned into -man macros or more noise than signal. Damn info!

Info pages are utterly useless--I don't think I've ever had anything useful from there. If I've screwed my system up so badly I need to use ed, even "man" might not necessarily work (these days GNU commands are just so damn clever). Best thing is usually to grab another computer or your smartphone and start Googling.

If you want an ed(1) tutorial why not try Brian Kernighan's original? http://nietzschegym.com/edtut.pdf

My survival kit for linux man pages is "/" for search and then "n" until option is found.

I think a lot of C programmers would find reading the source to the V6 ed interesting:

  http://www.tuhs.org/Archive/PDP-11/Trees/V6/usr/source/s1/ed.c

1333 lines of C, basically no library use at all (no printf, not even malloc -- the only thing dynamically allocated is the document, and it uses sbrk() to manage the memory itself), compiles down to about 6K on a PDP-11. And that includes a regex engine!

It contains a lot of 1970 C-isms that won't work today ("=+" instead of "+=" for example) but an experienced C programmer shouldn't have a hard time following it.

As a bonus, look at the commented-out getpid() implementation at the bottom of the file.

I remember doing a=+1 in K&R days under IRIX and getting a warning that "=+" was deprecated. I had just started with Unix then, and had to get someone else to explain the source of that message.

How does the "goto errlab" work? It looks like it calls errfunc() then gets to the reset(), which doesn't return. Does setexit() determine the entry point for a reset?

This really is a bare bones system. An insert at the start shifts every character to the right by one, and memory growth is only 1024 bytes at a time. This leads to quadratic times. I wonder if that performance was noticeable on the PDP-11 when editing a large-for-the-time file.

Yes, setexit() and reset() were the precursors to setjmp()/longjmp(): http://man.cat-v.org/unix-6th/3/reset

It probably also has vestiges of the assembler version in its structure and symbols. troff(1) certainly has; I had trouble groking one particular function's logic and asked Kernighan. He kindly replied and said that one had always troubled him too and was just a transliteration from Ossanna's assembler. Glad it wasn't just me. :-)

I used to work at a place where their mission critical software was running on ancient HP-UX boxes which we feared to turn off because we didn't know if they would turn back on. My job was to port this software to modern Ubuntu VMs. Once I was done, we had a celebration in which we turned off the HP-UX boxes forever... except of course there was a bug in my version, so we soon had to turn them back on while I worked on a fix.

To everybody's surprise, the machines did turn back on, but they couldn't connect to the network because a few configuration files were wrong. To our dismay, after we tried to use Vi to edit the first one, we discovered that the ESC key did not work anymore, and we were stuck in edit mode forever! We had to reboot the machine, which takes about an hour on these old machines.

And then of course I configured the files using Ed and saved the day! Surprisingly, I was the youngest programmer there and I was the only one who knew how to use it.

You could have just typed CTRL+[ or CTRL+C, both exit insert mode in Vi. It's in the man page: http://nixdoc.net/man-pages/hp-ux/man1/vi.1.html

I love ed. A long time ago when I was mastering it, I made an ed cheat sheet. Here it is the article about it:

http://www.catonmat.net/blog/ed-unix-text-editor-cheat-sheet...

And here is the cheat sheet itself:

pdf: http://www.catonmat.net/download/ed.text.editor.cheat.sheet....

txt: http://www.catonmat.net/download/ed.unix.text.editor.cheat.s...

doc: http://www.catonmat.net/download/ed.text.editor.cheat.sheet....

Awesome stuff! I love your blog, by the way. It gives me a standard to which to aspire.

I guess I'll date myself by saying that when I started studying CS, I used ed and thought nothing of writing programming assignments with it. It seemed totally natural. I recall a typical sequence of writing a few lines and then printing out a block of lines to the screen to check. Then jumping somewhere else printing out a block of lines, editing some more etc. I recall hearing about vi and thinking it was overly complex and obviously wouldn't have worked on the hard-copy terminals (really, hard-copy terminals - no screen!) that were scattered about the lab. I don't think I could have conceived of eclipse...

What year was this?

I use ed frequently for writing quick scripts. Its edge is that you can use it while retaining visual history of the prompt.

Though, you could instead just redirect cat to a file and then do any tidy-up in your usual editor.

It's also sometimes useful when you're in a remote system that is behaving strangely due to screwed-up terminal definitions or some other catastrophe.

I'd be interested to read a sam tutorial focussed on using just the console version of the tool (i.e. not using the windowing system). I understand it takes a lot of ed's features and beefs them up.

I do so as well. Most often it is sudo ed /etc/hosts.

A Sam language tutorial by Rob Pike:

http://doc.cat-v.org/bell_labs/sam_lang_tutorial/

A cool thing about sam is that it has a client-server design, so you can run the GUI on your local system and still edit stuff remotely. This works very well even over very high latency connections.

See the -r flag: http://man.cat-v.org/p9p/1/sam

I still use ed for any short/small edits, it is hard to beat.

It works even better if you have a terminal like in rio/9term where you can edit/send history very easily.

cat also works quite well in such terminals thanks to 'hold mode' (cat > foo, press Esc, edit your text freely, press Esc again and ^D).

Yes, I'm a user of ed regularly as well when I would like to mingle interacting with it with my other before and after command line use. If you use vim then `:set t_ti= t_te=' may be of interest to stop it switching to the terminal's other buffer and back; that way on exiting the file's content remains to copy-and-paste from.

The article was OK. I'm not sure it mentioned addressing line 0 for the start of the file, and comma is more common than % for all lines. ed(1) is brief and lots can be learned from it.

As for structural regular expressions as used in sam(1), see Rob Pike's paper. http://doc.cat-v.org/bell_labs/structural_regexps/

Thanks for that link. 1987! Where's my structural-regexp-enabled awk!?

Having been a user of *NIX for over 20 years and a professional Systems Administrator for over 10, I have never once had a reason to use ed. Even the smallest embedded environments usually have some variant of vi available. The only time that I've needed to use ed was edlin in the dark days of early MS-DOS, which is very similar. This is borderline masochistic.

It's not whether 'vi' is available (it usually is), but whether it is behaving properly. Situations do occur where any fullscreen editor will fail and where 'ed' will work as expected. And at such moments, even though you don't know the full extent of the power of 'ed' you will be happy to know how to change a line and save the config file (and actually quit 'ed'; that's harder than it sounds)

An editor in need is an editor indeed!

Can you give a specific example? I have a not insignificant amount of experience with thousands of servers under my belt and I have yet to ever need to type "ed" into a server. I did find edlin useful in DOS 4, but that was ancient times.

Apples to oranges.

You narrowly missed the days when it was useful. When I started regularly sysadmining in the late 80s/early 90s (after being a mere user for the a few years) I found I had to learn ed. /usr was always a separate filesystem so any time there was a problem booting you didn't have vi available. If you didn't have at least rudimentary ed skills fixing the simplest problems would be rough.

In linux at least the separate /usr never seemed to get that popular. If you can mount the root filesystem you've got basically the whole OS at your disposal.

I will say that learning ed (or, better yet, ex) well does make you a more efficient vi user. When all you have is command-mode you get very adept at using it. It's actually sort of fun, too. Just hit "Q" in vim some time and see how you do with only ex commands.

I own a Unix introduction book from the early 80s (I think) that describes a customized/extended version of ed as the author's choice of text editor. vi is mentioned as "this newfangled editor that has more features than anyone needs and is such an incredible resource hog that it leaves barely any memory for the actual document on a large PDP-11".

DOS's edlin is a very crude line editor, unlike ed(1) which has regular expressions, the g// command, etc.

Agreed, but the point was that there are very very few legitimate reason in 2012 to use ed unless you just happen to like it.

There's an obscure "ed" mode in diff as well:

       -e  --ed
              Output an ed script.

http://en.wikipedia.org/wiki/Diff#Edit_script

    If you’re using any Unix at all, then ed really will always be there,
    no matter how old or limited the system.

So as it turns out, my Arch Linux setup does not actually have ed.

  If you’re using any Unix at all, then ed really 
  will always be there, no matter how old or limited 
  the system. Well, unless you use Arch Linux, anyway.

Yeah, I corrected that after I noticed a few people commenting to that effect. :)

That's cheating!

My bad. I just assumed he skimmed the article as I often do.

Either way, thanks for the article. I've considered my grip on native Unix tools to be a weakness, but there's generally not much good intro material like this.

My OpenWRT box does not have ed either.

In my experience, the closest approximation of an editor which will always be there in the real world is sed, not ed.

> Arch Linux setup does not actually have ed.

They did specify Unix.

I don't know if unix is the most ahem "specific" specification ever given. What specification of unix do you think the author was referring to? In any case, they have corrected themselves in the blogpost.

well, if he was referring to 'unix' as specified in the Single Unix Specification (which incorporates IEEE Std 1003.1)

then all unix systems have 'ed'

http://pubs.opengroup.org/onlinepubs/009695399/utilities/ed....

If arch linux doesn't include it, it's not a 'unix'; it's not even a good "not unix". So the author wouldn't have needed to correct his article but Arch would need to include 'ed' :)

I get where you're coming from but I just want to raise two quibbles:

> If arch linux doesn't include it, it's not a 'unix'

...but according to that version of SUS, I'm sure the original unix isn't unix either. In any case, I don't think you can just assume any particular spec when someone says the word 'unix', most people usually just mean *nix-family (and if the spec is relevant they'll probably cite it).

> it's not even a good "not unix". I imagine arch is much closer to compliance if you install everything in the 'core' repository (which is recommended and includes ed). In any case, if lacking ed makes something a "bad not unix", I'm not sure most people mind being wrong ;)

If anyone's interested, a while back I put together a table of commands available in various versions of ed, from 1st edition UNIX up to POSIX, as well as sam. It's largely academic, but I did find it interesting how features crept in.

http://lubutu.com/misc/ed-languages

That was a good explanation. I hope never to have to use ed.

That's the usual attitude, I think. "I'm glad I know how to use it now, but I hope I never have to."

You may occasionally want to use its little brother, `sed`, which has substantially similar syntax. It is often the case in Unix programming that you have a pipe full of text data and you would like to transform every line of it; `sed` allows you to write these transforms into one line of instructions on the console line.

So it may sound crazy that you have all of these terse commands, but just imagine that you're trying to write a one-line program to reformat text documents and it suddenly makes a bit more sense.

I used to use sed, but more recently I've been using 'perl -pe' instead, which is a drop-in replacement for the basic 's//' use I typically make of sed. The main reason is that I've had some performance problems with sed, depending on which of the several sed variants is installed. Many of them seem to have small fixed buffers that really limit throughput when processing large (multi-gigabyte) files, and the performance ends up being really unpredictable, while Perl's regex engine is fairly good and similar across platforms. (Also, I'm trying to avoid having to maintain both POSIX-style and PCRE-style regex syntax in different scripts, but that may be a matter of preference.)

Good old Perl Pie!

I had to use it more than once, typically when running on a hosed system, typically from within some miniroot/initrd/ramfs when the machine wouldn't boot and no full screen editor would run because of missing termcaps. However, cat is actually easier to use as a quick emergency editor (if you know how to use ^W,^U,^D and friends). Unless you're stuck using an ASR-33 :)

Even today 'ed' has practical uses apart from being the default emergency backup of the backup editor.

One situation where I always use 'ed' is when I reinstall a machine (with the same ip) and ssh tells me the keys don't match together with the line number in the file known_hosts. It's easy as: #ed ~/.ssh/known_hosts 15d wq

(assuming the mismatching line was 15). Only sed would be faster if it wasn't for the time spent always looking up the "-i" (edit in place) flag, so I just use ed.

Also knowing a little 'ed' will make working with 'vi' easier, assuming 'vi' is not your usual editor.

Learn ed so that vi becomes easier - nice joke!

More relevant is that I already know a bunch of tools for doing deletion of line 5. For example: perl -i -ne 'print unless $.==15' . It's more complicated, but it's a smaller number of tools for me to remember.

Actually, I probably would have used mv to a temp file + awk 'NR!=15' + rm temp file. Even knowing perl and python I still use awk pretty often, and it comes to mind much easier than thinking about ed or sed. Plus, I still have the temp file around in case I need to revert a mistake, like if I accidentally typed '51' instead of '15'.

not a joke at all. A regex you use in 'ed' command mode, you can use directly in vi after typing ':' I never made any heavy use of 'vi', but I can use it because I know a little 'ed'. Conversely, if one knows 'vi', it's easy to use 'ed' when one needs a tiny edit and doesn't want the whole screen to be filled. Even though it's just a moment, I find it distracting, especially in the ssh case I mentioned where I expect to login over ssh and suddenly I would need to change the whole screen just to make a trivial edit; 'ed' is a real timesaver there because it hardly disrupts the workflow.

I guess the benefits of 'ed' depend on your line of work though. For a systems administrator, I would make it a job-interview question.

The only vi editor I used for more than a version commit was 'elvis' on Minux. Otherwise I'm an emacs user. (I know "q", "q!", "s", "dd", "i", "/", and "a". That's all I need for commits.)

The reason I found it humorous is my difficulty in coming up with strong reasons for someone to start with ed and then transition to vi, while it's easy to come up with reasons to start with vi and then learn ed.

And your best case examples don't come up often in my experience. I usually manually edit more than one line at a time. So it doesn't seem like a very pressing reason.

Out of curiosity, I looked for what sys admin jobs call for. "Significant experience in the use of at least one Unix-based editor (e.g., ed, vi, Emacs, pico)", "Can edit files using more than one editor", "Use vi editor extensively", "Regardless if you use joe, pico, emacs or MS Word for your daily editing, those will not be available in a rescue system and vi is different." Most fall into the vi camp, many only want a (common) editor, and only a handful say "ed - it's the unix editor!", and then only jokingly.

Oh! I almost forgot to mention. I used to use BSD Mail, and at the start I used the default editor, which was 'ed'.

Ah. I guess I didn't express myself very clearly. I'm not advocating starting with 'ed' and transitioning to 'vi'. I'm an emacs user myself. But as you know, emacs is not always installed by default (especially on remote server installations). 'vi' is always there, but I already know and prefer emacs. 'ed' is also always there, and catches some corner cases 'vi' doesn't. As a bonus to knowing 'ed' (in addition to a fullblown editor), should the occasion arise when it's convenient to have a visual editor and emacs is not installed, your knowledge of 'ed' will allow you to do useful things with 'vi', even if you never normally use it.

As to what sysadmin jobs call for, having knowledge of 'ed' is not something I would put in the requirements, but it is something I would ask during an interview as it would hint at knowledge of the myriad of obscure tools that unix has, and/or having tackled delicate problems that would have required a fallback to 'ed' in the past.

As such we're mostly in agreement I think. I agree there are no compelling reasons for learning 'ed' as your primary text-editor. But there are compelling reasons against not having a working knowledge of 'ed', although it depends to a large extent on your field of endeavour.

"I know just enough vi to compile emacs" ;)

I think knowledge of ed helps learning vi, just as knowledge of all the Unix programs that come before help learning Perl because it has so many influences. It always bemuses me that some people try to learn it as their first language. :-)

I love that they felt the need for ed to have a "quiet mode".

"Well, it's terse, but let's see if we can't make it laconic"

Monday is a "Ride to work day" for motorcyclists. I think this same thinking should be applied to ed!

Think of it: One day, every year, there should be a "Use ed (the standard editor) day". On that day, all your text editing must happen with 'ed'. Who's with me?

    a
    I'm with you!
    .
    quit
    ?
    exit
    ?
    q
    ?
    q

"ed really will always be there"

When I set out to learn ed, I was surprised that it was not included by default on my Arch Linux install. It was easy enough to add via Pacman but it still surprised me.

It is a very sad and distressing trend that some Linux distros don't include ed in their base systems.

Ed is as essential to a Unix system as cat or grep.

Why? We have better alternatives, and clearly things can work without it.

From what I understand, it's used as a scriptable editor in UNIX automation. It may not be used that often, but it's something I think people expect to be there when a particular piece of behaviour is desired. (Not that I've ever used it myself either interactively or scripted).

It also appears to be part of POSIX.

I imagine though that ex would be the more capable alternative, and since vi is also ubiquitous, I guess that makes ex ubiquitous too. So I can see your point.

What better alternatives are there?

Even if you prefer vi for interactive edits, I still prefer ed for small edits.

Then there are times when your terminal is borken (or as somebody mentioned, perhaps even your keyboard is broken), this still happens more often than you think.

Recently I had to use some ajax-term thing which was just awful and completely unable to run vi properly for who knows what reason, anything that depends on curses can not be considered reliable.

But more importantly it is the standard and portable tool for scripting the editing of files.

I'm honestly curious - what's a use case for scripting the editing of files that's not better satisfied by using a scripting language? I see the value of Arch's focus on minimalism more than keeping ed around in this case, but I'm happy to be wrong.

If your terminal is broken, there are plenty of ways (reset, stty sane, whatever) to fix that. I'm also not sure why having an extra binary on my system is a good safety guard against a broken keyboard.

The ajax term case is valid I guess, though I'm not sure how common that is. I was in a case like that a year or so ago and ended up using cat and sed to get the thing back on the network.

Use case for scripting that's not better satisfied by using a scripting language:

Ed reads the entire file. If you need to move back and forth through the file, or move lines from one place to another, or reprocess the same line repeatedly, it can be far more convenient than a scripting language.

There are many editors which will do the job, it is a matter of taste. If you break your terminal by some obscure action, then fix it rather than choosing your entire toolchain around the premise that you will be spending significant time in a broken terminal.

I appreciate historical respect for the tool but this falls short of showing it to be indispensable to everybody.

Because ed is the standard text editor.

The only commands I can use in Vi/Vim are the ones I remember from ed, lol.

Like burnt in from the 80's.

I think sed pretty much displaced ed. I still use ed but sed offers the same functionality and it's easier to script. One exception is the j command for joining lines.

For better or worse, in some of today's base Linux distributions, you might find ed missing. You will almost always find sed though.

sed is great but it's a stream editor; try moving the last line to the start of the file.

sed '$G;1h;1!H;$!d'

Nice try, but efficiency aside, it breaks.

    $ sed '$G;1h;1!H;$!d' <<<foo | hd
    00000000  66 6f 6f 0a 0a             |foo..|
    00000005
    $

I was a MUD coder in a past life (and sort of am still); learning ed is essential for that environment. I'm glad I learned to use it, and gladder still I don't have to use it every day.

It's great for quick fixes and one-offs, but having to engineer with it is rather unpalatable.

I'm curious, why is it essential? I'm a bit of a MUD enthusiast myself, and I've never had to use ed. On the other hand, my favorite MUD's long-form editor (like for newsposts and journal pages) has an ed-like design.

For making changes on the mud itself, there was no other way to write code unless you were an admin and had direct machine access.

There are some vi-like mud editors out there, but they were always wonky when I tried them, and caused more problems than they were worth (weird screen redraws, keys stopped working randomly, etc.). That said, I believe Dead Souls mudlib[1] may ship with one now, but I haven't looked at the lib in a few years, so I can't say for sure.

[1] http://dead-souls.net/

EDIT: link to dead souls FAQ entry on this topic: http://dead-souls.net/ds-admin-faq.html#144

Honestly - punch cards are brill and don't use any of that new fangled electric stuff.

Cool. I've made a video describing an "Hello World" C program with Ed. http://youtu.be/jxlcIMPyAt4

No thanks, I'm sticking with TECO.

Here's a TECO that will actually work on modern systems (that is, it's not written in PDP-10 assembly language):

http://videoteco.sourceforge.net/

And the manual:

http://www.copters.com/teco.html

More information:

http://c2.com/cgi/wiki?TecoEditor

Very good tutorial for vimers.

Don't miss out on the emacs/ed/vi jokes:

http://www.gnu.org/fun/jokes/ed-msg.html

My favorite:

Let's look at a typical novice's session with the mighty ed:

  golem> ed

  ?
  help
  ?
  ?
  ?
  quit
  ?
  exit
  ?
  bye
  ?
  hello? 
  ?
  eat flaming death
  ?
  ^C
  ?
  ^C
  ?
  ^D
  ?

  ---
  Note the consistent user interface and error reportage.  Ed is
  generous enough to flag errors, yet prudent enough not to overwhelm
  the novice with verbosity.

That's one of my very favourite pieces of tech-related comedy. I still laugh every time.

Reply on news.ycombinator.com