Using sed to make indexes for books (1997) | Hacker News Reader

Using sed to make indexes for books (1997)

pement.org

67 pointskawera17 comments

Is there any non-artistic reason why one would like to do it this way instead of using any programming language that's actually any readable?

On a personal note, it doesn't require a compiler and doable on any barebones Linux box. Of course, this assumes that setup conditions as described are satisfied.

The non-artistic remark is subjective--I'm imagining those with a strong inclination towards regular expressions should find this fairly straight forward.

I write a lot of scrapers and parsers, and sed is great because it's fast and almost always pre-installed.

There's a script I use every day that takes the current date as an integer, calculates an offset, reads line of a Bible reading plan e.g. "Acts 19:1-20 Paul in Ephesus", gets the chapter number "Acts 19", and then links me to that file. I use sed to get the text between* the start and ":". I first wrote it in AppleScript on a Mac, but later I wanted to port it to a jailbroken iPhone. awk wasn't there, but sed was.

I also used sed for a Mac/Linux cross-platform script that scraped EPG data from MC2XML (information about TV guides). sed was faster than awk.

In practice these days I usually just use TextWrangler and find-replace with Regex, instead of writing a script.

* If you think you can do textBetween faster - please contribute your own implementation over on Rosetta Code.

http://rosettacode.org/wiki/Text_between

TextWrangler has now been sunsetted. :-(

https://www.barebones.com/products/textwrangler/

I know :(

I don't want to pay for BBEdit though, and I'm still on 10.9 so the compatibility hasn't caused problems for me yet.

https://www.barebones.com/products/bbedit/faqs.html#no_licen...:

”Do I have to pay to use BBEdit?

No. You can use BBEdit without purchasing a license, in the same way you’ve been using TextWrangler (yes, for free). If you want BBEdit’s complete feature set (web authoring tools, and other exclusive features), you can purchase a license any time you like.”

They also claim that gives you more than TextWrangler’s feature set (https://www.barebones.com/products/bbedit/comparison.html). If they didn’t add nagging for upgrades, it seems they simply renamed TextWrangler to BBEdit.

"Is there any non-artistic reason..."

I have seen some programmers quoted as saying they believe writing programs is "art". I think one of the interviews with Arthur Whitney has him saying writing programs is analogous to writing poetry.

Assuming one does not use backtracking as is done by Pement here, IME, the speed of sed routinely beats other, larger scripting languages for many tasks.

The basic functionality of many unix utilities can be replicated using sed, e.g. tee, etc.

When one considers the small size and simplicity of this program, and that it is probably as commonly found on unix operating systems as echo, ls or cd, knowing how to read and write sed can be very useful.

Yes, it's often a simple task and sed is sufficient to accomplish it. The sed script I used to generate the indexes for my recent books has four expressions in total, and I do not have to care about all the I/O stuff, so it is probably the easiest solution.

I guess because it's 1997 and your free readable programming languages for convenient text processing were limited to... uh... does Perl count as readable?

Yeah I didn't discover python until 1999. Incidentally, that was after writing an inter-system plumbing program in sed. The guy from Andersen Consulting was so sad, because he had confidently declared that the program couldn't be written, period, let alone in one day. Then he was happier, because he realized that he would make tens of thousands of dollars rewriting it in C++. One of the managers at the poor abused client just asked, "why didn't you write it in awk?" Sorry dude, I never used awk at university... Actually we were all wrong, because I should have used sh.

Yes. CPAN is full of readable Perl. Spare us the unfair generalization.

To be fair with the previous comment, do you claim CPAN was full of readable Perl, back in 1997?

sed and it's bigger brother awk, are excellent tools to have in your daily use toolkit.

O'Reilly have an excellent book: "sed & awk Book by Arnold Robbins and Dale Dougherty"

For awk, there is the classic book:"The AWK Programming Language by Alfred V. Aho,‎ Brian W. Kernighan,‎ Peter J. Weinberger"

There are a collection of use cases where sed and awk (and bash) are perfect tools. IMHO, the best advice is to switch to a real scripting language (like perl or python) as soon as the complexity increases. This way you never need to learn in detail all the intricacies of these tools. You just need to know the small use cases and to master your favorite scripting language. The life is too short and too many interesting domains exists to lose time mastering the quirks or sed, awk and bash. Real scripting languages do not have so many quirks and limitations. This helps to produce less buggy software. Time is precious, choose what you want to learn.

I believe The Unix Programming Environment used this technique to create their index.

A few of the early O'Reilly books.

UNIX Power Tools may or may not be one of them, but remains a phenomenal book. It's what got me "over the hump" in grokking Unix and how it and its tools worked, many, many moons ago.

Larry Peek is a God.

Unix also came with dedicated to the purpose tools for making indexes. ptx is one.

Reply on news.ycombinator.com