We were wizards – a foreword to Learning Perl (1993)
jwgoerlich.com
jwgoerlich.com
I decided to switch from sed to Perl for all my string replacements and at the time it made my life so much easier! Eventually I started expanding my Perl use to a few lines of code at the time, and now I write a few larger (100+ LOC) scripts in it a year.
Funny how my personal journey matches the story of Perl back in the 80s: it was born to replace more primitive tools.
-lN, --line-length=N, -l N, --line-length N
All work the same, set line length to N.
-iSUFFIX --in-place=SUFFIX, -i NEXTARG --in-place NEXTARG
Don't all work the same. The argument is only actually taken if it's attached to the flag directly. In the latter two forms, no backup is made.
GNU mktemp is similarly annoying, but different, with the --tmpdir flag.
-pDIR, --tmpdir=DIR, -p DIR, --tmpdir NEXTARG
Only the last one is different. If you use --tmpdir literally without attaching an argument to it, it defaults to $TMPDIR, otherwise it takes in DIR.
This is inconsistent and unpredictable. A flag should either take an argument or not.
I also dislike Python's argparse variable-length nargs behavior for similar reasons. Even with an integer nargs, it makes for an ugly command line, but with variable-length ones it just gets hideous, and can make it impossible to pass an argument beginning with a hyphen to a flag.
This was the same road I took in the late 90's - started with simple scripts and ended up with more and more complex scripts when I was learning all the shortcuts. Until the day I could not understand anymore my own code from a few days back :)
I believe they should have been Py3 when they wrote them. It was not my decision, but it did become my problem.
perl6 is now called raku
The cool thing is that Perl is compatible, and maintaining compatibility is the top priority for Perl developers. so, many distros keep updating it because it's very safe to do so. When many projects switched from GPL2 to GPL3 macOS famously decided to either freeze some bundled tools, switch to alternatives or removed them entirely. It took them years and years, but all this time Perl has been a notable exception.
As a result nowadays you can be very certain that every machine you run your code on has a very recent Perl installed, so you can use many new stuff, too. It's an old language, but the code you can write can be pretty modern, which is nice.
It's still Perl: lots of symbols, some variables get set up implicitly for you, some keywords look silly to modern eyes (`my` to declare a variable, `sub` declares a function). Some newish stuff is handy. Like, these days you can specify types of parameters in function signatures, and it does a rudimentary type-checking for you. And Perl regexes are still better than in ANY language out there, for 35 years and counting!
If we take Apple at their word then it's an exception only in that they haven't got round to it yet, not that it isn't also going to be culled -- the Catalina release notes mentioned Perl explicitly alongside the other two:
Scripting language runtimes such as Python, Ruby, and Perl are included in macOS for
compatibility with legacy software. Future versions of macOS won’t include scripting
language runtimes by default, ...
The trend then was towards standardization towards linux, but then that got compromised as that's when macbooks got popular with developers.
One such case is adhoc small scripts mid pipeline. There's just certain Perl syntax that's so deeply embedded in me that no matter how much I try to learn sed/awk/tr, I can't help but reach out to my old love in times of weakness and need. I just know how to talk to Perl in the most intimate of ways that's been hard to reproduce in other languages.
I still feel a little like I've betrayed Perl a bit every time I use Python.
Perl, Larry Wall, Randal Schwartz, Tom Christiansen, brian d foy, O'Reilly, and the many other participants of comp.lang.perl.* had such a lasting influence on my programming career. I wouldn't be who I am today as a programmer without having experienced all the Perl community put out into the world.
Even though wizards foretold nPm by inventing CJAN - like Cpan for JavaScript. And also IO::Async and Lehman’s work was very much ahead of NodeJS.
But we were not brave enough to transpire to JS neither .Net, nor JRE so others figured out how to do it better before us… even though we tried.
Finally like all proud wizards we eventually split the magic school in two faculties and went in ever deeper obscurity and magic.
The submitted post exemplifies the worst characteristics of what we can call coding culture. Larry Wall was a polarising character. He could be witty but also tiresome.
The C and C++ coding cultures have had their problems too.
Python has been successful as a language and as a coding culture. I don't like the bigotry of the Python community towards other languages, but the openness and focus on coding ergonomics has been different from most (all?) other prominent languages.
It can't hurt to have a couple of MAANA (was FAANG) giants providing material support.
> And we had too many sigyls and weird symbols
This I think is an unfair criticism frequently aimed at Perl by people who haven't reached fluentness. The language is expressive. To attain higher complexity, languages will adopt new symbols and forms. Anyone who knows Python and uses DSLs such as pandas and numpy knows that "sigils and weird symbols" make their appearances in the mutant notation.
Python has its own TIMTOWDI at this point.
To whom? Unless you were working on the Perl language itself, there was no need to interact with him.
I joined the Perl6 dev mailing lists, listened in on them, offered my support, gave a broad stroke on design ideas, backed them up with reasonable arguments and asked for comments. Larry chimed in basically saying, "no, because this sucks" and not too long later the Camelia butterfly[0] thing was born.
So yeah, I bounced.
Larry may be brilliant, but maybe he also has a history of not knowing when to delegate tasks (and trust). And now Perl WAS something, rather than IS something.
We all ask the children to name the pets. Is cute, but often we live to regret it. With logos is the same.
I am qualified to help design a logo. And if the feedback was, "great ideas, but not the direction I want to go. What about..." or anything similar, that would be a fine way to show that my offer for doing free work is still happily welcome, with reasonable direction from the owner of the project. You know like a discussion.
There's some great OS logos. The Linux Penguin was great. BSD Daemon, etc, etc, etc. I love the camel, but that's not owned by Perl. It's not an impossible task.
It's Python's dependency management, and there is more than one way to do it:
pip, pipenv, poetry, conda, setuptools, hatch, micropipenv, PDM, pip-tools, egg, ActiveState platform, homebrew, or your operating system's package manager.
Relevant xkcd: https://xkcd.com/1987/
Yes. This. Thank you for calling this out.
(Not saying it was ever my paradigm for elegant language design or that it's their job to conform to my taste.)
Perl allows to come at peace with UNIX being done in C, by exposing similar programing language capabilities, mixed with Lisp like wizardy, without having to deal with C security faults unless required for performance reasons, that could not be improved in any other way.
But in reality it is how reality works: more complicated things consume execution time: a is Int.range(1..3) IS NOT THE SAME AS: "load 1 to register A" - it contains 'if' or few on every single use. Or function call(s).
We need CPU's that support such things natively (or via cooprocesor or something, with atomics and full transactions :) ) or need to accept slow code or layer like OS for "rich types" and "type safety"...
on my first day of my first software internship in the mid 90s, my boss handed me the ora programming perl 5 book and said "read this." it served me well as a swiss army knife for many years.
There are many ways to have OOP? Moose, Mouse, Moo, MooseX::Declare, Class::Accessor — you are pretty much guaranteed to pull in them all and have them coexist in runtime. Now Corinna or whassname is coming in, but it has no users yet so it doesn't count. JSON? You're guaranteed to have both JSON::XS and CPanel::JSON::XS. The list just goes on and on. HTTP clients? You have Furl, LWP::UserAgent, something built atop IO::Socket. TLS with HTTP? LWP::UserAgent::SSL, IO::Socket::SSL, and someone will use Net::SSLeay raw just to watch the world burn. You pull in all of the latter ones courtesy of modules doing API integrations because their authors just have their favorites.
Web? Oh, you want to do web. Mojolicious, Dancer, CGI::Application, just CGI.pm... Granted, at least you won't have libraries pull those in willy-nilly, thank goodness.
I think this is hyperbole. Generally, you won't be mixing Class::Accessor and Mo* based dependencies unless you are dealing with a shitty, barely maintained legacy codebase.
There was a progression to things, mostly because of performance reasons why there is a bit of a proliferation. Moose (and the underlying Class::MOP) were about providing a meta-object protocol for Perl which didn't exist before it. As such, it was focused on being correct over fast. A whole community sprang up from stevan's art. Then sartak's Mouse came along and made some performance improvements with some compatibility caveats. Finally, mst's Moo stripped away the "reflection" aspects of Moose to really squeak out even more performance.
MooseX::Declare was always a (beautiful) research project. I spent some time in that space welding POE and MooseX::Declare together. But these weren't suitable for any kind of production code. If you see this live somewhere, you should definitely rip it out lol.
As for TIMTOWTDI, this is sorta why project Andy Lester's Phalanx came about (it was trying to bless a chunk of modules as a sort of smoke test for Ponie (perl5 implementation on the Parrot VM (which was the first step toward a Perl 6, now Raku)). It was trying to provide an opinionated set of high quality libs for SWE.
Unfortunately, working in Perl really requires a bunch of skill and discipline if you want a long-term maintainable project. This means really looking at your dependency tree and making difficult decisions on which libs to use specifically to avoid installing all of CPAN. npm's installing the world behavior was foretold by Perl.
(Looks at the dependency graph) No, it’s a fact.
> Generally, you won't be mixing Class::Accessor and Mo* based dependencies unless you are dealing with a shitty, barely maintained legacy codebase.
I don’t. But the modules my project depends on (transitively) do, because those modules were developed and last touched during different fad eras. So when the app starts, I have, like, four OO systems, three ways to do JSON, two to do YAML, and four ways to do HTTP in the memory.
Also, the authors of said modules tend to have Egos and Opinions (capitalization intended) that prevent them from converging on common things.
Don’t fall into the meme trap of hating on Perl. Although it’s only now used in a handful of companies run by OGs and possibly by devops, I urge you to take a look and spend a month using it - really using it. .. because it really is a beautiful language and it really makes Stream-of-Conscious programming the norm
I miss it, but understand why it fell out of favor. It's the closest I've ever gotten to singing whalesong with a machine.
As fast as you type you're coding. Even though I've now spent years in other languages compared to Perl, it's still stop-start in every other language.
I've said it many times, but Perl really moulds to your brain rather than the other way around. You bend Perl how YOU want it.
I once had a long conversation about this with a linguist and Perl programmer that I randomly met in a pub. However, I was quite drunk and don't really recall too much, and we never saw each other again.
Computer programming theory has large swathes of inspiration from theoretical linguistics - think lisp. That's nice for theoreticians because it makes things like parsing easier to think about rigorously. Perl on the other hand, to my knowledge is the only significant programming language to be inspired by practical linguistics, and therefore appeals to concepts like context and ambiguity way more than other languages. This means that a lot of computer scientists academics absolutely hate it, and so it was very much neglected in the education space.
The other side effect of this is you still get a good number of talented programmers with backgrounds in the humanities and social sciences who do very well on perl.
About 15 years ago I was put on a Java project that ran for a year with half a dozen developers toiling away (along with an outsourced team that management eventually added to write unit tests). I spent half that time fighting with Eclipse and the other half divided between tearing my hair out trying to wade through the AbstractSpringFactoryInterfaceAbstractFrameworkInjectorAbstractionBuilder nightmare, being lectured at by people who learned everything they know about computers from in-flight magazines that this is the way professional software development is done, waiting for the one person who could actually build the application (which he did partly by unzipping the jar files and manually tweaking their contents), and actually coding.
To my complete lack of surprise, the project never achieved a working milestone and was cancelled. What did surprise me was that a coworker who was sympathetic to my grumbling persuaded the CTO who was trying to rescue the project to let me try it my way. On my own (using Perl and PostgreSQL), in a couple of weeks, I had a fully working prototype. We got the green light to proceed and built the company's flagship product on that stack. It was not my brilliant engineering as much as Perl's extraordinary ability to connect mind and machine (coupled of course with my brilliant engineering) that made it possible. I owe Larry Wall for a huge career boost.
Funny, a lot of people say that about Lisp...
Python and perl are actually pretty much exactly the same - analogous to the Judean People's Liberation front versus the People's Liberation front of Judea. However the languages are optimised for a slightly different purpose. Python helps you to think more like the computer does, whereas perl helps the computer to think more like you do.
CPython's implementation is also straightforward enough that you can look up ceval.c to get a gist of what an opcode is doing
Incidentally, the fact that you cannot write this type of one-liner in any meaningful way with Python is one reason I do dislike that language.
python -c '
import re, sys
for line in sys.stdin: print(line.lower().count("foo"))
'
python -c 'import re, sys; print("\n".join(str(line.lower().count("foo")) for line in sys.stdin))'
python -c 'import re, sys; print(*(line.lower().count("foo") for line in sys.stdin), sep="\n")'
python -c 'import re, sys; set(print(line.lower().count("foo")) for line in sys.stdin)'
While Python is my preferred language for most tasks, I still use perl as an ad hoc stream processor with `perl -pe` and nothing really beats it at the task. Anything more than 10 lines and I find python easier to deal with (or at least it's my comfort zone).
It hearkens back to the days when motor cars were curiosities that required frequent tinkering and had no safety requirements. The sky was the limit, but your car could blow up.
Perl was rightly bashed because it was the poster child of cowboy coding, which flies in the face of good engineering practices: Understandability, predictability, reproducibility, measurability, safety. This allows you to use a product with confidence. I had the misfortune of inheriting some cowboy code, and to this day I call a pox upon the writer. His lack of discipline caused YEARS of misery and maintenance costs; far outweighing any temporary benefit he provided.
Newer programmers might not have lived during the bad old days of cowboy coding, but I did. Nothing was documented. Nothing really worked because there was no edge case support. The mentality was "If it was hard to write, it should be hard to understand" and "If you're not happy with it, feel free to modify the code" (their impenetrable code). Basically, pass-the-buck for your own lack of discipline, and then blame the user.
Personally, I'm glad we've moved on towards becoming a serious engineering discipline, complete with UX being a thing.
That’s the meme, but it’s simply not true.
I’m going to use Python as a basis of comparison because that’s the language which, from my perspective at least, seemed to replace Perl.
> understandability
A lot has been said about sigils but ultimately a language is only as familiar as a developers exposure to it. Python appeared easier to understand for the newbie because it favoured words over expressions but now that Python has matured it’s become just as incomprehensible too.
> predictability
I’ll be the first to admin that there are a couple of really massive footguns in Perl which are remnants of its evolution as a shell tool. But there’s other aspects to the language which are really clever. For example the way Perl does type comparison is much more predictable then Python.
On the whole, I think those languages are about equal.
> reproducibility
CPAN was also light years ahead anything also available at the time. And still is in many regards. For example, Python environments is a mess. Perl is one of the most portable languages out there. Want to change the state of the runtime? Just add a line to the top of your source. Simple.
Python and JavaScript are a completely pain in the arse in comparison.
PHP is another example of a language that gets this right. Few people on here (myself included) will have much love for 2010’s PHP but it just goes to show that a lot of the languages “real programmers” praise do actually suck at a lot of the basic principles.
> measurability
Not really sure what this is intended to refer to. Maybe APMs? Either way, a lot of diagnostics like that came into vogue after Perl’s decline so this isn’t really the fault of the language but more just 3rd party tool writers putting their focus elsewhere.
That all said, don’t be fooled into thinking such tooling doesn’t exist for Perl.
> safety
Perl basically invented unit tests amongst the scripting languages. It also has stricter run modes to differentiate between one time scripts and important business code.
Saying Perl lacks safety demonstrates a lack of experience with the language.
——-
I’ve written production Perl, PHP, JavaScript, TypeScript, Python, C++, Go, Bash, Visual Basic and even Pascal. De-mangled other people’s spaghetti code in all of those languages too. Probably a few languages I’ve forgotten about too. Perl is one of those languages people love to hate but is, in my opinion, one of the most misunderstood.
[edit] I should have added a comment about Perl’s backwards compatibility. Few other languages even come close to the longevity of Perl 5. And that commitment extends down throughout Perl’s ecosystem. With JavaScript I’m constantly scared the next npm update will break the project.
Don't forget Tainted Mode! It's 2024 and I still have never seen it adopted in any other language. It's essentially SELinux-lite for variables :)
IMHO Perl lost to Python because of 3 reasons: 1) Google at the time chose Python and it was the new cool startup everyone wanted to work at, 2) Pandas and Numpy weren't matched in the market (I'd even argue this is still true in 2024), and 3) Perl 6 was way too ambitious and scared people off (even though Perl 5 was never going to go away overnight (and still hasn't))
You forgot safe mode and tainting, which while imperfect and has some sharp edges, is a first class construct that when correctly wielded outright prevents classes of attacks like shell or SQL injections or XSS.
Some have been reinventing it in a piggybacked way, and so, partially/poorly (comparatively), e.g Rails's safe/unsafe strings in ActionView.
But I agree perl can have bad management problems because of different people's capabilities, and different people's approach to showing their individuality.
I reserved my criticisms of Perl for things like signal handling and it's implementation OOP(to name a few), but overall it's a good language.
Even today many of the "serious engineers" of the time still roam the industry and show up for interviews every now and then, completely unable to write code. Some even capture younger souls into this trap, a sad picture.
To quote the immortal Bender:
>> Oh wait, you're serious, let me laugh even harder.
People waxing poetic about their time using Perl back in the late 1990s/early 2000s don't have a vote because they clung onto good bits and have long forgotten the ugly.
> it really makes Stream-of-Conscious programming the norm
Are you sure it's a Good Thing? Really sure? What if you have teammates? What if you have a life? What if you need to revise your stream of consciousness after a year? Still sure?
Yes I'm sure.
Having worked in many languages over many decades, I've learned that a high quality codebase is built by high quality developers, largely independent of the languages they use. You can build an unmaintainable mess in any language as easily as you can build a high quality codebase in Perl. I would concede that you can more easily build an unmaintainable mess in Perl than in many languages but that's up to you; you can do most things more easily in Perl than in many languages ;)
The happy medium is to write your program twice: Write it solo in Perl for rapid thought-to-code, figuring out architectural issues along the way. Then, write it again in Python for scalable collaboration using the revised architecture. This gives you a working solution for the problem FAST and adheres to "Do it, then do it right".
Unfortunately, this doesn't scale to systems. I do hope LLM-driven transmogrification becomes a real thing.
I have an affection for both Forth and Lisp, two languages that have also been called highly productive but write only.
I think a working early version- even if its a spreadsheet- is 10x of a supposedly 80% done version of something else. The latter of which I must now return to unfortunately…
And that still happens at stream-of-conciousness.
I use Perl a lot because it doesn't get in the way. The functional aspect of it lets you map from problem to solution easily.
For all of Perl’s shortcomings, I’ve always felt that no other language came with Larry and Randall’s sense that programming was fun. The Perl folks always understood that our skills were rare, and the powers they conferred were inherently magic - even if a bit dorky.
Being in-process certainly sped things up. And it was easier to understand than the shell based solution.
But it would have been easier to write and read in just about any other language.
Perl 4 at the time only had very primitive data structures. Most of the time you were concatenating and splitting strings together to mimic real data types. The baroque way file handles were a first class data structure made abstracting over them painful and hacky.
That $foo and @foo and %foo were different variables caused innumerable bugs.
The exposure of function arguments as a stack directly in the language made for weirdness all its own.
The mimicry of shell expressions brought its own pain.
And finally, Perl’s greatest strength and horrific weakness: the regular expressions. One liners that would bring tears to your eyes the next time you had to debug it.
I get where Wall was going with Perl, but the emphasis on beautiful natural language for various idioms really crippled the language horrifically from the start. Going into Perl was forgetting nearly everything you knew from other languages and entering this unreal fantasy forest of code, with monsters lurking behind every rock and twist in the road.
No language even comes close to Perl’s regex support. First class support for regexes as programming tools, and unparalleled speed.
As a die-hard Perl fan, that seemed a ridiculous oversight for a language being promoted for Internet programming.
Did not know that about the first Java release. That shit is bananas.
Many eons ago, I remember me (and my team) being so tired of writing bash wrapper scripts - we just didn't know any better. To my surprise/delight, I discovered that the Solaris boxes we were using came with Perl pre-installed (which in hindsight is totally expected). So I decided to learn about it the proper way and the Llama book was exactly what I needed.
People have called Perl code line noise. But when I read the book for the first time, everything make sense to me because my mind likes mnemonics. $ is for ($)calar, @ is @(rray), % has a pair of circles like key/value pair and so on. I didn't find it odd that you'll get the array in list context and count in scalar context - it made complete sense. Till date, Perl is the language which I reach out for short-term tasks which need to be done immediately, because it helps me get the job done at the speed of thought.
I picked up some bash scripts as candidates to be re-written in Perl and I will never forget my mind being blown by the sheer lightening speed of Perl scripts; stuff that took many seconds or even minutes in bash were done in a second or less, even with my unrefined code. It was like a new world opened up and indeed, I felt like a wizard.
With Python, you have lists, tuples, and dictionaries. Those are data structure objects that are useful for different things. Tuples use (), lists use [], and dictionaries use {}. After you learn that, you just learn the few methods and manners to populate them and you're done.
There's nothing wrong with either way of course. Both appeal to different kinds of folks.
[1] https://hackaday.com/2024/01/09/floss-weekly-episode-765-tha...
.sh Saint Helena
.py Paraguay
.rs Serbia
.md Moldova
.tf French Southern and Antarctic Lands
.cc Cocos Islands
.so Somalia
.bz Belize
.ps PalestineTo this day I still recommend it even if opportunities to use Perl are gone.
If I had to use a language just for Unix scripting, it would probably be Perl though.
But yet, Randal had beat into me the idea of explaining "why" on everything. You'll notice the basic pattern in Learning Perl and my other writing is that we show the expedient thing, tell you how that fails, improve that a little, show how that fails, and end up at a better final solution. That path shows quite a bit that's ancillary to the task at hand, but also shows how the pieces work together and how different parts affect the solution. I'm usually frustrated by other language tutorials where they say "just do this" without saying anything about why the various parts are there.
Learning Perl was originally written with a C programmer audience, so it assumes that the reader is comfortable with ideas such as variables and looping. I have another book I might recommend for beginning to code: Squeak: Learn Programming with Robots. Sure, it's Smalltalk and literally written for teenagers, but I think it works. It reminds me of the fun days of Logo.
Two things i didn't like about it which i still don't
1. It's a conceptually huge language. There are several things to learn to be effective or you fall into the problem when the 10% that you use is different from the 10% that your teammates use. I found python attractive because of this.
2. I disliked Walls book. It was long winded and very hard to sit down with when I wanted to find something. I expected something like K&R but found it tedious and boring.
As for the Camel book, what I liked about it is that it explains why Perl is the way it is... the underlying philosophy. Once you know that, it starts making a lot more sense. You usually didn't get that with books like "Learn Perl in 21 days" or something.
One way to do things? Pfffft.
However, this was around 2005 or so and at the time, Python was significantly smaller (conceptually) than Perl. Atleast to me.