Why does OpenBSD still include Perl in its base installation? | Hacker News Reader

Why does OpenBSD still include Perl in its base installation?

marc.info

100 pointsmfontani157 comments

Perl 5 is in my humble opinion a really convenient to have installed. I use it all the time as a replacement for those shell scripts that become too big or too cumbersome, or to remove out of the equation the quirkiness of different implementations of standard tools (i.e. GNU sed has certain flags that BSD does not have, gawk is different from mawk, etc). A Perl onliner is often faster and works the same way pretty much everywhere.

That's exactly how I treat Perl as well. Things like lists and hashes make it easy to do things that are difficult or impossible in bash.

If my bash script gets much beyond simple looping, I switch to Perl.

Why Perl over Python though? Most Unix distros come with Python bundled and Python has a bigger ecosystem of libraries etc available. And it's generally easier to read/write.

>> Python has a bigger ecosystem of libraries etc available.

CPAN (http://www.cpan.org/index.html) is quite large and has libraries to do almost everything you can imagine.

I am not sure if CPAN or pip has more libraries though.

>> And it's generally easier to read/write.

Python looks nicer, but not everyone likes significant whitespace.

The Python 2 to 3 change has also caused some adoption issues. The Perl 4 to 5 happened quite a while ago. Perl 5 has been integral to Unix-like systems since the web went mainstream.

Yeah and the Perl 5 to Perl 6 move... oh wait...

And Larry & Co realised that it was sufficient different that now it is no longer called Perl 6.

and yet kept the logo that literally has P6 embedded in it

Exactly. Widely-used programming languages continue to be used even when the authors move to a new version.

Perl 5 lives on and is still developed and maintained: https://www.perl.org/get.html

Perl 6 became Raku: https://www.raku.org/

Python 2 is still in use, but officially deprecated: https://www.python.org/doc/sunset-python-2/

You could probably start a business maintaining a Python 2 distribution and selling support to businesses and organizations that rely on Python 2 and are not in a position to upgrade: https://www.activestate.com/blog/python-2-eol-report-card-is...

Uh, raku looks interesting, the web site is also quite accessible. Sad that it took so long, maybe Perl6 would have regained some territory otherwise.

Writing this while looking at CrewDragon docking at the ISS. Which took SpaceX 18 years, and many people thinking they would never make it. If anything, an inspiration to the people currently working on and using Raku!

Perl5 is not EOLed, so no one is forced to "move". I haven't heard of any real business actually moving from Perl5 -> Perl6 (Raku, to be precise) for support/legacy reasons.

Perl 4 to 5 was before my time, but I read it wasn't really a major change, just to sell books. Perl 5.6 to 5.8 was the big moment where strings became (optionally) arrays of characters instead of arrays of bytes. This caused some amount of breakage, but was a long time ago now; anything else broken since then has been pretty minor.

/usr/bin/perl scripts written more than twenty years ago continue to work.

So do /bin/sh and /bin/bash scripts.

/usr/bin/python scripts IME depend on what /usr/bin/python points to; some break/broke due to the python 2 to python 3 switching, which hasn't been as backwards-compatible as previous examples have.

But! If you write modern bash, it won't run everywhere, definitely won't run on mac!

Bash may be installed in /usr/bin (Arch Linux + symlink in /bin), /usr/local/bin (OpenBSD) or not at all, though.

"#!/usr/bin/env bash" or "#!/usr/bin/env python" are the recommended ways to run both, AFAIK.

Where is bash not installed?

Bash is not installed by default on any *BSD I have used (FreeBSD, OpenBSD, NetBSD, etc.). You need to install it from the ports tree or package repo.

Most (all?) commercial Unix I have used also don't have bash, but those systems are pretty dead these days.

OpenBSD's fork of ksh is actually really good so I typically feel no reason to ever install bash on there. On others, the default shell kinda sucks and I tend to install bash. In particular the old /bin/sh on older Unix tends to be worse at interactive use and not support some constructs like {a,b}

Except Ubuntu where they ship dash by default which is not really bash compatible.

I think you might misunderstand me with what "bash is not installed by default" means. On the systems I'm mentioning bash is not on disk anywhere in the base install and you have to install it extra. I don't think that is the case in Ubuntu.

I'm typing this on FreeBSD. It's in /usr/local because it came from ports.

    $ which bash
    /usr/local/bin/bash

Now I try my OpenBSD machine. It's not installed at all.

    $ which bash
    which: bash: Command not found.

> Except Ubuntu where they ship dash by default which is not really bash compatible.

Why would you expect it to be?

dash is a POSIX shell:

* https://en.wikipedia.org/wiki/Almquist_shell#dash

Even when called as /bin/sh, which implies Bourne/POSIX behaviour, bash leaks all kinds of functionality into the environment such that if you tru to run your "/bin/sh" script on another shell it may not work.

We had all sorts of issues when Debian made the switch: we had to tell people either (a) fix the code, or (b) change your shebang to bash.

If you want Bash behaviour call bash. It's like when I type in "vi" in the command line and I get an editor with all sorts of colours: WTF? If I wanted Vim I would have typed in "vim", but for editing a small file all I need is POSIX vi without the dog and pony display show. /rant

I have some FreeBSD and OpenBSD machines. /usr/bin/env python doesn't work on either of them because the ports call it python3.7, python2.7, python2, etc. and there is no symlink simply called 'python'.

Sometimes I used to create such a symlink myself but I don't use or run a lot of python day to day so my current systems do not have this.

Because Python is sometimes too "formal" for dirty glue scripts that consist in invoking other commands and parsing their output. Perl blows pretty much every other language out of the water when you need to process text fast and without too much cruft.

Perl's being more relaxed about creation and manipulation of data structures helps a lot, too those kinds of scripts.

Consider this:

  my %h = ();  # an empty hash

  $h{'foo'}[3]{'bar'} = 'bletch';

Now my hash contains one entry with key 'foo'. That entry is a reference to an array of 4 elements. The first 3 elements of that array are undef, and the 4th is a reference to another hash. That hash contains one entry with key 'bar' and value 'bletch'.

In a lot of other languages you would need to explicitly allocate an array of at least 4 elements for $h{'foo'}, and then explicitly allocate a hash for $h{'foo'}[3].

With Perl they just come into existence when you try to store something in them.

Is this always good? No. There are many problems where this automatic reification would just hide bugs. If you are, for example, processing data that is supposed to be clean and well specified then parsing it into explicitly allocated data structures that match what you expect and will throw errors if you try to put something in out of range is great.

But when you are dealing with data that is not clean and not well specified, being able to get it all in your data structures easily even when there is something wrong or inconsistent in the data can be great. Get it all in, and then you can run cleanup code on it looking for unexpected things, and fixing them or alerting you that you need to add more cases to the fixing code.

There's a similar thing on the reading side of Perl data structures. Try to access a hash via a key that does not exist, or an array by an index past the end? You simply get back undef.

Is this always good. Like automatic reification, the answer is no. There are many problems where it would be a grave error if the code tried to access an element of a hash or array that it had not previously but in there.

For problems where your logic would include a lot of places where you check if an element exists and if it does you add to a total or if it is a non-empty string you include it in something else, the code can be cleaner in a language with Perl's approach. Adding undef to a number is like adding 0, and undef compares as equal to an empty string.

> Is this [autovivification] always good? No.

Good thing then that the programmer has the choice to use it or not.

    repl>>> no autovivification 'store'
    repl>>> my %h
    %h = ()
    repl>>> $h{'foo'}[3]{'bar'} = 'bletch'
    Runtime error: Can't vivify reference at (eval …)

> Is this [returning undef for non-existing keys/indexes] always good. […] the answer is no.

Good thing then that the programmer has the choice to use it or not.

    repl>>> use Hash::Util 'lock_keys'
    repl>>> my %h
    %h = ()
    repl>>> lock_keys %h
    ()
    repl>>> $h{no_such_key}
    Runtime error: Attempt to access disallowed key 'no_such_key' in a restricted hash at (eval …)

    repl>>> package Array::Bounded {
                use Tie::Array;
                use base 'Tie::StdArray';
                sub FETCH {
                    my ($self, $index) = @_;
                    die "Attempt to access out of bounds index $index in a restricted array" if $index >= $self->@*;
                    return $self->[$index];
                }
            }
    repl>>> tie my @a, 'Array::Bounded'
    repl>>> @a = ('a'..'f')
    @a = ('a'..'f')
    repl>>> $a[5]
    f
    repl>>> $a[6]
    Runtime error: Attempt to access out of bounds index 6 in a restricted array at (eval …)

I feel that's an apples vs oranges comparison. I'd prefer to write complex scripts in python, but for simple scripts, I find a mix of bash and perl to be useful.

If you want to make a comparison though, I think perl vs sed is a more meaningful one. You can do things that's possible in sed with perl one-liners in a more portable way. It's installed by default on major Linux distributions, and even works on macOS without the GNU vs BSD incompatibilities.

> I'd prefer to write complex scripts in python, but for simple scripts, I find a mix of bash and perl to be useful.

Unfortunately, simple scripts that are found to be useful often grow into complex scripts.

A few weeks ago, I wanted to write some functionality that seemed "simple" enough to be a good fit for make+bash. A week later, I ended up tossing the whole thing and rewriting it in python.

Not all scripts needs to be complex, though. There would always be demand for quick and easy scripting languages fit for handling simple cases. Yes, some scripts are bound to grow complex, but that doesn't really justify writing every single script in Python. It's just better to make the switch once the benefits exceed the costs.

In principle, I agree, but in practice, I often find that it's common to make the switch too late, so I'm leaning towards making the switch early.

I find Perl easier to write, mostly because my personal experience has a lot of C and shell scripting, and perl borrows a lot from both.

It is useful and not difficult to be productive in, but does manage to feel more unnatural than both of those though.

That's how I use Python for the last few years.

My scripts end up longer and more verbose typically (I might just not be great at Python) but also a lot more maintainable (I am not great at writing maintainable Perl and I use Python more often).

I also use Node.js for this a lot because of the ecosystem containing so many accessible scripts and dependency management while not great is heaps better than most other ecosystems.

Of course OpenBSD can't do that with Python rather then Perl (as explained in that email). They have a different set of constraints. I am only saying in practice I have found Python to be the better alternative for my personal stuff.

Perl is a write only language for me.

I've got 20 year old python scripts the have been upgraded over the years that are still maintainable, easy to read and do the job just fine.

When I read the perldocs much of it looks write-only. But in keeping with Perl's spirit of "there's more than one way to do it", one can actually write straightforward Perl code that is as understandable as Python.

I'm a Perl fan, but there are some places where it's hard to avoid ugly code. Like collections of complex data structures, especially combined with references. No matter how hard you try, you get messy looking code.

This is exactly when I switched to Python, when I realized that I always had to look up how to dereference pointers to non-trivial objects.

For what it's worth, there's now postfix dereferencing which avoids the whole having to go back and wrap @{ ... } around things.

I can write code that is easy to maintain even after not seeing it for decades, and I can write code that I have trouble maintaining after a long weekend away from it, and I can do this in nearly any language I know.

Take good old fizzbuzz. Want to write it clear and simple in Perl? No problem:

  for (my $n = 1; $n <= 100; ++$n) {
    if ($n % 15 == 0) { print "fizzbuzz\n"; }
    elsif ($n % 3 == 0) { print "fizz\n"; }
    elsif ($n % 5 == 0) { print "buzz\n"; }
    else { print "$n\n" };
  }

Want to get clever and complex in Python? Here you go:

  def fizzbuzz(n, *args):
    cur = ['' for x in range(1,n+1)]
    for m, postfix in args:
      cur = [y+postfix if x%m==0 else y for x, y in zip(range(1,n+1), cur)]
    cur = [str(x) if y == '' else y for x, y in zip(range(1,n+1), cur)]
    return cur

  print("\n".join(fizzbuzz(100, (3, 'fizz'), (5, 'buzz'))))

Of course you could do an unclear and overly clever Perl version, too, or a clear and simple Python version.

The point is that python leads you to clear and simple. No one would write the monstrosity you did, not even by accident, while in Perl, even that little fizzbuzz looks noisy cus of the explicit new lines

I haven't used Perl recently, but the last time I did there was a say function, so

    say "foo";

was equivalent to

    print "foo", "\n";

My point exactly. Python begginners would have trouble recognizing the second option, which is good, because that’s the bad option: “Omit needless words”

Perl obviously even lets experts shoot themselves in the foot on the simplest code, see the fizzbuzz above me, While a beginner would never write the complex python version that was shown, not even by accident.

Optimizing for beginners is fine, but let's not act like it's "correct". It's a Python design choice, some people like it, but there are consequences to it. Like almost all language design choices, it's a trade off. Usually the trade-off for simple to learn is that it's restrictive to professionals and experts. For new users, teaching classes, and bringing amateurs onto an existing team, it works well. For a team where everyone has 5+ years of experience in Python, I imagine there will be some annoyances where they fell boxes in by the language. That's the trade off in action.

An example of a langauge at the extreme other end of the spectrum would probably be APL. It's indecipherable gibberish to me every time I look at it, but I'm not even an APL amateur, much less professional, and that language seems built to be used by people that are very familiar with it. Is that wrong? No, they just aren't optimizing for learnability and community growth, as Python does, instead opting to make it as simple and useful as possible for experts to get what they need accomplished.

You can counter by saying that most code is read more than it's written, so readability is more important, and to that I would say that it depends on the type of code (scripting languages lend themselves to a lot of one-off scripts), and that readability is also a function of experience. If you want your code easily readable my amateurs, then keeping the capabilities you can use to those an amateur can understand is beneficial. If you want your code readable and understandable by experienced programmers, use constructs and control flows that they are experienced with, which is a much wider set of capabilities usually (if the language supports them).

How is that version of FizzBuzz shooting oneself in the foot?

Even that Perl isn't as tidy as it could be. I'd be inclined to have `for my $n (1..100)`, and use `say` instead of print. I might even skip the `== 0` but just saying `(! $n % 15)`, but I'm on the fence about that.

I also wouldn't keep all the blocks on the same line, but it can be forgiven in a comment box.

> I'd be inclined to have `for my $n (1..100)`

I actually had that at first...

> I might even skip the `== 0` but just saying `(! $n % 15)`

...and that too. I changed them to be more likely to be obvious to even someone who has never seen Perl before. A lot of languages besides Perl have a "for" loop variant that is like a C "for" loop.

The formatting was indeed because it was in a comment box. I didn't want to take up too much vertical space on the page.

I would have written that Perl code as

    for (1..100) {
        if    (not $_ % 15) { say 'FizzBuzz' }
        elsif (not $_ % 3)  { say 'Fizz'     }
        elsif (not $_ % 5)  { say 'Buzz'     }
        else                { say            }
    }

Just saying

Perl is a write once language for me.

In that I write it once and 20 years later it still works even though I upgraded the compiler many times.

I've tried to use Python for this kind of scripting, too, but went back to Perl. Python-the-interpreter is annoying. Is it /usr/bin/python or /usr/bin/python2 or maybe even 3? Does this server image / distro even come with Python? Sure, Python-the-language is nice and all, but when we talk about 1-10 lines of code it's not a big deal, and Perl... It's just there. Even on Windows Git Bash and mingw come with Perl.

Python's subprocess is a bit more clumsy than Perl's built in features for running commands, with the proviso that running commands in Perl is best described as "idiomatic". Maybe if you didn't learn it 20 years ago like I did you'd have a hard time with its irregularity and colloquialisms.

Is there any non-shell scripting language that makes it as easy as shell to run external commands, redirect I/O etc.? I don't know of any. (Perhaps that is the very definition of a shell.)

Agreed. I really want a language that makes it easy to run subprocesses and do other Unix/shell magic, while being more straightforward and modern than Perl.

I think a good way of doing this would be to make a shell that has more expressive power and a clearer syntax.

Ruby uses the same sorta commands as perl does I think.

This is one of the few things really missing from python. I am wondering how easy it would be to write a PEP for an outsider.

TCL ships with tclsh which is regular TCL (so a first-class programming language) but allows you to invoke external commands as though they were functions (so arguably is a shell in your sense, though I wouldn't advise using it as your login shell).

subprocess.run() isn't too bad.

> Is there any non-shell scripting language that makes it as easy as shell to run external commands, redirect I/O etc.?

Would Scsh count?

My guess is that it's because Perl is very comfortable for a team that mostly writes C on a Unix like platform.

The general syntax, operators, etc, are close, and familiar things like stat(), gethostbyname(), etc, work almost the same.

Edit: One of their Perl scripts for reference: https://github.com/openbsd/src/blob/b66614995ab119f75167daaa...

And pkg_add, which has quite a lot of Perl: https://github.com/openbsd/src/tree/master/usr.sbin/pkg_add

> - you need something under and acceptable licence, so python is out.

I'm not familiar with these types of things, can someone shed some light on this comment?

> awk would kind of work, except it's not that readable

I don't think I've ever heard someone say a plus side of perl is it's "readability". And this is coming from someone who really enjoys perl

Marc Espie elaborates a bit on this in another post on the openbsd-misc mailing list[0]:

> As for the license, python’s license appears fairly similar to Perl’s artistic license. I would worry a bit about the strong terms in

>> 6. This License Agreement will automatically terminate upon a material breach of its terms and conditions.

> for which no equivalent is visible in Perl’s license.

[0]: https://marc.info/?l=openbsd-misc&m=157781106212088&w=2

What the hell is this? Is this even legal? Aren't these completely different concepts?

Why wouldn't it be legal?

edit: why the downvote? It states explicitly that the license is only granted as long as its requirements (T&Cs) are upheld. Which seem obvious to me, assuming the license does have conditions?

Though I guess it's less obvious to the legal profession as the MPL has a similarly explicit clause (5.1, and 5.2 also terminates the license if you initiate patent infringement litigation against "against any entity", which seems oddly broad)

What exactly does that part mean? If you're in violation of the license, you cannot correct the violation and be compliant again, but the license will be revoked immediately and there's nothing you can do about it?

It's strange though, they also have

> 8. By copying, installing or otherwise using Python 3.8.3, Licensee agrees to be bound by the terms and conditions of this License Agreement.

which feels very 90ies-EULA-style. If I download ("copy") an archive that contains Python, I should be bound by that License Agreement? Since there's not severability clause, wouldn't that just void the limitations they've set?

I think in practice, if you corrected a violation you'd just be using the software under a different grant of the license.

Perl is what you want it to be. You can make it readable or unreadable. There's Perl code out there that looks like C and Perl code out there that looks like Brainfuck

In any case, it's being compared to Awk here.

>> Perl is what you want it to be. You can make it readable or unreadable. There's Perl code out there that looks like C and Perl code out there that looks like Brainfuck.

Or Latin:

https://metacpan.org/pod/distribution/Lingua-Romana-Perligat...

Or Klingon:

https://metacpan.org/pod/Lingua::tlhInganHol::yIghun

Or Poetry:

https://en.wikipedia.org/wiki/Black_Perl

Perl could be considered too flexible.

Presumably BSD distributions want BSD licensed software. Python has this thing: https://docs.python.org/3/license.html which is GPL compatible but might not be bsd compatible.

Perl is not BSD licensed. OpenBSD's worry is apparently section 6:

> 6. This License Agreement will automatically terminate upon a material breach of its terms and conditions.

which I'd have assumed implicit to pretty much every license — or at least every license with conditions — but I guess not if the PSF (or Mozilla which has a similar clause in the MPL) felt the need to specify it explicitly.

It might be related to the fact that both Python and Mozilla were, at one point, owned by actual companies. The wording might be specific to specific concepts of commercial law that are not particularly relevant elsewhere.

I wouldn’t assume so. Contracts often specify the opposite, that violation of one term does not void or negate the contract.

I don’t actually know, though. It just doesn’t seem like a safe assumption.

IANAL, but I have several in the family and have often been in a role to review contracts for professional services (I focus on the technical, legal focuses on the legal).

> I wouldn’t assume so. Contracts often specify the opposite, that violation of one term does not void or negate the contract.

I have not seen this language. Typically violating a condition of the contract means you are in breach. Sometimes remedies to certain breaches are enumerated in the contract.

The language about one item not voiding or negating the whole is often in regard to the legality of the contract, something like below (excerpted from an MSA from a client):

> If any term, condition, or provision in this Agreement is found to be invalid, unlawful, or unenforceable to any extent, the parties shall endeavor in good faith to agree to such amendments that will preserve, as far as possible, the intentions expressed in this Agreement. If the parties fail to agree on such an amendment, such invalid term, condition, or provision shall be severed from the remaining terms, conditions and provisions, which shall continue to be valid and enforceable to the fullest extent permitted by law.

This is basically saying if one term in the contract is voided, it does not void the whole contract. This is very different from "if you breach one term, the contract is not considered breached".

Practically speaking, there is no such thing as “BSD-incompatible license”. The only problem here is license proliferation.

Anything that significantly restricts use isn't something that you can distribute as part of a BSD licensed system. The restrictions on that piece would need to be met by anyone using or redistributing the system, which is a significant departure from expectations.

Certainly, you can use GPL software with a BSD licensed system, and you can distribute GPL packages to be installed on a BSD licensed system, but if you include GPL software in a BSD system, it's no longer a BSD system.

Indeed, but it’s not license compatibility - in Open Source world that problem can only happen with GPL.

How is that not license compatability?

If the license prevents otherwise useful software from being used in the base system, what would you call it?

The term „license incompatibility” is used in situations where you cannot use of a different license in a single program. Usual example is incompatibility between GPL and anything that’s not a subset of GPL.

The situation here is different: it’s about distribution, and it’s up to you, the distributor, to decide whether you want to include restrictively licensed software or not. With license incompatibility, you don’t get that choice: you just cannot legally distribute it.

There's the joke I like the best: Perl looks the same before and after encryption.

If you give a Perl interpreter a Rorschach test, you might be surprised: https://boingboing.net/2019/04/05/pollack-perl.html

People write unreadable code. Its not a function of language.

The kind of people who write unreadable code have long since moved on to write unreadable code in trendier languages and legacy critical code has long since been cleaned up if its of value.

Unreadable Perl is no longer an issue in 2020.

you need something under and acceptable licence, so python is out.

Can someone explain the difference here? OpenBSD is BSD licensed [1]. Python is a BSD-style, permissive free software license which is compatible with the GNU General Public License [2][3]. Licenses are always contentious, but how does this rule Python out? (Granted, the dynamic libraries technical point makes this kind of a moot question.)

[1] https://www.openbsd.org/policy.html

[2] https://docs.python.org/3/license.html

[3] https://en.wikipedia.org/wiki/Python_Software_Foundation_Lic...

It is interesting that PHP is ruled out as insecure. I am not sure that for system administration tasks modern PHP is particularly worse than Perl and these days more people know PHP than Perl.

Yeah, for scripting, PHP can easily be more readable, especially because of its amazing (sometimes confusing) Standard library.

It lets you parse INI files in a single call for eg.

As does Perl if you use Config::Inifiles or Config::INI::Reader (depending on whether you want a full blown solution or just a quick INI file => hash solution).

Yeah, quite weird. PHP as a webserver has a large amount of known language design things which decreased security (most have been removed over the years afaik), but for scripting? I can't think why it's particularly insecure compared to Python or Perl.

perhaps its built in web server? python has that too though.

No, it's not interesting at all. PHP is fairly good at web things, but that's it.

I use php for small scripts that need to read csv and database lookup from the command line. It has built ins for these that work without any environment setup which is great sometimes.

Perl has some easy oneliners too. Depending on where I'm working sometimes I don't want to set up an environment.

i dunno, i wrote PHP scripts for the CLI that run a $100m+ business and execute many millions of times per day. PHP is great for things not web-related, i think many just didnt take the time to learn.

Does PHP have anything approaching CPAN? That is one of the key reasons I still dust off Perl for quick and dirty things from time to time, as well as the compact and accessible regex syntax.

Yes: https://packagist.org/

What specifically do you think makes PHP especially great for that?

php.ini global configuration file is a pain that other script languages don't have.

A pretty amount of tools still use Perl under the hood and I'm pretty sure some of the "granted, they just work" tools that we're used to be just there or install, use Perl in some way.

Two examples I can give in my case are the good old rsnapshot, and CSF firewall

GNU Parallel is also a very useful tool, and it's written in Perl 5.

A useful tool with a highly annoying message to the user if you're using Fedora.

That's odd, but interesting. What warning?

Is it Fedora-specific?

It's specific to the tool, but most distros disable the message.

> If you want GNU Parallel to be maintained in the future, and not just wither away like so many other free software tools, you need to help finance the development.

etc

For more details, check out the Arch Linux patches that patch this super-annoying message out:

https://git.archlinux.org/svntogit/community.git/tree/trunk?...

The message appears every time when using GNU parallel unless a very long flag is given or the executable has been patched.

I think it goes against the philosophy of most open source software that comes with Linux distros, where you are given software freely, but pay back by contributing to the overall open source community, if you can and want. (This is my subjective interpretation of the matter).

I see. I mostly use Debian, and it seems that their "parallel" package removed the notice.

More info at: https://sources.debian.org/patches/parallel/20161222-1.1/

Was that the tool requesting to be cited if it was used anywhere near a publication?

I wanted to use it but also wanted to respect the wishes of the author, so, as I disagreed with the note, had to use something else.

Coz Perl rocks. Long live The Camel.

If you had used Postgresql you may be familiar with psql, createdb, createuser or pg_createcluster. All are perl scripts.

So if you remove Perl, either you renounce also to include Postgresql in your distro, or you would need to rewrite a non trivial amount of scripts in a different language to restore Postgresql support.

This post is about base system, which doesn’t include postgres. Obviously, if you don’t have Perl in the base, Postgres package could just include Perl as a dependency.

Then it is a different problem. Thanks for the clarification.

Is postgres in the base distribution for any BSD or Linux distribution? I'd assume it would just have a dependency on Perl.

createdb, createuser and pg_createcluster are not part of the PostgreSQL, but part of the postgresql-common packaging for Debian compatible distributions: https://salsa.debian.org/postgresql/postgresql-common/-/tree...

It's nice to read something positive about Perl again

OpenSSL/LibreSSL still requires Perl to compile.

Normally I do not have Perl installed, to conserve space.

NetBSD thankfully has always done without a large scripting language, only recently have they included Lua. I wonder how long it would take to rewrite the OpenSSL perl scripts in lua.

> recently

10 years ago

First 17 years without a scripting language.

How is TCL less secure than Perl? I've seen a lot less weird surprises in TCL code than Perl code.

* In my recollection, Tcl relies a lot on string interpolation and run time evaluation, both of which carry some risk.

* Perl has a built in taint checking facility which is quite good at preventing the naive use of non-vetted user data; I don't recall whether Tcl has anything similar. https://perldoc.perl.org/perlsec.html

Tcl is not less secure than Perl.

context: the start of the thread is about the base system

> I was wondering if I need the package manager in the minimal installation of the system as I only use built-in deamons (httpd, sshd) and UNIX utilities (vi, sed)? By package manager I mean pkg_* executables as well as its dependencies (most notably Perl). (The size of /usr/libdata/perl5 is about 50MB on my machine). I just want the minimal installation without any unnecessary scripting language.

> Looking at FreeBSD for a moment it seems like Perl left the base system in May 2002 (18 years ago)

https://marc.info/?l=openbsd-misc&m=159005284016345&w=2

I have always wondered what a scripting language designed with the values of the OpenBSD project would look like. It would interesting to sit down and think how pledge, veil, and privilege separation among other things would effect the design.

I have never programmed Lua, but it also seems to fit the bill fairly well.

- Acceptable license? MIT

- you need something that builds everywhere. It probably does? At least it compiles with Turbo C 1.0.

- Modicum of security: probably.

Lua is also small compared to Perl, is easily embeddable, and seems to have a sane syntax compared to Perl.

I do understand why they use Perl though, probably more OpenBSD developers/users are familiar with it.

I suspect they would need the lua-posix extension, which appears to be maintained by a fairly small number of people. Lua doesn't have some basic stuff they would want, like stat(), chmod(), sleep().

Edit: Sort of makes me wonder why a VERY C-like, small, dynamically typed, scripting language never emerged. Squirrel is kind of close, but not quite what I mean.

>> Sort of makes me wonder why a VERY C-like, small, dynamically typed, scripting language never emerged.

Csh (https://en.wikipedia.org/wiki/C_shell) was the de facto standard shell on many Unix systems and still is widely used.

However, it has some issues:

http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/

https://www.grymoire.com/Unix/CshTop10.txt

Standard library is another factor. Lua does not come with any batteries. Even sockets are an external dependencies. So you would need to ship (and maintain) a lot of libs.

That might me the direction FreeBSD follows. It started with integrating Lua into the bootloader, replacing Forth. There’s now also the userspace flua(1) interpreter, used in a few places in the build system, IIRC.

Generally- so far, flua only drives the generation of the various files we derive from syscalls.master.

There's been some WIP to also use it for some pkgbase stuff, as well as an ldconfig lua script for use in post-install scripts now that pkg supports post-install.lua.

I've had my eye on a couple of other components that could use a rewrite with flua. A new flua-based replacement for kernel config(8) could be quite excellent, for instance.

Why is TCL out? Is it about a perceived bad track record or about inherent properties of the language?

I'm also curious as to what the "security" issues are that TCL would be rendered "out".

I guess the general answer is that everything is out because they have a lot of Perl already and not a lot of developers so why waste that time rewriting everything?

Having said that I wrote a bunch of Tcl last year for a speech[1] and the Tcl community is nowadays very small. It was hard to find answers to some questions online: the active period of the community predated Stackoverflow so there's not much on SO. And the library support is very thin compared to the giant Perl/Python/Javascript/... communities. None of these things are reasons why Tcl is bad - I happen to like it and my talk was a success partly because of Tcl - but it may be a problem for OpenBSD to adopt it.

[1] https://www.youtube.com/watch?v=9E5A608xJG0

I meant specifically about security. Because everything is a string?

I doubt it's much different than Perl. Both languages have very loose type systems. In fact the internal implementations are rather similar since both use strings but optimize numbers (IVs etc in Perl which are visible to the programmer, in Tcl the bytecode interpreter optimizes ints/floats to an internal representation but at the language level everything still looks like a string).

Perl's tainting feature is the only security-related advantage over Tcl that I can think of right now, but it's not used unless you explicitly enable it.

No, that doesn't make it any less secure security-wise. It's probably because Tcl isn't an ALGOL type of language. I love Tcl by the way and I wish it was used more than it is.

Why not? Perl is an awesome tool for both sysadmin and web dev, two of the main tasks I use it for.

Why not?

It's a good question because OpenBSD (rather famously) chases simplicity.

Perl is antithetical to simplicity, often jokingly referred to as "write-only software". Not to mention the myriad of packages that usually end up being brought in (thanks CPAN!).

So, it's a fair question, not just "why is there a perl interpreter?", which is a decent question given openbsd's overall lack of shipping with much; but also "Why are large parts of the glue code written in perl?"

EDIT: I know it's frowned on to talk about downvotes, but I received quite a few and I'm not sure why. Please leave a comment if you're going to flag a comment.

Idk why people would downvote, but I don't agree that OpenBSD ships with little. In fact, I find it to be full of useful tools; it's very much a "batteries included, complete operating system" (as opposed to many small/minimalistic linux distros that really have you go through the pain of having to install every little thing you might need). It's small but not minimalistic. It's small but they managed to pack a ton of useful functionality in small size. And that's what I love about it; the rest of the world seems to be very fixated on extremes.

Agreed. It includes a huge set of tools needed by admins dealing with fundamental stuff such as routing and firewalling. I guess people bashing it are more into higher application layers.

Having pf, bgpd, ospfd, isakmpd etc. included in base for systems I administer are crucial for my daily tasks, while I couldn't care less for node.js. But application developers not dealing with network stuff probably have the opposite needs.

It's not about "which OS is better" but about "which tool is appropriate for the job".

So here's a theory about the downvotes.

Some person P likes technology X, but they are aware that others consider it a sport to dump on X. A rare article about X comes up in a popular forum. P clicks on it with dread to see how X is being dumped on this time. But it says positive things! P's dread is replaced with optimism. Then P reads the typical dumping-on-X post, which says something like "X is a hideous monstrosity. I fart in its general direction" or "X is antithetical to simplicity, often jokingly referred to as etc". P's dark mood returns and P does as people do when they feel like someone has been unnecessarily cruel and provoking: they press the shun button.

Alternative theory.

Perl aficionados have a long history of being on the receiving end of passive aggressive language zealots. Usually they are Pythonistas, but others have joined in as well.

You see this in messages that say "perl is a write once language" or "its too complex" or "line noise" or "why is it still alive when X is better" ... Really, its tiring after 25+ years using it. So, in an effort to express a valid and terse critique of the comment, the respondent does the right thing, and down votes the specific comment making those, frankly, inane points.

From there, people draw up interesting theories as to why the passive aggressive behavior was down voted.

The reality. Perl is a modern, highly capable, fast, easy to use language. It is comprehensible by mere mortals, while having powerful features that other languages often attempt to replicate poorly (regex, etc.) It is portable, running the same code across *nix, windows, macos, ... . It is fast[1] (yes really).

It doesn't claim to solve the worlds problems, but is a tremendously useful and portable tool, embedded in many companies products. Specifically, when they need to get something done, and done well, perl in the toolbox is a godsend.

Claims it is dead, hard, slow, incompatible, etc. tend to come from a vocal minority of people with an agenda.

I don't claim my theory is correct. I do believe it fits a very long standing pattern in language advocacy.

For me personally, I use the best tools for the job. Perl is often the right tool, but not always. Sometimes its python. Sometimes Julia. Sometimes C/C++. Rarely these days Fortran (moving to Julia).

This said, I am not going to bash any of the tools I use, or that others use. If visual basic .net works for you, use it. If you need to be portable across systems and OSes, you probably don't want to be writing things in powershell (literally had this conversation at $dayjob a few times over the last few months).

Languages come with dependencies and baggage. You can write great maintainable code in Perl, and terrible not comprehensible code in Perl. As you can in any language. Arguing that one language or the other "enables" you to do this "more easily" is a perfect example of that passive aggressive stance that is not helpful to anyone.

Use the right tool for the job. Perl, more often than not, is the right tool.

[1] https://scalability.org/2020/05/on-optimizing-scripting-lang...

> Alternative theory.

I'm pretty sure you just expressed the same theory in different words...

If you write a lot of systems code on Unix in C, write bash scripts, and write awk scripts then Perl largely just makes sense. It takes large chunks from all of the above. So if you don't like UNIX, C, (ba)sh or awk (all of which came from Murray Hill) chances are you won't like Perl either.

Maybe not so much C, but I saw Perl 4/5 as very attractive for sysadmin-type engineers, because it was pretty much explicitly designed to make onboarding as easy as possible for awk, sh, sed (and BASIC) programmers.

Quite a few people I know got started with Perl because they were reasonably comfortable with awk, sh, or sed, started to outgrow their sweet spot, and found that perl provided a very natural transition from each of these languages, could do everything those scripts could do in fairly similar ways, and was obviously more powerful.

Perl5 is pledged.

No it's not. It has language bindings for pledge(2), which is different. There can be no one size fits all pledge for an interpreted language like perl, as programs can make arbitrary calls to anything they want. But those programs can use pledge, as long as they consider the requirements of the runtime itself.

http://man.openbsd.org/man3p/OpenBSD::Pledge.3p

That's what I wanted to mean, sorry. OFC an interpreter itself can't be pledged.

Saying something is pledged has a very specific meaning. And in this context, has no relevance to the discussion as none of the perl components in base make use of these bindings.

There are many who call GNU people license zealots, and who imagine that GNU somehow shuns BSD licences. But the opposite is true! GNU people happily use BSD licensed code, and include BSD licenced code in the GNU “free software” term, while it is the BSD people who, as can be seen here, reject software like Python and GCC for having “unacceptable” licenses.

GNU/GPL folks are zealots about user freedoms, not license. Linux leadership (accepting GPL) are less zealotic about user freedoms, but sometimes are zealots about keeping control (rejecting to play nice with the OpenZFS project). BSD people are zealots about getting rid of GPL code so that their product can be used by some stranger in a proprietary way. There is money both in working on GPL software and more liberal licenced software (GPL variants), but it seems everything is set up so that the two camps must not directly cooperate.

You can always combine BSD code with your GPL and release the code as GPL.

You cannot however mix GPL code into your BSD code and the resulting work as BSD.

>You can always combine BSD code with your GPL and release the code as GPL

That is completely wrong, you cannot change the license at your will ESPECIALLY from BSD to a GPL. The BSD-Code stays BSD-licensed-code, you can include it into a GPL Project, but NOT change the License, doesn't matter if its from BSD to GPL or vice versa.

Since your not the owner of the code, you have absolutely NO right to change the license.

As the sibling mentions, you cannot re-license code you do not hold copyright to.

That said, you're very close to the mark. GPL applies to distributions of the copyrighted work and executables based on it. So, the concern is around an OpenBSD release - they are shipping a full OS with source code and compiled work product. The GPL is "viral" in that you cannot make this release without licensing your whole distribution as GPL. OpenBSD does not want to be GPL, and so does not include GPL-ed code in base.

The inverse is not true. A Linux distribution can include GPL and BSD-licensed code easily, because the BSD license imposes no constraints on redistribution.

Of course you cannot relicense someone elses git repo.

Copying BSD code and distributing it under another license (even a commercial one) seems to be totally OK though as long as you put in the correct notices.

That is what GPL is meant to protect against.

Right, but the GPL license's protection also means that a BSD cannot be distributed under the BSD license (or similar) if it includes GPL components. This is why OpenBSD excludes GPL components.

GPL enhances user freedom at the expense of restricting what programmers can do.

> Right, but the GPL license's protection also means that a BSD cannot be distributed under the BSD license (or similar) if it includes GPL components.

I fail to see how this is different from what I wrote above (except for the missing word in my post above):

> You cannot however mix GPL code into your BSD code and [distribute] the resulting work as BSD.

I didn't pay enough attention to usernames as I was replying and thought I was talking to multiple people through the chain.

We are in verbose (and unnecessary on my part) agreement.

The “correct notice” being the original BSD license.

The point of using BSD license is to get rid of restrictions. Using GPL brings restrictions, same way using closed source components would. It’s as simple as that.

https://dev.perl.org/licenses/ is not BSD.

So, your point doesn't hold up even for this case.

I've never written a single line of but written python, bash and c and go.

I am not sure how important perl is.

So why write this comment? What is your comment adding to the discussion?

Knowing when to shut up is a valuable skill to have.

I would strongly recommend you do learn Perl 5 (and Raku, and Haskell, and lua, and functional languages, and ...), even if only to be able to know enough about them to be enlightened about what each of them gets right and wrong.

For Perl 5, you could do worse than going through the _free_ "Modern Perl" book, at http://modernperlbooks.com/books/modern_perl_2016/

Thanks for recommending this book. Mostly I write TypeScript day-to-day, so naturally I typically reach for nodejs when I want to "upgrade" a script from bash to a more expressive language in a cross-platform way. I've been frustrated with that tooling, however, for a whole bunch of reasons, at lot of it to do with how clunky nodejs actually is in shell pipelines. I've thought about reaching for python, but it has some similar issues to nodejs: you need a particular environment, and some things which ought to be natural in a shell environment just...aren't...in python. I think someone here suggested Lua, and I just...don't care for Lua (although I haven't tried it in a scripting-like context).

Anyhow, I've been reading this for a half-hour or so and I'm struck by how incredibly similar common Perl idioms feel to JavaScript as it is commonly used today. I think part of this familiarity has to do partly with type coercion, partly with destructing assignments, partly with function "context"(as in Perl) as it is commonly deployed in the nodejs ecosystem, partly because of the way default arguments and function passing relates to the default scalar variable in Perl.

I've been playing around with OCaml and Rust a bunch recently, but haven't had any good reason to reach for them "in real life." The fact that Perl is "everywhere" (much the way JavaScript is "everywhere" as a side effect of browsers being ubiquitous), means that I'm likely to start reaching for Perl instead of nodejs when I need to move beyond Bash. It really does seem to be the right tool for the job.

So, a hearty "thank you" for the recommendation.

Glad to hear you opted for adding another tool to your toolbox :)

You might find it amusing that "JavaScript: The Good Parts" directly inspired the "Modern Perl" book.

That is entertaining to know!

Thank you for writing this, it seems very well done so far. And thanks for also making it available in the way that you did: I almost certainly wouldn't have played with Perl today without your book, made effortlessly available on my e-reader via Pragmatic. I'm curious why it's not available in physical copy from Pragmatic? And I'm not confident about the correct "buy" link to use to properly support the book.

One sidenote that I hate to mention, but: I couldn't figure out how to open a PR for this or even email someone who would care, but I found a code-typo on the CPAN website: it's instruction to "install via App::cpanminus" and then use with "cpanm name::module" is...an unfortunate early typo on that site.

I'm curious why it's not available in physical copy from Pragmatic?

I believe it is:

https://pragprog.com/cart/add/skus?sku_id=821_820

I'm not confident about the correct "buy" link to use to properly support the book.

It's more valuable to me if you share and enjoy. I give away electronic versions so that it's useful, not to make money from the time I spent writing it.

I found a code-typo on the CPAN website

Is this the page you were reading?

https://cpan.metacpan.org/modules/INSTALL.html

It's not explained on that page, but the distribution App::cpanminus installs a program called cpanm, not cpanminus (as one would expect).

Huh. Yes, that's what I was talking about. But `cpanm` didn't end up anywhere in my path, and at the same time, `cpan Module::Name` successfully downloaded and installed modules that I could use.

Reply on news.ycombinator.com