Hacker News Reader

Comment by Kelteseth | Hacker News Reader

Honest question: Is anybody using perl for new projects? We have a medium (~100k) codebase written in perl and are actively working to convert it to python.

View on HN

Yes.

At my $work, most of our code is in Perl, roughly a half-million lines of it dating back to the mid-1990s.

There's no point in rewriting it in something else, since the cost in resources to do that would outweigh the potential salary savings by hiring cheaper programmers who know Python or C# or whatever. (My experience with some other companies is that they want to move away from Perl because Perl programmers are more expensive, not because it's a worse language.)

Likewise, there's no point in starting new projects in another language since we'd then have to hire people people competent in both of those languages.

My experience is that people who turn their nose up at "old" tech like Perl are not worth hiring. If they don't care to learn about the technology you are using, then they won't care to learn about your industry or your company's business logic.

I worked at a place like this and it was a nightmare. We had a few dedicated Perl programmers who swore there was nothing better since sliced bread, but there were so many legacy Perl programs it spilled into everyone's responsibilities. I know for a fact everybody else absolutely hated having to go in and decipher decades old Perl code.

You could replace Perl with any language and people would have the same experience.

Most other "popular" languages share a pretty good chunk of their syntax. You can dig through PHP/Java/C#/Go/Javascript/C++/Python pretty easily if you already know one of them. Perl and its magic variables are absolutely undecipherable if you haven't worked with the language before (or in a long time).

When I learned Perl, the syntax reminded me of C and bash. The magic variables are also from bash.

Maybe because I knew C and some bash and awk, Perl wasn't alien to me at all.

Coming from a Pascal and Java background, it was a breath of fresh air. Programming was actually fun.

>>Perl and its magic variables are absolutely undecipherable if you haven't worked with the language before (or in a long time).

Yeah, either the language gives you magic variables, or you make your own one's in the programs you write. Programming isn't sorcery. Somebody has to write the code for features a language doesn't give you. That the is whole problem in essence, the more minimal your syntax goes, the more code you will write for features the language doesn't give you.

Good part about the Perl magic variables is at least they are documented.

perldoc perlvar will tell you about all the magic variables.

That's a problem with the codebase at your place of work, not the language.

> I know for a fact everybody else absolutely hated having to go in and decipher decades old Perl code.

There's genuinely nothing I enjoy more at work than deciphering, debugging, documenting, and ultimately refactoring Perl code. Even if I'm improving it to extinction e.g. turning parts of a legacy stack into a service with an API, ready for a replacement in some other language to be swapped in.

> people who turn their nose up at "old" tech like Perl

The problem with Perl isn't that it's old. It's bad in ways that other old languages aren't. Consider all the people who like C but not Perl, even though C is older.

Please describe those uniquely bad aspects of Perl that other scripting languages aren't.

Aren't you missing advances in programming language technology (performance, tooling, instrumentation) by sticking with such an old interpreter?

Perl gets new releases on a yearly cadence. We're now at 5.36 IIRC. A far cry from 5.8 or even 5.10.

The new releases usually come with bug fixes, speed-ups, additions to the core language, optional syntax features, etc.

The only way one gets to use an "old" interpreter is if they're stuck on an old OS release (i.e. RHEL?) and/or they don't build their own version and use the "system" Perl.

>stuck on an old OS release (i.e. RHEL?) and/or they don't build their own version and use the "system" Perl.

I build my containers and put Perl right in them. Makes the container a little chubbier but no outside dependencies.

There's quite a bit of legacy running in our industry. Most of it is not even Perl. Think Java, Jquery, and even Python(2.x) these days.

Most shops don't have the budgets for total rewrites, or even partial and gradual rewrites.

The way old things die, is that new projects don't get started in those languages. To that end, Perl is something of the past already. It is still a very useful technology for a lot of Unixy work on adhoc tasks. But I doubt people learn Perl the way, they learn Python these days. Older people(including me), call upon Perl for lots of work even till date. This is similar to how people continue to use vim/emacs despite things like vscode. There is always going to be a audience for these tools. But the crowd moves on.

No.

The Perl interpreter has been regular updated and improved. There's a new release of Perl every six months, and the libraries on CPAN provide access to various other kinds of technology and tooling.

Doesn't even have a basic JIT does it? Still using a 90s-era basic interpreter and GC architecture.

And there's only one implementation. Doesn't seem very healthy.

Python (or at least, the implementation used 99% of the time in the real world) doesn’t have a JIT either, so this property isn’t limited to niche, dying languages.

Well yeah compare to Python - Python has multiple alternative implementations, an active academic and industrial research community coming up with new ways to improve performance and add tooling, including revolutionary work such as meta-tracing, has new GC work, has a couple of JITs depending on what tradeoffs you want, new explorations into specialising interpretation.

Perl doesn't seem to have anyone making this kind of effort? You're stuck with the tech from decades ago.

The implementation is open source and not owned by a for-profit company that could decide to kill it off in the future.

This is a good thing, because it means that we don't have to worry about a program running on one implementation vs another.

Also, one of the things that Perl is great for (unlike many other scripting languages) is backwards compatibility. A program written in the 1990s for Perl 5.004 will probably run with minimal modifications on Perl v5.36.

> The implementation is open source and not owned by a for-profit company that could decide to kill it off in the future.

That's the same for almost all languages though? Which major languages aren't open source or are owned by a company?

> Doesn't even have a basic JIT does it? Still using a 90s-era basic interpreter and GC architecture.

So?

> And there's only one implementation. Doesn't seem very healthy.

LOL, okay, this is just hunting for complaints. I could list a half dozen languages that don't have multiple implementations (Rust, Java, Scala, C#, PHP, Go) and yet I've never seen anyone raise that as a concern.

> I could list a half dozen languages that don't have multiple implementations (Rust, Java, Scala, C#, PHP, Go) and yet I've never seen anyone raise that as a concern.

That's because you're misinformed! These languages all have multiple implementations (Scala's is just multiple backends), so yeah people don't complain about it.

Yeah, no they don't. Show me another implementation of any of those languages and I'll show you a toy no one actually uses.

For example independent Java implementations used in production include OpenJDK, OpenJ9, and SVM.

And the vast majority of people use OpenJDK.

But hey, why not, I'll give you that one (tbh, I knew including Java on that list was a bit of a stretch).

Care to try with any of the others?

Wanna try to convince me, next, that HHVM qualifies as a reimplementation of PHP despite no one outside of Facebook using it?

But you know what, no, I have a different question: Why does this actually matter? You've positioned this as some sort of objective downside to Perl without explaining why anyone should care. So why don't we start there, rather than my taking your objection as being somehow relevant.

Please, explain to me: Why should I care that Perl (like so many other languages) only has one real implementation?

> Why does this actually matter?

Because it fosters innovation. Perl isn't innovating as an implementation. Why's that bad? Because you don't get better performance or capabilities, and you're spending more to run it, and spending more to manage it.

> Care to try with any of the others?

For example Mono is a full clean-room implementation of .NET including C#.

> tbh, I knew including Java on that list was a bit of a stretch

Sure. Sure.

This is partially wrong. Perl compiles the program to its own internal representation in one go, which is then run in a VM. It also uses reference counting, it does not have a GC.

It is true it doesn't have a JIT, though.

> This is partially wrong.

Which bit was wrong? It has a basic bytecode VM - isn't that the 90s era interpreter tech I was talking about?

> It also uses reference counting, it does not have a GC.

Reference counting is GC.

Same with python. GIL and everything.

It's not - Python has innovation going on into new techniques for interpretation and GC. Perl doesn't have any activity like that at all.

I'm working at an all-Perl place, and I'm about to move to another all-Perl place.

Neither has any plans to move off it. The opposite, actually.

Like them, there's likely dozens of others. Dozens, I say!

Where in the world is this? I would actually fancy working with Perl, but here in Norway there's nothing to find. :(

Look on https://perl.careers or LinkedIn or https://jobs.perl.org for remote perl development gigs, and you'll find one sooner or later, assuming you qualify ;-)

I’m hiring, email in my profile :)

I started a new project in Perl, which I've commented on before a couple of times in response to a similar question.

One of the main reasons I chose Perl is it's commitment to not introducing breaking changes.

https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...

https://news.ycombinator.com/item?id=31261071

A better question is: is anyone planning on moving from Perl to Raku, or from Perl to another language?

At $work, we're sticking with Perl [reasons elsewhere in this thread]. But if we were going start projects in other languages, it would probably be Python or Node.

For personal projects, I'm more interested in Rust than Raku.

Python is fairly stable, node is prone to fast change.

That is, if python isn't using external dependencies.

If you are writing an app for decades long usage, node is probably a bad choice.

Perl, python, will be around... well, I dunno, decades more at least.

But perl? As someone else said, the code you write now, will work 30 years from now. That's its thing, its happy smile.

Imagine, just imagine writing code that doesn't vanish 2 months after you leave a place. Imagine code your great grandkids might tough.

You're doing God's work. Last I used Perl was ten years ago. I decided to give it another shot for small scripting needs. The ability to write one-liners to do stuff is exciting at first, but my brain isn't used to a language with such a complex grammar. Just the number of ways you can write loops is crazy. Oh btw, is there any other language for which writing BNF isn't impossible? [1]

It doesn't help that most of the times, Perl fails with a very confusing error message that can leave you scratching your head. And it seems all the Perl wisdom has gradually vanished with very few answers to your questions. It wasn't a bad language for its time, but with the number of alternatives, I just don't see a valid reason to use it.

[1]: https://stackoverflow.com/questions/4625408/where-can-i-find...

I have a side project at work that I'm doing on evenings/weekends that I've written in perl. One part of it parses verilog source code and builds an AST. It is a recursive descent parser that was written in the most straight-forward way and no time has been spent optimizing it for speed. The AST builder and some auxiliary routines that make interfacing to the AST simpler to manipulate is less than 2500 lines of code and took a couple weekends to write and get working.

Surprisingly, I timed the parser part and it took only 0.06 seconds to process a 2500 line file. Yes, perl is 30x slower than C, but computers are so fast that performance (for my needs) doesn't matter too much.

To be fair, it is only the subset of verilog that one would typically synthesize (so no events and forks/joins, no timing annotation, etc). Doing the whole language would make the code larger but not much slower.

I have been using Perl (5) for new projects until very recently (and, realistically, I might still use it in the future). I also did some contracting until very recently for a company whose product makes extensive use of Perl. I've been writing Perl professionally for over 10 years.

The backend of my network programming challenge[0] is all Perl daemons, including the web API, the solution checker, leaderboard calculation, and new problem announcement mailing. (The frontend is Next.js, but it's not using any server-side JavaScript, it's delivered as a static site).

Perl is good because it is very quick to write. Common idioms are very terse, first-class support for regexes is a superpower, and the TIMTOWTDI philosophy ("there is more than one way to do it" - polar opposite to Python) means you can often find a quick and succinct way to make most small code changes you want to make.

I am personally looking to move away from Perl for new projects for the following reasons:

1. Perl doesn't have sensible support for multi-threading.

There's "use threads", which is quite heavyweight because (as I understand it) every time you make a new thread with "use threads" it creates a memcpy() of the entire interpreter.

There's "use forks", which is the same as "use threads" except it uses fork() so that the "copy" is less expensive (because Linux gives us copy-on-write). "use forks" is still bad because inter-thread communication still involves serialising your data and passing it over a pipe, and you don't necessarily notice where you're doing it.

Then there's Coro[1], which is actually very very good, for what it is. Coro allows you to write code in a straight-line synchronous style, but allows you to cede control to other threads when you need to wait for a result to become available. Coro calls itself "the only read threads in Perl", but it's still not real threads in Perl because it still doesn't let you take advantage of more than 1 CPU. In my experience Coro also interacts poorly with XS (i.e. native code) modules that aren't reentrant, and it's difficult to predict when you might be using such a module, and sometimes your Perl scripts will segfault or leak memory through no obvious fault of your own. But Coro is a really neat hack.

If you actually want to make use of multiple CPUs, the best option (IMO) is to fork() manually and keep inter-process communication to a minimum.

2. Further than multi-threading, Perl doesn't even have sensible support for async/await style concurrency.

The closest thing to async/await style concurrency is AnyEvent[2]. AnyEvent is quite good, and I think is the best way to write asynchronous code in Perl. But it still doesn't hold a candle to basically any modern programming language, because you still have the "callback hell" style of passing closures as an argument to anything that has to be asynchronous. This is extremely annoying and results in enormous, complicated, and error-prone refactors if you ever have to change a component deep in your stack from synchronous to asynchronous. For example, if you are moving away from a hash table "database" to an actual RDBMS, and you don't want your database queries blocking your entire application, this is extremely painful, because everything that lives higher up the stack needs to be rewritten in "callback hell" style. See "What color is your function?" for more on this.[3]

3. Lots of things that should be caught statically don't actually cause problems until runtime.

This is mainly because Perl is dynamically typed and isn't compiled, but it is annoying, and avoidable in other languages.

What language am I moving to? I don't know yet. Probably Go. I don't like how opinionated Go is about a lot of things that Perl doesn't give a shit about, but it is at least quite small.

I don't like Rust because it seems overly bloated (the recommended way to parse command-line arguments - "clap" - involves downloading 140 megabytes of dependencies off crates.io, which just seems unreasonable), although the borrow checker is a very interesting idea.

I'm also interested in Zig, but I think the Zig community is possibly too small at the moment, and not being fully memory-safe seems overly risky. But I haven't written enough Zig yet to know whether or not I like it.

[0] https://protohackers.com/

[1] https://metacpan.org/pod/Coro

[2] https://metacpan.org/pod/AnyEvent

[3] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

> I don't like Rust because it seems overly bloated (the recommended way to parse command-line arguments - "clap" - involves downloading 400 megabytes of dependencies off crates.io, which just seems unreasonable), although the borrow checker is a very interesting idea.

Where do you get that from? https://lib.rs/crates/clap states that the code itself is 1MB, and the maximal amount of dependency code is 7mb, with all optional features enabled.

`du -h` of a project that did nothing but use clap was 380 megabytes or there abouts.

EDIT: A blank "cargo new" project plus a clap dependency weighs 143 megabytes, not 400. Maybe the 400 megabyte project had another dependency as well, I can't remember. I've edited the comment to say 140 instead of 400.

    $ cargo build
        Updating crates.io index
      Downloaded clap v3.2.23
      Downloaded textwrap v0.16.0
      Downloaded proc-macro2 v1.0.47
      Downloaded clap_lex v0.2.4
      Downloaded hashbrown v0.12.3
      Downloaded clap_derive v3.2.18
      Downloaded once_cell v1.16.0
      Downloaded libc v0.2.137
      Downloaded os_str_bytes v6.3.1
      Downloaded unicode-ident v1.0.5
      Downloaded syn v1.0.103
      Downloaded 11 crates (1.4 MB) in 3.71s
       Compiling proc-macro2 v1.0.47
       Compiling version_check v0.9.4
       Compiling unicode-ident v1.0.5
       Compiling quote v1.0.21
       Compiling syn v1.0.103
       Compiling autocfg v1.1.0
       Compiling libc v0.2.137
       Compiling os_str_bytes v6.3.1
       Compiling heck v0.4.0
       Compiling hashbrown v0.12.3
       Compiling textwrap v0.16.0
       Compiling strsim v0.10.0
       Compiling termcolor v1.1.3
       Compiling once_cell v1.16.0
       Compiling bitflags v1.3.2
       Compiling clap_lex v0.2.4
       Compiling proc-macro-error-attr v1.0.4
       Compiling proc-macro-error v1.0.4
       Compiling indexmap v1.9.1
       Compiling atty v0.2.14
       Compiling clap_derive v3.2.18
       Compiling clap v3.2.23
       Compiling clapdemo v0.1.0 (/home/jes/clapdemo)
        Finished dev [unoptimized + debuginfo] target(s) in 36.61s
    $ du -h | tail -n1
    143M

> Finished dev [unoptimized + debuginfo] target(s) in 36.61s

As anyone who is used to languages like C or C++ knows, debuginfo tends to be huge. It's not surprising that your "target" directory (where both the intermediate and final objects are stored) is that big.

> Downloaded 11 crates (1.4 MB) in 3.71s

Here, you see that 11 of your dependencies together (including clap) were a download of only 1.4 megabytes.

And you won't see these 1.4 megabytes in your "du" output, since the downloaded crates are stored in ~/.cargo instead of within your project directory.

> It's not surprising that your "target" directory (where both the intermediate and final objects are stored) is that big.

Actually this is surprising to me. All it does is parse command-line arguments. I am surprised that this creates 143 million bytes of information.

I accept that I was wrong to say this data is "downloaded" off crates.io, since it is in fact created locally.

> 11 of your dependencies together (including clap)

This is obviously a matter of taste, but to me, this is too many. In the event that sensible command line parsing is not part of the core language, it could at least only be a single dependency.

There are several reasons:

1. Unoptimized builds tend to be excessively large. LLVM's opt binary (citing this because it's something I have easy access to) is 30x larger on an unoptimized build than an optimized build. [I think Rust is even worse than C++ at the unoptimized/optimized size ratio]

2. Debugging information is also excessively large. It's not counted in the above statistics I gave because I use split debug information (which keeps it out of the final executable for easier linking), but you're looking at debug information being a significant fraction of code size--maybe half of the total binary size could be debug information.

3. Because you include the build directory, you're carrying two copies of everything, minimum--both the .o files and the executable they link into.

For a rough comparison of LLVM, the llvm source directory I have is 1.5G, the optimized release build I have is 15G (of which 1.7G is the actual binary executables), and the unoptimized debug build I have is 86G. Of course LLVM is unusual in producing a very large number of statically linked executables.

4. One of the dependencies inside clap is the procedural macros stuff (specifically, quote/syn crates), which basically requires building your own copy of the Rust parser. If you drop that dependency, the build directory sizes should be far, far smaller.

Doing it with 'clap = "4.0"' gives me 46MB. Still surprisingly large.

I have:

    clap = { version = "3.2", features = ["derive"] }

  $ du -h | tail -n1
  =>
  $ du -sh

> 2. Further than multi-threading, Perl doesn't even have sensible support for async/await style concurrency.

I've never thought so. With Coro's rouse_cb/rouse_wait, you can use almost every callback-style methods in async/await manner.

I'm using perl as glue tool for a lot of my projects. I could write it in python, but then it would turn into a big hairy monster, and I rather want to keep my tools as tools.

Reply on news.ycombinator.com