Dwindling CPAN Releases
perlancar.wordpress.com
perlancar.wordpress.com
1) Most long time libraries have pretty much reached stability and the (still relatively high) backwards compatibility promise of perl5 means they don't have to be updated to support changes in the language, which still has releases every year.
2) The quality of uploads to CPAN is of the form of solving more complex problems or interfacing with core Unix/network/internet/lower level lib stuff stuff - you don't see so much of the leftpad type modules on CPAN.
3) It definitely feels like the number of authors has shrunk significantly, even though the graph suggests that has stayed relatively level. I suspect there's a long tail of active authors and most signups never actually upload anything
4) The biggest problem now is finding modules for newer APIs - I wrote some, including for more well known services such as GoCardless and Monzo
5) Holy crap, the graph covers almost 24 years of uploads. You will probably see the same pattern in Python, JS, Go, and so on if you come back in an equivalent number of years - the most common problems get solved first, and libraries uploaded for those.
6) The graph covers uploads to CPAN, I'd guess a far more important one would be downloads from CPAN as that will probably give you more information about the relative level of use of the language.
That being said, I'm a professional dev and we own a decent amount of perl. That codebase is by far the most difficult to work in out of anything we own. New hires have trouble with it (nobody learns perl these days). Lots of it is next to unreadable.
I agree that like any other language, perl can be written well and can be written poorly. That being said, perl makes it _really_ easy to write something poorly, in contrast to something like python which is a bit more rigid and tries to have one right way to do things.
I still use perl pretty extensively for little one liners, but at this point we are actively trying to get rid of as much perl that we own as possible. We find that rarely writing anything longer than 10 lines should be done in perl, if only from a readability and maintainability standpoint.
I love perl. It was THE game changer for many many years. But perl is falling out of fashion quick.
I'd have no problem working in the language with some very rigid guidelines and a bunch of experts.
But the reality is that nobody is hiring perl experts, perl experts are becoming more and more scarce, and most new hires have a bit of experience with py or rb. I dont dislike perl, I'd just argue it's almost always the wrong thing tool for the job these days.
In a previous life I worked at a large E commerce company. You've likely ordered from them. Pretty much everything ran on perl/perl Mason for over a decade. At that time perl was the cool new thing that everyone wanted to pick up. Now (or when I left a few years back), most of that code represented a liability if it needed to be changed.
IMO this is just the cycle of a programming language.
I remember reading Spolsky articles where he talks about VB6 like it's some shiny new amazing scripting language, akin to python 10 or so years ago. But alas, time moves on and so do most programmers.
I've always tried to look at programming in a similar manner as writing research papers, your ideas are no good if no one understands how they're put to paper. Explicitly stating your intentions is better than assuming the reader knows the subject matter. If something is clever, overly-compact or your reviewer expresses confusion, rewrite it or at least spend an extra line or two in explanation.
At the language level, I miss Perl's handling of regexps and shell commands (which consist of large amounts of this codebase). Python requires a fair amount of overhead to handle both and always feels a little clunky. While I'm far more fluent in Python, I typically reach for Perl when needing to quickly handle log files, automate shell tasks, and prototype ideas. On the flip side, now that Python supports type hinting, function signatures are much more valuable, and overall I feel I can write far more robust tests in Python than Perl. Given that the direction of the project is to move from a collection of one-off scripts that just kept on growing to a tested, maintainable, modular codebase, Python is the better choice.
I would totally agree.
I learned Perl after first learning Basic(and Visual Basic) then Java.
Coming from Java meant that I always tried to use clear variable names, good comments, broke my code into small and simple functions etc. I repeatedly had people say to me "Wow! I didn't know you could do that in Perl!"
A lot of Perl's bad reputation came from sysadmins and other busy folks who either weren't familiar with good syntax or were in a hurry and basically were playing Perl Golf to get things done quickly. This is probably where the line "Perl is a write only language" came from.
I will concede that Python and it's forced indenting leads to much easier to read code for beginners. That being said, trying to understand what's going on with advanced Pandas usage written by Numpy wizards is no walk in the park either.
I also totally agree that the Perl one liner is still my go to even though I do everything else in Python these days.
The biggest problem I have with Perl is like C++, the language is so stupidly big that every single programmer knows a different subset of the language.
I used to bump into this problem when I was interviewing for VLSI jobs and somebody would always do the "Let's talk about Perl" round. I was quite fluent at Perl at the time, and there would ALWAYS be some idiom that would stump me.
I solved this by bringing my own Perl program on a printout with me--I also brought the Python version of the same program with me. Suddenly, the interviewer was in MY subset of Perl and almost always went "Oh, you're a lot better at Perl and programming than I am" instead of dinging me for not knowing weird Perl trivia.
Recent Perl versions have function signatures.
Have you tried the Test2 test suite? It’s a tremendous improvement over the previous system.
Type::Tiny is an amazing library that provides a rather nice type system.
rg '<filter pattern>' <logfile> | \
perl -MData::Dumper -n -e '
m/\d{2}:\d{2}:([\d.]+)/;
if ($last) {
$diff = $1 - $last;
print "$diff\n";
$hist{$diff} += 1;
}
$last = $1;
END {
$Data::Dumper::Sortkeys = true;
print Dumper(\%hist);
}
'So is Duke Nukem Forever.
It's pretty sad too because I distinctly remember that one of the main selling points of Perl when I started using it in the early 2000's was that CPAN was more complete and diverse than any other scripting language out there.
I've been searching for alternatives for about 10 years now, at first because my novice self was heaving headaches reading other people code and even my own. I hated perl but I hated everything else even more as I came to know them. No autovivification? Some assignments copy the data and some create references and you need to memorize the conventions of which does what? 3x to 10x lower performance? My program crashes because of white space? Fuck that! Back to perl5.
I'm sorry to say nothing can replace perl5 for me even now in 2019. I wish something would come along even if it takes years. We need alternatives and competition, perl5 will get too old at some point and 6 is... I don't even want to go there.
Wow, thanks. I had been looking for this word to explain what happens in awk. Now I know what it's called!
Just the syntax sugar for them is more recent.
$hash{1,2,3} = 10;
This actually ends up doing the following $hash{join($;, 1, 2, 3)} = 10;
It builds up a single key from the list using the special variable $; as a separator to the elements. With perl 5 you gained the ability to make references which allow you to actually make multiple dimensions and to do it on arrays, not just hashes. $array[1][2][3] = 10;
That also allows for more complicated data structures and much easier ways to iterate through the structures.This is a nightmare for beginner coders trying to make sense of Pandas or R subsetting.
I had heard a few perl monks criticize it a bit, as an "end run around use strict", but I was kind of dismissive. It turns out, that's a really apt way to describe it. It basically tosses the niceties of use strict out the window, so long as its a hash. And sooo many things are hashes/hashrefs in Perl.
The first time I wasted a day tracking down a bug that turned out to be a well hidden typo in a hash key assignment, changed my mind. Well... maybe the third time.
$foo = $bar{Autovivication}
instead of $foo = $bar{Autovivification}
in your program, it will abort instead of returning an undefined value :)See Hash::Util for details.
I've just written a utility I needed in Perl. I tend to think about some things in the long term, and this utility is something I want to be able to run 10 years from now.
[it's a concistency checker (hasher) for file archives, to find out if the bits in your copy are really the ones you put there, important for multi-GB photo archives — see https://github.com/jwr/ccheck]
I do most of my programming in Clojure, which has excellent backwards compatibility and takes a long-term perspective. I still run some unmodified Clojure code from 8 years ago. And yet for really long term stuff, I would much rather write it in Perl. It has been around for a very long time, is very mature, and can be expected to last, even if it isn't the latest fashionable trend.
Take a look at the various programming languages+environments around you: there will be very few where you can expect to be able to run your code even 2-3 years from now, much less decades later.
As a comparison, I peeked at some Delphi/pascal code I'd written and a book a few weeks back and it was still parseable by my brain years later.
More is on the way. Github issue thumbs up-driven development means it's the new C++ kitchen sink.
Not sure I agree with that. Languages which share widely used conventions and syntax can be picked back up much more quickly.
I've been coding in Perl for 20 years, I'm a CPAN author [1] and I just published my third CPAN contribution yesterday.
I've written over half a million Perl lines of currently in use enterprise cloud software[2] and I'll keep coding in Perl because I don't like the big snake dictatorship of "There should be one — and preferably only one — obvious way to do it."
[1] https://metacpan.org/author/ERUCI
[2] https://aws.amazon.com/marketplace/seller-profile?id=434e986...
That is a python design principle https://en.wikipedia.org/wiki/Zen_of_Python
Now there are multiple (sometimes very confusing) ways to import stuff in python, so I am not quite sure if this principle is being followed.
There are many different ways to package Python, none of them are feature complete, and the result is an absolute maintenance nightmare for those of us who have to maintain tools that have Python dependencies.
Perl got packaging right, there's one way to do it, and it just works!
This is a huge barrier to entry for the first time or casual contributor. And when you try to figure it out (or figure it out again), you have to triage all the old and new overlapping documentation.
In practice, perl came closer to achieving this than python.
1) Works
2) Is ubiquitous
3) Is stable
4) Is normal (semicolons, braces etc)
On which systems do you have Perl readily available but not Python?
Perl documentation online applys to versions of perl that are installed. Python stuff is all bleeding edge - you'll research why something isn't working and after 30 minutes find it's expecting python 3.7.
The whole concept of "virtualenv" gives me an impression of fragility I don't get with perl.
How ancient is that system?
No matter, that system is going to be decommissioned and functions will go over to a new EL7 install.
It just stalls that project till the new server is ready.
This is not how you deploy software
Perl 5 is Perl 5 everywhere.
Where I work, a few of our legacy systems are still dependent on ~v5.10, which can be a real pain when trying to port code back to legacy. I agree that the Python versioning problem is perhaps more prominent, but Perl 5 doesn't really get a free pass here.
Because of the stability of perl5, I barely care what version of perl5 that ends up being. (Perl 5.10 and up is pretty much what I want, 5.8 and 5.6 are a drag if i might need unicode)
Not that installing Python is especially difficult, but still.
What was your first programming language? Because what seems like normal to you mostly depends on you.
QBasic, then Visual Basic for real programs.
Look across a bunch of common languages - Java, Perl, Python, Javascript, PHP, and of course C/C++/C#/ObjC, braces + semicolons. Even Bash and Go have braces.
>>> from __future__ import braces
File "<stdin>", line 1
SyntaxError: not a chance
Back to the topic, while I don't think its ranking doesn't generally reflect the actual popularity, the TIOBE index [1] is a good definition of "common enough" languages. So let's see... oh! There are 5--7 programming languages out of top 20 that do not generally use both braces and semicolons! (Python [not-so-strictly-necessary semicolon only], Visual Basic .NET, SQL [semicolon only], Assembly language, MATLAB, Visual Basic and Delphi/Object Pascal [semicolon only]). I also happen to know less popular but well-known programming languages that do not use both braces and semicolons at all (ML, FORTRAN, Lua, Forth and of course all lisps). So please allow me to reword your claim: why should braces and semicolons be normal?There is no "normal" syntax. There is more to languages than those inspired by C.
Almost anything can "work"
> 2) Is ubiquitous
So is awk, don't write software in it.
> 3) Is stable
This will become questionable as Perl slips into non-maintained mode.
> 4) Is normal (semicolons, braces etc)
Perl syntax is very far from "normal"
Perl 5 has dereference operators, which got called "sigils" by lazy documentation and book authors.
EDIT: To clarify, I mean there should never be the expectation that you can start working in a language in earnest without properly learning it.
$ → S → scalar
That's how I learned and remember them.
What makes that less obvious is when you work with references to arrays; the reference itself is a scalar, so it gets the scalar sigil. You can dereference it, though, by sticking an array sigil in front of it, thus telling the interpreter "hey, this is a scalar reference to an array".
Python3, which is incompatable with python2, came out in 2008.
If you're going to judge python by python3 you should at least hold perl to 5.020
Same age as Haskell.
def foo(bar=Bar(1)):
Unless bar is a primitive type, in which case each invocation gets the same value?
def foo(bar=1):
Perl wears its idiosyncrasies as badges of honor. Python keeps them hidden so they can sneak up on you when you least expect it.
Yet after yesterday's attempt to get a small cherrypy webserver running, python doesn't.
As another data point, I've had very few good experiences with Python. The primary exception has been with PyQt5, and chiefly by going with Qt's conventions instead of Python's (i.e. being as Qt-like and un-Pythonic as possible).
https://stackoverflow.com/questions/54831915/python-3-5-2-in...
> I want to run this on a production Server with ubuntu 16.04. My Sysadmin tells me that I shall not use pip or github repository, because of ... something wrong with using anything but apt ... he says.
Sysadmin is right.
First response?
> sudo apt-get install python-pip
Now there are ways to convert modules to debs (or rpms on redhat style machines) so they can be managed cleanly across an estate, but that's not the python way.
Before that I was a huge Perl fan. I even ran a localized version of perl.cctld in my country. With documentation in our local language.
I ported EBBS/BBS to A/UX and ran one for a few years.
https://en.wikipedia.org/wiki/NCSA_HTTPd
https://github.com/virtadpt/eBBS (my name is in the credits)
Perl also has no shiny features. It's just extremely dynamic and paradigm agnostic, great at shell interactions and natural at text processing (specially with regexes), good enough at binding C/C++ libraries, has package management built-in (CPAN) and an automated testing culture. Perl basically set the baseline for a lot of today's languages.
Perl5 devs put a lot of focus on making Perl even more dynamic during the 2000s, relying on CPAN/community to fix language quirks and plug in novelties. It worked to some extent, Perl got to the point where it could be written like Haskell, Ruby... During that time it also got a powerful but optional class system (Moose) and reinvented itself as a web language (Mojolicious, Plack).
Unfortunately it all came too late, most Perl5 devs moved to something else or maintained Perl5 code frozen in legacy mode. Fewer and fewer started new projects using Perl. For instance, nowadays I tend to pick a language/ecosystem pair based on these concerns first depending on what the job requires:
- asynchronous event processing, specially network related -- I pick JavaScript/Node.
- real parallel processing and highly performant systems, I'd use Go.
- "we got a package for that" -- Python or JS at the forefront here. It depends on what package I'll need for the job, typically it's not going to be in CPAN.
- great code reuse/portability -- to me JS wins here. And still the only single language for both server and browser.
- compiled hard-packaged binaries: Go, but also Rust, Elixir, Nim, and others can fit the bill.
- code style and paradigm -- a very personal choice, I think Typescript and ES2019 are both great here. Golang and others are also great if I feel like staying away away from OO.
Perl would have a hard time competing in any of these fronts, but it still is one of the greatest for practical extraction and reporting, text or database wise and easily available or pre-installed on many servers out there new and old.
doesn't this apply to python, although now there are additional competitors?
> reinvented itself as a web language (Mojolicious, Plack).
highly opinionated and potentially flame creating , but -
perl was the original web language in the CGI era, PHP pretty much dethroned it due to simplicity of server integration, rails knocked them both out of 'leading edge', then the Node/SPA era arrived.
mojolicious / plack are pretty much things to bring perl up to date w/r/t the 'rails' type
I love Perl but I'm currently looking at around a decade of experience which is really hard to take into a new role. I'm working a bit on Wordpress now but PHP is basically close enough to Perl for me that it sits in a weird uncanny valley where I get things just wrong enough to cause headaches every time.
I did not start using python3 until last year, but I was aware it was evolving quickly to one day become usable.
I'd need to do some more polishing before putting it up on GitHub or something (namely: sanitize incoming single-quotes in CSV cells, and possibly break the steps into subroutines), but I was yet again impressed by the ability to automate a bit of text data processing with less than an hour's work.
Are you a politician? Because that's a very round about way of putting it. :P
If the rate of increase is negative, there is an actual absolute decline. Is that want you were meaning to say?
https://en.wikipedia.org/wiki/Derivative_test#Second_derivat...
https://trends.google.com/trends/explore?date=all&q=perl%20t...
https://insights.stackoverflow.com/trends?tags=perl
There are also anecdotes such as this:
https://www.perlmonks.org/bare/?node_id=1225020
Perl6 was supposed to help with this. It has yet to do so.
I remember the time before Irssi, when even CLI clients regularly had remotely exploitable vulnerabilities in them (including the 1337 BitchX). Irssi was programmed with safety in mind. Either way, scripts increase attack surface.
I'm not using IRC clients right now (I quit after realizing more than 50 windows open isn't productive), but I really love both Irssi and Weechat's interface (including default themes) and CLI. Did have to finegrain the configs but the same's true about nearly any application these days.
I have a crontab that runs every five minutes: 30 lines that use one CPAN module, no bugs in years. The code has the kind of hackiness that's okay when it fits on one screen, and it fits on one screen precisely because of that hackiness.
How expensive can it be to rewrite a hundred or a thousand lines, if a perl script keeps growing? And what are these languages (you use plural so I do too) that are optimal for every case?
A hundred lines, or a thousand lines, is not so bad. But where do you draw your line in the sand that you're going to stick to? If you don't insist right at the start on coding to your maintainability standards, why would you make that change when your script went from 30 lines to 100, or 100 to 500 (no doubt always driven by an urgent functional need)? It's always easier to add one more change to the existing codebase; the only point where you'd rewrite is when the codebase becomes literally unmaintainable because no-one understands it, and at that point migrating it is extremely expensive.
> And what are these languages (you use plural so I do too) that are optimal for every case?
I'm not saying optimal (I doubt that's possible), I'm saying pick a language that's adequate for every case. I'm partial to Scala, which was explicitly designed as a "scalable language" that would work for large and small codebases; other ML-family languages (e.g. OCaml) are similar. In my book any language with the ML featureset is more than adequate for large codebases, and any language that has a REPL/interpreter and doesn't require Java-like explicit types is adequate for scripting tasks. (Side note: I've always found it odd that Perl didn't have a first-class REPL, since IME that's one of the biggest natural advantages of scripting languages). If you want to start from the scripting end, Python, Ruby or even TCL offer a level of consistency and least-surprise that let them scale to larger codebases than Perl, though I wouldn't really want to use any of them on a truly large codebase.
(I still believe it - too clever constructions which feel smart at the time you come up with them :)
Personally, I don't tend to have a hard time reading code I myself wrote, even long after the fact. But I will concede that Perl code _can be_ hard to read if the author has been too clever.
That's the coder, not the language.
Which is more obvious what's happening.
print ''.join('%(pre)s%(num)s %(bot)s on the wall, %(nul)s %(bot)s,\n%(tak)s\n' % (lambda c,b: {'pre':['','%s %s on the wall.\n\n' % (c,b)][abs(cmp(c,'Ninety-nine'))], 'num':c, 'nul':c.lower(), 'bot':b, 'tak':['Go to the store and buy some more... Ninety-nine %s.' % b,'Take one down, pass it around,'][abs(cmp(x,0))] })((lambda x,o: [(['Twenty','Thirty','Forty','Fifty', 'Sixty','Seventy','Eighty','Ninety'][x/10-2]+'-'+o.lower()).replace('-no more',''), o][int(x<20)])(x, ['No more','One','Two', 'Three','Four','Five','Six','Seven','Eight', 'Nine','Ten','Eleven','Twelve','Thirteen','Fourteen', 'Fifteen','Sixteen','Seventeen','Eighteen','Nineteen'][[x,x%10][int(x>=20)]]),'bottle%s of beer' % ['','s'][abs(cmp(x,1))]) for x in xrange(99,-1,-1))
Or my $firstline = 1;
my @tens = qw/Ninety Eighty Seventy Sixty Fifty Forty Thirty Twenty/;
my @teens = qw/Nineteen Eighteen Seventeen Sixteen Fifteen Fourteen Thirteen Twelve Eleven Ten/;
my @ones = qw/Nine Eight Seven Six Five Four Three Two One/;
sub go {
my $num = shift;
my $bottles = "bottles";
if ($num eq "One") {
$bottles = "bottle";
}
if ($firstline == 0) {
print "$num $bottles of beer on the wall.\n";
print "\n";
}
$firstline = 0;
print "$num $bottles of beer on the wall, ".lc($num)." $bottles of beer,\n";
print "Take one down, pass it around,\n";
if ($num eq "One") {
print "No more bottles of beer on the wall.\n";
print "\n";
}
}
foreach my $ten (@tens) {
foreach my $one (@ones) {
go("$ten-".lc($one));
}
go($ten);
}
foreach my $teen (@teens) {
go($teen);
}
foreach my $one (@ones) {
go($one);
}
print "No more bottles of beer on the wall, no more bottles of beer,\n";
print "Go to the store and buy some more... Ninety-nine bottles of beer.\n";
print "\n";
People can do clever things in either language, people can write clear code in either language.The Perl culture is overall about being practical but also being clever and leaning towards code golf.
The Python culture is overall about being explicit and readable.
As usual, YMMV, but that's the impression an awfully lot of people get, so there must be some truth to that.
Perl is great because it has nuances many languages don’t, for instance sigils which allow the language to evolve safely (adding language constructs won’t clash with vars.) and “magic variables” designed to allow shortcuts like we use in spoken language. But, just like people shouldn’t use much spoken vernacular in writing, a lot of those features are designed for quick “conversations” and one-off scripts. And it should be peer reviewed before publication.
Python is great for many things too, but it’s no C++, and it’s also no Perl.
I mean, a lack of type annotations is normal for a scripting language, but Perl always seemed to go a bit far by not even having a standard way of showing parameter names.
I remember doing `my ($arg1, $arg2) = @_;` or something similar, but even that felt weird, and not everyone adopted that convention.
sub mymethod($$)
my ($arg1, $arg2) = @_;
}
Apart from on the smallest programs. So it does have a method signature (obv it's an untyped language)There are shortcuts (just like you could write "doSometing() unless $var;"). Perl does give you options.
Python has different options for passing methods - passing variables, passing a dict,
P.S. while looking at python method declartions, you can send variables, a key-value, multiple keyvalue, and then there's this: https://stackoverflow.com/a/16785702
Which comes out with
f = lambda **dic: ' '.join(dic.get(key, 'None') for key in 'abc')
Which is no more or less readable than the most confusing perl I've ever encountered."Cool Shortcuts" are a problem in any language. There are values in anonymous functions on occasion (especially in sorts), there should be a very compelling reason to do so.
That does not work. Prototypes are ignored for method calls. You are expected to not abuse prototypes for signatures as they serve a different purpose.
> it's an untyped language
That's wrong even with the most lenient/charitable interpretation. Using an unexpected type of data is a fatal error.
perl -e'$x = {}; $x->[0]; print "survived"'
Not an ARRAY reference at -e line 1.
Type constraints have existed for a very long time, for example see http://p3rl.org/Kavorka::Manual::Signatures#Type-constraintsHere's a short example:
use 5.020_000;
use warnings;
use experimental qw<signatures>;
# this works:
say add(2, 3);
# this dies: Too many arguments for subroutine 'main::add' at ...
say add(2, 3, 4);
sub add ($x, $y) {
$x + $y;
}You golfed the Python to fit on one line with one-letter variable names, while you took a very literal approach with the Perl code.
... in any language.I'm not a programmer, I'll bash something out to solve a problem and move on, but in general I find reading other peoples perl far easier than reading other peoples python. Maybe that's because I have more experience with perl, because it's well suited for problem solving.
I avoid cpan and pip as much as possible as it makes code much less portable, and unless the module is available in apt (and thus trivial to deploy) it would have to be really compelling to get me to wrap into a custom deb, but that's because I'm not writing products, I'm writing utilities to help run my network.