Perl - the Detroit of scripting languages
speakerdeck.com
speakerdeck.com
The talk was held by Stevan Little, the creator of Moose, the person who experimented in Moe on a modern Perl 5, and the person who is currently working on a Meta-Object-Protocol that can be put in the Perl core to make good OO available out of the box, on which Moose, Moo and friends can built. He dearly loves Perl and has done a lot for it.
This talk is not a blind troll, but a warning, an overview of his past efforts and ends on a positive note.
Please do not comment before actually reading the entire thing.
Lastly, i would like to thank you for refraining from commenting on the contents. That is all i ask from anyone unable to take in the content. :)
- I can have a prototype running in a matter of minutes
- I have access to a vast library of modules doing everything I need (which helps the former)
- I write quick and efficient code
Do I really care about what other people think about the languages and tools I use? No.
However, I do care about efficiency and results
And, uh, working with a language that has been out for tens of years makes me believe my time investment is safer than with the fad-du-jour JS framework/library/whatever.
BTW, Maybe buying a home in Detroit (especially now that it's officially bankrupt and will hopefully be able to break out of the unions commitments) would be a good business move? That's a cheap place to bootstrap a business for sure.
The benchmarks just do not support. I mean, roughly in parity with Python and PHP, 20x slower than Clojure on average, 30x slower than Haskell, and on many tasks 100x slower than C. Over the whole bredth of the Alioth shootout programs, Perl trades places with JRuby as the slowest tested language.
Performance handling in high level languages is more about knowing where the landmines are (total heap usage in GCed languages, various troublesome malloc()s, weird caches in dynamic languages...) and trying to figure out as quickly and cheaply as possible if the program you're planning on writing is going to hit them, and how much it's going to cost to mitigate it. It's not relevant if the actual runtime overhead of the problem is 5% or 1000x, the end product is either fast enough or it isn't, and the workaround is either viable or it isn't.
Trying to protect yourself from this by picking the fastest language in a shootout isn't going to work because the perfect, gotcha-free language hasn't been invented yet. There are sound performance reasons to avoid any language, not just Perl.
So far I have not found anything faster than perl unpack (1), but I would be happy to. I plan to investigate Go soon.
Would you have another suggestion?
1 : or marginally faster, with the advantages negated by the time it takes to write the code. However, a 10x improvement would be very interesting to me.
I managed to optimize a similar problem as yours until it was I/O limited. 1/10th the CPU would not change anything in the overall system.
What else have you tried?
So, that rules out Brainf--k I guess
Any toy example can be made to look clean.
Perl uses sigils for two reasons. First, to provide a separate namespace for variables and language keywords. Second, to allow amount contexts.
Reasonable people will disagree about the value of both of those reasons, but those are the reasons why sigils exist in Perl.
You complained about the simple presence of sigils, without even stopping to think and consider what advantages they bring. I may be rude, but close-minded? No.
Imagine how ugly C would be if every variable that wasn't a pointer dereference had to be prefixed with $.
I am not necessarily opposed to meaningful sigils, e.g. Ruvy's @, but using them everywhere is a pure misfeature that hurts readability.
First off, in Perl the parens for a function call are not mandatory.
As such it is very useful for functions and variables to be visually difference, especially when you're using functions to generate parameters for other functions without any temp variables inbetween.
my W = qq_mult ConjugateQ, qv_mult RotationQ, Vector;
You can't tell at a glance what's going on and will need to look carefully. Adding sigils makes it quite clear:
my $W = qq_mult $ConjugateQ, qv_mult $RotationQ, $Vector;
Further, most languages have only one type of variable, a name for a single thing. Perl has multiple types of variables that behave differently. For example:
my $res = munge $one;
I see this and know that the function munge is passed one single variable. However, consider this:
my $res = munge @two;
If there wasn't the @ there, it would be easy to assume that munge gets exactly one argument. However the @ there alerts us to the fact that @two is an auto-flattening array, and munge will end up with anywhere between 0 and MAX_INT arguments passed to it.
Similarly with hashes:
my $res = munge @two;
They also auto-flatten, so they need to be marked as being different from scalars, but they also flatten in a very different way from array, in that they flatten into a list that alternates the keys and values. So they need to be marked differently from arrays.
Lastly, due to functions, variables, array and hashes having explicit sigils, they can be recognized by editors without any heavy analysis, enabling editors to mark these four types with different colors, which is extremely useful. Personally i don't see the sigils anymore, and instead just see the colors with which the types are highlighted.
(Bonus set: Actually C is often written with sigils too. I've often seen code where variables are prefixes with p_, s_, a_, i_, etc. They are not enforced by the language, but people often force themselves to use them. The downside: They are inconsistent from project to project and have to be relearned every time.)
Actually, that's clear as mud. Is this a function call of 3 arguments? Function composition/chaining, an array of 3 items?
Regardless, please defend how that is not just an improvement over, but better than, say:
my $W = qq_mult $ConjugateQ, qv_mult $RotationQ, $Vector;
w = qq_mult(qv_mult(CondugateQ, Vector))
try {
die "foo";
}
catch {
warn "caught error: $_";
};
try(
sub {
die "foo";
}
catch(
sub {
warn "caught error: $_";
}
)
);
As for the example, with mandatory parens it would look like this:
w = qq_mult( ConjugateQ, qv_mult( RotationQ, Vector ) );
my $W = qq_mult $ConjugateQ, qv_mult( $RotationQ, $Vector );
Where is this general case? I clicked thru the presentation and it boils down to here are some obscure yet interesting and trendy corners of the computational world where Perl doesn't work well, therefore we have to change everything.
We could have had the same presentation 20 years ago, just the trendy weird corners would have to change to match the times. Doesn't mean those trendy corners are useless, just not required to write 'good code'.
Not that the goals are inherently awful. I don't do anything requiring UTF-8 at work. Someday I probably will, and it'll probably be a PITA.
I tend to see this kind of presentation as rabble rousing. "Here's some stuff that other languages care about, although you don't care, but it obviously doesn't prevent any of you from being profitable and gainfully employed writing good code so its obviously not required, which is why Perl doesn't have it. If I can convince you to care about it, Perl would gain it or more likely you'd just leave Perl. Hope you feel uplifted, here's some internet meme pictures, k thx bye" I'm not sure other languages consider it "insightful" when they get a cut and paste rephrasing of the same presentation, which is possibly the most interesting part of the whole story.
It's like saying: Hey, all I do is bang in nails! The hammer is only tool anyone ever needs!
Shell scripts are for things that have almost no logic besides running other programs.
Perl is for things that do have internal logic, and aren't big enough for the lack of formal function parameters to be an issue.
Java is for things that talk to the database (or need other libraries, such as for reading/writing Excel files), or are too large to keep in my head all at once (most of which talk to the database anyway).
C is for the one program I have that needs the suid flag set, and the one wrapper that resets the process group ID. I don't like C.
The server that all these need to run on, is an AIX box that doesn't have Python or OCaml or Ruby installed. If it did, I suppose it's possible I'd use one of these for some of the cases where I use Perl or Java now. Or probably not, since less people know them (except maybe Python?).
PL/SQL is for things that run inside the database, because it's what the database (Oracle) comes with.
C#/.NET is for things that run on my and my coworkers' laptops, because Visual Studio provides a wrapper/installer that makes it dead simple to publish updates.
http://search.cpan.org/~barefoot/Method-Signatures-20130222/...
(And others on CPAN.)
>> [C for suid flag]
Afaik, you should be able to use anything for suid stuff.
---
Edit: What I do see is all the fastest test program for the interpreted languages, ruby, python, etc are all trading places for the slowest language - the differences seem to be completely minimal. (As there seem to be more perl test programs per benchmark the slowest of them is often near the bottom - perhaps that was your methodology.)
On that particular benchmark Perl is faster than the fastest JRuby, and Python benchmarks. And faster than 11 other benchmarks, but the secondary perl benchmark program is the slowest of the bunch...
My problem is with people claiming that some popular scripting language, say Ruby, is massively faster than all the rest. Which as a general rule is simply not true (although it may be the cases, for some well defined set of problems).
I don't make money writing benchmarks. I would challenge that given a flexible enough language, they're nearly meaningless.
Another very important point is most of my Perl is glue between enormous systems. Spending any time optimizing it would be a waste of time/money. I'm not writing real time video encoding here.
That's the case for any microframework whose language you're comfortable in.
BTW, Maybe buying a home in Detroit (especially now that it's officially bankrupt and will hopefully be able to break out of the unions commitments) would be a good business move? That's a cheap place to bootstrap a business for sure.
Not really because part of having a business is staying alive :)
- Perl 6 will STILL be in development
- You just lost 10 years of development experience and related marketability because now all those fad-du-jour systems are established industry leaders
- Nothing would have have changed back in Perl 5 land because everyone can only develop in 5.8.x or 5.10.x in case they break something.
The upside of course is that there will still be COBOL and perl programmers making a living ...
I'm trying to think of what the fads were in 2003. That was the tail end of extreme programming. Kernel03 or kernelcon or whatever it was called had a presentation about replacing devfs with udev since adsorbed into systemd. Simultaneously the new JACK system running on top of ALSA was going to take over sound on the desktop. In 2003 Moodle was the new leader of online course software. I would claim PHP4 was a very popular web programming language in 2003. Firefox was beginning its explosive growth. Bonjour/Rendezvous was new and making noise in networking. By far the most dominant handheld computing platform was the Palm, I still had my couple year old 3 and my wife had one of the M100 series. Dot-net and/or mono were going to take over the entire world; we're still waiting. I believe by this late the cuecat had already sunk along with most of the dotcom era stuff.
At this point I'm running out of ideas from 2003. I don't think "all" would be a good adjective. I'm having an easy time remembering fads that took off, and forgetting things that sank.
Seems doubtful to me. With yearly releases and a two year support cycle, only the enterprisey distribution nonsense is still infected with that particular brokenness.
Depends on what you're willing to give up. Emergency response time can be pretty important.
(disclaimer: I live in Flint, MI, another of those top-5 cities)
Me too! Most violent city, 3 years running!
I can certainly appreciate the desire to make something fun, either for your personal edification, or to try to engage the constructive aspect of your community, I have to wonder: is there enough manpower left in the Perl community to handle all five-six of these revitalization efforts, while also continuing the march to Perl 6 and maintaining 25 years of backwards compatibility? The analogy to Detroit really couldn't be more apt.
In 1998 Perl had a substantial portion of all programmers simply because it was the best free tool for a wide range of jobs. In 2013 nearly every problem has a language more appropriate than Perl. You don't have a captive audience anymore. These days, almost every language is marginal and makes a living by cultivating a small but fanatical base. In order to play the new game, Perl has to find a new pitch, and "we finally work like real modern programming languages" is pretty weak. There are thousands of languages with real object systems, real exceptions, real functions, etc. Until Perl throws its weight behind something more interesting and substantial than "backwards compatibility with filth" and "someday, it might be nicer" it can expect the current trend of net emigration and a Detroit-like heat death to continue.
Citation needed. Seriously, please stop making such broad statements based on your feelings and opinions and instead please make an earnest effort to separate objective facts from your subjective perception.
And just to address another point: Perl has multiple real object systems that are inter-compatible (with one small exception), and are considerably more powerful than the OO of Ruby or Python. The only real complaint you can level against Perl is that they need to be installed from CPAN; which is not a very useful complaint since all the useful bits in Perl need to be installed from CPAN anyway.
Besides that, language appropriateness is necessarily subjective. I love Prolog and Haskell and use them all the time. My passion for them makes their thorns seem small and easily avoided, but the reality is that those thorns are sufficient to keep out a lot of curious people. I didn't really figure out how to write a Prolog program that handled command line arguments until a month ago. Prior to that, I would just run Prolog, consult my script and run my function. Simple enough barrier, but one that would have kept most of my friends away.
I think you took my "real X, Y, Z" list as a literal insult, which isn't what I meant. I don't know Perl. I was just referencing the presentation, not trying to sling arrows at Moose or whatever.
I do think that providing multiple options where most languages have one option built-in is unnecessary cognitive load. One doesn't "choose" an "object system" for Python--what benefit would one hope to gain? That the options are compatible is nice, but it's indisputably more work for everyone than just having one.
Having what are essentially language built-ins elsewhere available instead on CPAN is worth complaining about, because if you're going to do things differently from others, the difference should be motivated by something. That something in Perl is backwards compatibility and the desire to provide lots of options. The former is an explanation, not a rationale, and the latter implies, again, a cognitive cost. When you spend my attention and brain power, I had better get something in return. "Options" are only worth something if the alternatives are worth choosing between. In the case of OO, complete intercompatibility between them suggests the differences are minor; in that case, why have them?
Also, no, i did not take it as an insult, just as a false statement from someone with little knowledge of the subject matter.
> I do think that providing multiple options where most languages have one option built-in is unnecessary cognitive load. One doesn't "choose" an "object system" for Python--what benefit would one hope to gain?
That question is actually understandable and easily answered. Two benefits:
1. More expressive power for example. Perl has Roles that function in a way unmatched by most other languages (i say most because i don't know all languages, i honestly don't know one that has roles as good as perl). They make it possible to share methods among classes without needing to resort to base classes and diamond inheritance issues, while still retaining full introspection capabilities. Another system implements most of the sugar, but leaves out some really complicated features in order to make it much faster. Yet another provides the most speed by implementing the core in C. This makes it a bit harder to debug, and less portable, but if you need massive speed, there you go.
2. The ability to try out different things and work out in real life situations which features are needed and which ones are dead ends. This ability is amplified by the ability to simply extend your object system by writing more Perl. In comparison, Ruby has an ubiquitous one, but when they desire to change it, they cannot easily test the changes large-scale in real situations, and even if a change is undoubtably positive, they still have to convince Matz first, and then implement it in C.
> indisputably more work for everyone than just having one.
Actually that is disputable. In the past it was the case, yes. But nowadays they have converged on the same syntax and present themselves identically to consumers. That means porting between different systems is trivial, and people using a class don't even notice changes. And that is not just fluff. The perl community has been doing this actively, for example here:
https://github.com/rjbs/Email-Sender/compare/52fcbbb9b6a1c4a...
I'm not going to comment on your last paragraph because you wrote it while having decided that what you wrote before is actual fact and it is thus very flawed. Please stop doing that and use more question marks.
I appreciate your examples. I do think they would have been a better place to begin.
As for cognitive load, it is a thing. Converging on a single syntax will certainly reduce it, but as long as there is choice there is some. I do wonder to what extent you can have meaningful variation within the same syntax. Clearly it works for Perl and Perl users--I'm not trying to argue from first principles that it is impossible or stupid, only that it has a cost and that costs must have benefits or they are unnecessary. You apparently see some logical fallacy in that, but there isn't one. You just know the benefits and see the costs as reasonable. There's nothing wrong with your position on that, but you should have taken my words as an invitation to share what you know rather than blandly issue orders, a singularly unhelpful kind of discussion. An invitation was my intention, and I am sorry that I failed to convey it adequately.
(I'll assume you're not a troll despite acknowledging that you talk about a subject you don't know.)
You are seriously arguing against flexibility and configurability of your programming environment? I guess you don't like Lisp either?
To answer your specific point, re OO system: Today the de-facto Perl OO system (Moose with relatives) is better than the alternatives among other scripting languages. Without language extensibility, that wouldn't have happened.
Exactly where the optimum is for sizes of std libs can and should be discussed. IMHO, after a few work years, a week or two extra to learn an environment is worth it when you get things like Moose. YMMV, but since you acknowledge that you don't know the environment, well...
Not comparable. Better.
> your LINQ
DBIx::Class. I don't want to go to the effort of going into massive detail here, but in short: It allows you to write SQL access code under which you can use ANY database (SQLite, MySQL, MSSQL, Postgres, what have you), without changing a single line of your code. It will not only deploy your schema onto the database server, but also optimize your queries, so that for anything of slight complexity, it will likely be faster than your hand-written queries.
My explicit point was that it is better [edit: which was an answer to your question about value from flexibility, compared to the cognitive cost], don't do straw man attacks please. (Are you trolling?)
Edit: Thanks for the downvote. :-)
Edit 2: As answer to the next long, long comment with parts going all over: Of course complexity has costs. if you really just want a good reference on Moose to see exactly what it is, ask Mithaldu. He probably has better sources than me. This is what I give people (it is getting old): http://rjbs.manxome.org/talks/moose/Moose.pdf
Edit 3: And to avoid being called a "troll" -- don't write shit like "In 2013 nearly every problem has a language more appropriate than Perl." while you acknowledge you don't know the subject...
We're talking about the cost/benefit of flexibility as evidenced by systems like Moose. It's quite irrelevant to my point whether Moose is better or not, so it is quite a stretch to accuse me of a straw man attack. For that to be the case, I'd have to have said that Moose is worse than (say) Python's OO therefore flexibility is bad and Perl is bad by extension. What I'm actually saying is that flexibility isn't free, it has a cost, and that the cost must be offset by other benefits. If Moose is better, that's a benefit that might outweigh the cost, but nobody seems to be interested in telling me how it's better when they could instead attack the argument, by, for instance, hauling out the well-worn stack of logical fallacy accusations. If I sound irritable, it's only because I start conversations on HN in good faith expecting, you know, a conversation, and these kinds of vitriol-powered slamfests are not what I have in mind. Am I a troll indeed.
Edit: One need not know a language to find oneself in a niche where that language might be more appropriate. I didn't mean that remark to offend and I regret my wording. I certainly wasn't out to troll. That said, surely you do not believe one must learn a language inside out before knowing whether it is appropriate to a domain. I know many languages and work in several domains and never find myself thinking Perl would be better. I might not be in the right domains, but if we were having this conversation about C++, Java or Haskell it would be easier to rattle off a few example domains rather than scoff at the question.
You had a hard opinion about something you admitted you don't know. That is just not a serious position.
>>We're talking about the cost/benefit of flexibility
Of course it has a cost with flexibility. The same goes for lisp variants, which have an even larger flexibility than Perl in the macro system. Since you claim to know many other systems than Python, you should know this...?
Every design choice have costs/benefits. You don't need to repeat that trivial point as if it is news.
(That you think another OO system is better without knowing both says more about you than anything else.)
And now you argue that because you (allegedly) know other stuff, your opinion is relevant?
(Why don't you go argue that the flexibility of easily reprogrammable syntax is bad with Lisp people instead? If Perl is bad according to you, their life must be Hell?)
(And fyi, re niches, all the scripting languages are very similar in capabilities and fill mostly the same niche. That is why we see so many language wars trolls from the aggressive language communities. But I think you know this.)
If you really are a non-troll wanting to seriously discuss a subject's pro/con, don't start with insults and then act innocent. I was polite to the first dozen language war trolls on HN claiming things were shit they have no clue about. That was a long time ago.
(Also, the net is full with Moose info -- I gave you my favorite reference and suggested you'd ask others. If you really wanted to read about roles, typing of attributes etc and build a serious opinion.)
PS For the third time: Have fun explaining to Lisp people your theories about flexibility's cost being too high. I'd love to see your insults to their reaction... DS
I was sold on the promises of this thing they were calling Perl 6, that was supposed to break backwards compatibility and "fix all the things" in one shot. I followed it for a couple of years and then something happened...
Somebody pointed me to tryruby.org - I discovered a language that was more expressive than perl, followed similar values (i.e. There's more than one way to do it) and opened me up to a new world of meta programming. I embraced & absorbed it and today it's my weapon of choice for most of what I would have reached out to Perl for.
Without a proper role system, I can't replicate a lot of my designs. Without a proper meta-model, I can't write attribute traits to extend the OO system when I need to.
So any time a ruby program of mine gets large enough to need a significant number of classes, it gets ported back to perl5 first so I can do it properly.
If you want something lightweight to experiment, try looking at Moo (written by me for smaller projects) - you may find that declarative attribute definitions, method modifiers and roles become very addictive very quickly :)
> weird sub signatures
No core signatures, but modules on CPAN that make signatures available.
> faked OO
The OO in Perl works almost identically to the one in Python, and i assume Ruby is similar too. The difference is that OO syntax sugar is implemented in core in Python and Ruby, while in Perl it is implemented in Perl directly and available on CPAN but in core.
There's nothing fake about it.
As for your conclusion: I have the strong suspicion that the difference lies less in the language itself [1], but more in the fact that with tryruby you had a modern learning resource available that taught a good coding style; while pretty much all Perl learning written before February 2012 are mainly based on material originating in the early 90ies. [2]
[1] The main Ruby implementation doesn't even have multi-core-capable threads, but uses something that's basically a VM-level event system: http://en.wikipedia.org/wiki/Green_threads
I see what you mean. However, that syntax sugar is what people are used to from other OO implementations. Lacking that sugar meant that I found myself regularly explaining the workings of blessed hashes.
Can someone more patient than I am tell me if there's some actual content in there?
Edit: well... I guess if I could not control the speed of the slides... that would have been worse.
The talk might have been different, but there is nothing of substance in the slides.
Trends come and go, but it's the general purpose languages that remain.
Perl seems to fit in that space that was occupied by the various BASICs in the 1980s, meaning that it was the best glue for everyday tasks. It's quick to write, has the right tools, and plays well with system tools.
In general, in my experience, it's the quality of staff and not the tools that makes the difference. Good management, coders and business-side analysts make a much bigger difference than what language you use.
In general, in my experience, few experienced Perl programmers use vanilla Perl. Almost all of them rely heavily on libraries and write code according to their own knowledge of what's most efficient.
Be wary of the world of trends in programming. It is as fickle as trends in education, interior design or nutrition. Then again, I'm still waiting for the Pascal comeback.
But how can that be?! Building a large scale application with Perl would be like building a F111A Aardvark out of duct tape! Quite simply, because Perl is the perfect duct tape. Too many people (knowingly or not) think of Perl as a crappy version of Python when it's actually shell scripting on steroids.
You see, way up here, where projects are constantly in development, paradigms are constantly shifting, and people are building new and exciting things, Perl is an unwelcome high school acquaintance who shows up at the party and makes everybody feel awkward and bad. But way down there, where people are just trying to keep Sputnik up and running until the Soviet Union collapses, Perl is a rock star! If it's too "small" for Java but too "large" for .ksh -- guess what, use Perl.
In my opinion, if you want a scripting language with function signatures, little to no support for UNIX commands, real object orientation (Moose), yadda yadda yadda... just use Python. No one's going to blame you. Larry Wall's not gonna feel bad. You have a right to your impact gun and bolts.
Just leave my duct tape alone.
* Perl is widely used and has inspired many other languages. These are not small accomplishments.
* How many major revision bumps can a language undergo before it loses its identity? How many breaking changes are allowed before it becomes something else? Perhaps the name is about something more than the language as crystallized at any one point in time? But how do you convey that to the community as the language changes?
I'm fairly sure that the identity of a language is defined by its teaching materials.
Up until 2012 all Perl teaching materials were largely based on text written in the early 90s. And as well, any newbie program tended to look like it was almost Perl 4. This was in spite of there being a concensus and strong effort to push among the active Perl community that modern Perl was the way forward.
Then two things happened: The books Modern Perl by chromatic and Beginning Perl by Ovid were written. I used these two books to teach newbies and their code is nothing like the code of newbies of old.
If these two books reach enough penetration, they will change the identity of Perl.
I have a copy on my desk and this cannot be emphasized enough that one inward facing book on what boils down to style and good taste, has already had more impact on the language than the last two decades of outward facing comparisons.
Insert forest and trees analogies in that we've got way too many presentations over the decades about how one individual tree needs trimming or fertilizing or bonsai treatment as compared to how our neighbors orchard does the same, but not enough real forestry management presentations.
http://en.wikipedia.org/wiki/Moose_(Perl) https://github.com/MoeOrganization/moe
That is already the reality of Python and Ruby. ;)
And, Common Lisp and Scheme and Java and C and C++ and SQL and Pascal and BASIC and C# and JavaScript and ... pretty much every language that is (or ever has been) both popular enough to have interest enough to support multiple implementations and not proprietary to a single vendor.
Detroit - the Perl of cities :)
</irony>
Deleted comment
^^ here you go.