Why I stick with Perl
blog.jrock.us
blog.jrock.us
But it irritates me when I need to get at gpg from a web application, and can't just use a "libgpg". I have to fork a gpg process, setup file descriptors just right, write input to a pipe, wait for input on another pipe, and then parse the result
It boggles the mind that a perl programmer, of all people, would think this. The relevant code for the above is generally something like:
my $data = `gpg --decrypt --passphrase=$PASSWD $FILE`;
open INPUT, "gpg --decrypt --passphrase=$PASSWD $FILE|";
while(<INPUT>) { ... }
I mean, the whole premise of the point is counter to the perl philosophy, which is that you're already on a great and useful platform and just need a tool to tie things together nicely. If you want a single platform designed around the idea that all meaningful tasks are available in a library without leaving the sandbox, you are looking at Java or .NET, certainly not perl...
Basically, it's easy to do things if you aren't aiming for security or correctness, but in this case, I am.
$PASSWD was available to everyone on the box anyway. You stick it in memory, it's readable. That's the way life works. You probably stuck it in a database too, right? Did you lock down the socket from local users? Probably not. Did you store the database password off-line? Probably not, because you wanted unattended startup to work.
The bottom line is that if you want to do password storage in your web app that is secure against local users, you have vastly more work to do than just finding a CPAN library somewhere.
You win the Internets!
Not cool. At all. (I don't know enough security to determine if you are correct or not, so my comment only applies to your tone)
Say UID 100 is running PID 200 which has "my password" in memory. How does UID 300 read the memory of PID 200?
If it requires a security flaw, I don't think it's fair to compare to the ease of using 'ps' ... if it doesn't, please explain so I can learn something today. :)
I'm not saying local-attacker password security can't be locked down securely. Simply that doing so is a much more complicated proposition than "use a library". In practice, most web applications are not secured against hostile local users. I'm not sure I've ever seen one that was. So to my mind, ruling out some of the most powerful tools you have (perl's IPC facilities) in a vain attempt to achieve a security level that the rest of your architecture doesn't support seemed silly.
$ cat /proc/3791/mem
cat: /proc/3791/mem: Permission denied
$ ps aux | grep 3791
ntp 3638 0.0 0.0 24440 1480 ? Ss ntp 3638 0.0 0.0 24440 1480 ? Ss Jul30 0:01 /usr/sbin/ntpd -p /var/run/ntpd.pid -u 110:116 -g
I'm not sure where all these security discussions are going. My point was simply that taking the passwords out of the command line of gpg was woefully insufficient to secure them (note, again, that they're going to be stored in a database whose password is stored locally in a file), so using "security" as a reason to avoid perl's elaborate and very robust IPC facilities was a poorly grounded argument. Nitpicking about whether one (!) of my examples matches the original author's security assumptions is very much missing the point.
To get back to perl: note that you can feed the password to GPG via stdin if you want with just one extra byte to the open string, and a call to autoflush() and print().
Well, that's just what my example question above said.
Why on earth would you do that?
To avoid them reading the memory? It's pretty common for shared hosting to have each user running on their own local port and a reverse proxy listening on 80, just so that each user gets the benefit of the basic UNIX security model (we assume trust in the admin, of course). I guess I don't understand the whole password in a file thing, either. If those have proper permissions then the only person who have to trust other than yourself is root. Seems better than nothing to me.
The unix security model isn't amazing, but it does a reasonable job of protecting you from users other than root -- putting cleartext passwords in command line arguments is one way of throwing that away.
Then again, as you seem to be saying, manipulating cleartext passwords are usually a bad idea in the first place
In a secure shared hosting environment applications do not run as the web server UID. They run as the owner of the virtual host. Apache does this with mod_suexec.
Are you doing it differently? Why on earth would you do that?
Just because security by obscurity is not a barrier you should lean on doesn't mean that it lacks value in combination with other measures.
Script kiddies will struggle to isolate data from memory in order to get passwords, particularly if they can't get binaries on the machine and it has no compiler. On the other hand, they will have no trouble using a copy and paste ps usage.
And when I say script kiddies, I also mean the sort of 9-5 culture, corner-cutting production support teams that exist in some organisations and who might have accounts on shared hardware that your application runs on. Do you really want to make it easy for them to do favours for people on the business side who want passwords they shouldn't have?
> The bottom line is that if you want to do password > storage in your web app that is secure against local > users, you have vastly more work to do than just finding > a CPAN library somewhere.
I strongly disagree.
Damn straight. Check out the DNS spoofing defences - using random ports to issue requests on. There's a known limit to port numbers, and you could probably circumvent it with massive amounts of traffic, but using an unknown port number makes it hard enough that performing that kind of attack is significantly more difficult.
I moderated you up assuming you were about to provide a better way to perform the task then nastily calling an external command to do it.
I should have read your post fully first.
Then I am not like you.
1. Finding the available libraries for the job. There might not even be one if you're doing something somewhat novel. Or there could be a huge number of them, most of them crap.
2. Evaluating the libraries found in step 1. You might find that no library is a superset of the others, that for example there's 3 of them that each tackles 1/3 of the problem in exactly the way you want but you have to compromise and choose only one of them.
3. Dealing with the library's bugs, limitations, misfeatures, incompatibility with your usual programming style. To fix those problems you either have to dig deep into a codebase that's not even your own (if you even can), or you take the passive approach and contact the vendor to ask them to please fix the bug or implement the feature and then wait 2 weeks or 2 months or 2 years or foverer until it gets done.
Whereas with the "I have a strong NIH syndrome and will do everything myself" I have lots of up-front costs but at least I know pretty much exactly what they are and the problems that come up are fun things like: "oh, I need to master this super powerful technique", not boring ones like: "oh, I need to get in touch with the maintainer of this library to implement a new feature I need if he so chooses".
As for #1...CPAN search works pretty well. Google can also help. Obviously every hacker is doing something somewhat novel or else they would just be using existing software...but the point of CPAN is to insure you don't have to do the boring crap over and over again. Session management? Done. Database access? Yep. Payment processing? Sure. Templates? Definitely. Internationalization? Well, you still have to hire the translators. Processing and spitting out XML, HTML, YAML, JSON, RSS, etc.? Of course.
So, either you're writing all of this from scratch, or your software is anemic and lacks basic APIs and data export capabilities and interoperability and good security, etc. I'm sure your app will kick ass when you finish it two years from now.
As for 2...does evaluating a couple of competing libs really take longer than writing one from scratch? Maybe, for some classes of problem. But there's a lot of stuff that just takes a lot of code to accomplish.
"oh, I need to master this super powerful technique", not boring ones like: "oh, I need to get in touch with the maintainer of this library to implement a new feature I need if he so chooses".
I don't even really know how to respond to this one, but I'll take a stab at it. Is a template engine a "super powerful technique" that you're excited to figure out? Payment processing? Session handling? How about a test harness? Form validation? Having a good selection of available libraries saves you from the boring crap, not the other way around.
Hmmm...That seems a little odd in the middle of a conversation about a blog post that talked about pretty much nothing but CPAN. I guess you can talk about anything you want, but you can't possibly expect us to know you're not talking about the same subject as everyone else.
I find your adversarial tone inappropriate.
I'm sorry you feel that way. I'll try to tone done the sarcasm in any future replies to you.
2. scott_s references the latter, without referencing the rest of the article or CPAN.
3. KevinMS references scott_s' answer.
4. I reference scott_s' "Then I am not like you." and KevinMS's answer.
5. You somehow think I'm attacking CPAN and go berserk.
You might want to review "locality", "dependencies", "modularity" and other concepts.
Berserk? Really? You took my response to be the ravings of someone gone berserk? Huh. I don't know what to make of that.
"maybe you're just that much better of a developer than I am [...] So, either you're writing all of this from scratch, or your software is anemic and lacks basic APIs and data export capabilities and interoperability and good security, etc. I'm sure your app will kick ass when you finish it two years from now. [...] I don't even really know how to respond to this one, but I'll take a stab at it. Is a template engine a "super powerful technique" that you're excited to figure out? Payment processing? Session handling? How about a test harness? Form validation? Having a good selection of available libraries saves you from the boring crap, not the other way around."
Such extreme condescension stemming from an misunderstanding of what I was saying I consider "going berserk", yes.
"I guess you can talk about anything you want, but you can't possibly expect us to know you're not talking about the same subject as everyone else."
"I'm sorry you feel that way. I'll try to tone done the sarcasm in any future replies to you."
What a beautiful paragon of passive-aggressiveness. I applaud you!
http://www.joelonsoftware.com/articles/LordPalmerston.html
Because, though this is a relatively sober and even-handed example of the genre, it still inevitably contains howlers like this:
Search for "rails". Most of these programmers will depend on one library, Rails, but then get the rest of their code by cutting-and-pasting from blogs.
Someone has never heard of gems:
http://en.wikipedia.org/wiki/RubyGems
Which is hardly surprising. This is the canonical problem with articles like this one: The guy doesn't waste time tinkering with Ruby, because he's busy being productive with Perl, so why should he be expected to know anything about how the Ruby community works?
(This is why I try not to write about Python except to assert that it must be one of the best languages in the world: I'm literally unsure how to write hello, world in Python, but smart people use it all over the place, so deference and politeness forces me to assume that it's utterly perfect until proven otherwise.)
Anyway, I'll help out by saying that I think jrockaway's point still stands, albeit on fewer legs than before: Perl is a considerably older language, its community is more experienced, it's (among other things) a traditional tool of system administrators -- who are, in my experience, relentlessly pragmatic people who crave stability, reliability, and clear docs -- and so it has had a lot more time and incentive to organize and standardize its packaging and documentation systems. Ruby does have docs difficulties, as do many open-source projects which have recently experienced explosive growth.
I've heard of gems. They are more like tarballs than CPAN modules. For one thing, there is no canonical place for All Things RubyGem.
No. He has. So have I. He's commenting on the reality of a culture dominated by fanboyism and a framework advertised on the basis of being "full stack" which to naive eyes can seem to be "everything you need".
rubygems -enables- granular reuse of libraries, and many ruby people take great advantage of it.
But as a whole the perl community has got a hell of a lot more practice than the rails community at it - and of course rails has inherited the same fanboyish muppets that used to give perl a bad name (line noise? no, that's a phase you go through and grow out of if you're any good, takes about 3 months or however long it takes you to need to maintain your own line noise :), went on to give PHP a bad name, and in a few years will probably have given rails a bad name (possibly they are doing already? I dunno, being a perl head I tend to ignore that sort of groupthink idiocy, my language was unfashionable when I first learned it :)
There are muppets everywhere, but I don't think they dominate the Rails community.
Rails still looks hopelessly monolithic to a perl dev though - we can't quite see why you'd want to ship a DB library in core when, hey, web apps don't -all- talk to SQL databases.
As for the quality of said DB library, well, many smart ruby developers have torn it to pieces quite adequately so I'll just say "google is your friend" and not bother with a half-informed opinion of my own.
If you can't see why a web app framework might want to ship a DB library, you're blind.
a.) A lot of people really like perl and b.) A lot of people really hate perl.
I think people get really upset when they are told that Perl isn't actually dead. I mean, it is, right? How come people are still using it!? </sarcasm>
c.) A lot of people really like hating Perl.
I think it goes beyond basic dislike of the language...that would require knowledge of the language, which is pretty clearly lacking in many of the flames. I suspect it's often human tribal tendencies. Rubyists and Pythonistas see another tribe competing in the very same environment with very similar capabilities (because, let's face it, mild syntactic differences aside Perl, Python and Ruby are more alike than they are different when compared to almost any other language), and they also see an opportunity to combine to beat up on a currently larger and older tribe.
By the time you've spent enough time with a language to get past identifying with just that one tribe, you don't feel as aggressive an urge to pontificate on the suckage of other languages.
I've never said there aren't valid criticisms of Perl...just agreeing with msluyter's very astute meta-comment, and adding one more option. I believe that people exist in both groups "b" and "c"...I wasn't overloading method "b" to actually mean something else. Some people in "b" probably even have valid reasons for hating Perl...but, I think we can all agree that the majority of flames in any language thread come from a place of ignorance about the language. After all, if you really don't like a language, odds are good you don't do much work in it, and so don't actually know a lot about it.
But, if you think we need one more option, I won't argue. There probably are people in category "d" (in addition to me, it seems), as well.
Actually, the answer is that those three do different things. GETF is for property lists which look like '(key1 value1 key2 value2), ASSOC is for association lists which look like '((key1 . value1) (key2 . value2)), and GETHASH is for a hashtable which looks like a vector/array and not a list.
GETF returns the value of the first key that matches. GETHASH will return the value of the hashed key that matches. ASSOC allows you to return any value for whatever key you're getting or whatever test you're using.
So each one has a different purpose. The trade-off is how you choose to map keys to values. It may be faster to use a hash-table instead of an assoc-list, or it may be easier to interact with an assoc-list rather than a property list. The point is that there's a choice and a reason why the choice is available.
This reminds me of all the newbie Java programmers who use the ArrayList class instead of looking for just the right data structure for what they want to do.
Blog software needing 212 different Perl modules: Not a strong argument for Perl.
That is explained in the next sentence after that.
You should really read the entire paragraph containing a point you want to reply to. If you just want bash Perl, reply to a point without a direct quote :P
* posts encoded in any character encoding, built-in to modern languages.
* supporting different rich text formats, equally available in modern languages.
* computing the MD5 sum of the post for the HTTP ETag header, built-in to modern languages.
* handing filesystem paths in a cross-platform manner, built-in to modern languages.
* parsing the configuration file, built-in to modern languages (for JSON, XML, YAML, KVP's or eval).
* reading PGP signatures, equally available in modern languages --- not that I'm sure why I'd want that in my blog.
Except for Textile or Markdown, I'm not sure there's anything here that would make me reach out of the box in Ruby or Python.
This all seems pedantic, I'm sure, but here's my point: CPAN is constantly cited as the killer feature for Perl, but:
(1) here's an article that makes that argument and doesn't show anything CPAN provides that you wouldn't get in a more modern language, and
(2) if you're wading through 200+ modules just to get a blog up, I'm seriously not sure that's an argument for how CPAN makes a developer's life easier. TMTOWTDI, yeah, but there's also value in making it clear and obvious what the right way is.
if we're going to do that, we get to rule out, say, lisp. Which I'd rather write over most "modern" languages any day.
And so far as I can tell, you're basically saying "I think the perl standard library should have more stuff in it and because it doesn't have the stuff -I- want in it, CPAN is pointless".
Well, sure. Except that not having all this stuff in core means I can update, say, my rich text processing stuff or a configuration parser on its own, and only re-QA that component rather than having to wait for an entire language point release.
And "wading" is a nicely loaded description as well. Here's what I did when I installed angerwhale:
tar xzf <tarball>
cd <unpacked dir>
perl Makefile.PL
make installdeps
perl is just syntax. CPAN is the language.
And by that standard all this -is- built in. It's just optional :)
The rest of your reply kind of misses my point. If the 50-or-so modules you get from CPAN have built-in analogues in Python, why do I care about CPAN? Can I get a raytracer straight out of CPAN? Sure. So? What's a profoundly useful library in CPAN that isn't easily matched in Python?
I bet you have an answer, and I'd like to hear it.
What modern features is Perl missing? I hate to break it to you, but Python and Ruby have basically the same featureset as Perl.
What's a profoundly useful library in CPAN that isn't easily matched in Python?
A single profoundly useful library is not the point of CPAN. It's finding that random library that you need in some corner of an obscure internal application where CPAN really shines. Every language has their super-great web framework. The depth and breadth of module available on the CPAN is Perl's strength.
If I had to pick a single CPAN module I couldn't be without, though, it would be Moose; followed closely by Catalyst and DBIx::Class.
Moose --- doesn't Moose just make Perl's metaclass programming as simple as Python and Ruby's already is?
Catalyst is an web MVC framework. DBIx::Class is an ORM. Clearly we can get those libraries elsewhere. Is there something really amazing in CPAN that I can't get in a gem in Ruby? I get that a lot of gems are crappy (Ruby's a younger community), so I'm not going to come back at you with some random -0.1alpha svn trunk thing.
No. (this is coming from an old PERL programmer)
Of course, if you describe yourself as "an old PERL programmer" you may have missed a lot of the recent cool stuff on CPAN - the binary is called 'perl', the language 'Perl', and the mistake of calling it 'PERL' kinda went out with the 2001-era CGI.pm crowd (this isn't meant as an ad hominem - I've simply found that anybody who says 'PERL' as anything except a joke is -hugely- likely not to have taken advantage of most of CPAN and have no way of telling if you're an exception to that :)
Sadly I didn't get to catch up with them properly at OSCON to find out if they'd manage to steal much of the awesome stuff yet.
SQLAlchemy probably already has similar features though, so maybe they don't need to.
Ruby can't handle MI. Ruby's metaprogramming seems limited. But I'm not informed. I've had several perl ->ruby guys come back to perl going "hey! it's like ruby but with a flexible object system" to me. But I don't know enough about it to have an opinion myself.
Ruby doesn't do multiple inheritance, though you really do get 90% of what reasonable people do with MI with mixins, which work beautifully.
Moose.
I haven't been so happy programming since I first discovered scheme.
Or am I totally missing what Moose does?
The attribute metaprotocol and method modifiers are AFAIK rather more like the equivalents in CLOS or a smalltalk than they are like those in python/ruby, and the roles system is -way- nicer than mixins.
I actually considered once implementing mixins for perl the way ruby does them with an anon class to keep single inheritance, then abandoned the concept as being a colossal hack. python's MI OTOH is sane with a C3 MRO, so we stole that for perl - in the cases where a mixin or role isn't sufficient it works very nicely.
I'd suggest having a read of the Devel::REPL articles on http://chainsawblues.vox.com/ - that might show off the capabilities of roles+method modifiers better than I can explain.
a) uses a very human language, similar to other modern languages
b) is gaining interest, rather than decreasing interest like C, Java and Perl are [1].
Ack. re: Lisp.
1) I doubt Angerwhale itself includes 200+ modules. The core Makefile lists about 62 modules. It's likely the number quoted was counting both the modules used by Angerwhale and the dependencies of those modules and so forth.
2) Current best practices for developing a Perl application say that you should build the application itself as a CPAN module, even if you don't plan to release on CPAN, since that way you can take advantage of all the work that's gone into CPAN testing and deployment tools. However, if you follow that best practice correctly, it will mean you will take great care to list all your dependencies and their minimum workable versions. You will do this even with a dependency that is part of a core Perl installation. For example, one of the dependencies listed in Angerwhale is 'File::Find', actually comes with a base install of Perl. You don't have to do anything for the application to work. However, for strong audit and good programming practice, you really should list what you application needs. That just makes it easier to install and to troubleshoot installation issues. Also, in the future it is likely that Perl core will be smaller, so that we can have the flexibility to target Perl more carefully to different needs (like say to people using Perl for system administration, versus those in a very constrained embedded market). So this is just a good practice to future proof your application.
3) The number of required modules is nothing to be afraid of. When you install a CPAN module, like by using the commandline 'cpan' application, it will automatically follow, download, run tests, and install, any required dependencies. All you really need to do is something like:
cpan Angerwhale
And the build tool do the rest for you. This is also a good way to develop your application, so that you can use Makefile as a build and a deployment tool. This is just part of the standard, best practices for Perl development.
All programming languages come with extended and standard libraries. To be efficient in any language requires that you learn to 'wade through' as you say, those libraries, their documentation and test examples. There is no doubt the CPAN model has some challenges. CPAN is widely distributed, varying in quality, and lives in the complex negotiation of needs and desires which is a hallmark of the Internet. Like the rest of the Internet it can be wildly anarchic and is often brutally merit based. There have definitely been times when I have cursed it. However I do believe it is a wonderful testament to the viability of open source, distributed development.
Why not give them an actual bold text button? It's not hard.
And to answer your question, it's easier to hit the asterisk key than to reach for the mouse to hit the "italics" button.
I guess I expect tools written in a language which emphasises usability for developers to do the same for users.
What's your point?
They thought they'd call it Perl on Poles.
Then we realised that big monolithic releases were a good idea in the same way as vista was a good idea, and went back to depending on the exact things we needed for our app, conveniently cut up so we could depend on lots of little libraries rather than one giant one that contained lots of stuff we never used.
I'm sorry you find a culture of software reuse an argument against a language. Here, let me sell you this dot net - fifty pee and a windows license.
He's not complaining about using modules. He's complaining about using one hundred and fifty modules, and you're attacking him on an argument he's not making.
Your last statement is especially childish. See 'be civil' at http://ycombinator.com/newsguidelines.html
I'm trying to attack the idea that using lots of small modules is worse than having a few big dependencies - CPAN dependency chains often -look- big because we try to break things down into the smallest unit of viable reuse. It's a bit more work but it's part of what makes CPAN so effective a toolkit.
If you want, re-read my post and try and mentally apply a deadpan english voice with more than a hint of sarcasm and see if that makes up for my poor phrasing.
Choice of language makes a difference in productivity to your team, but in many cases, this will be swamped by other factors, like how experienced and knowledgeable they are and how well they work together.
Then again, there are highly specialized domains where a language will make you 100X more productive than another.
Hm. To me that's pretty much all that matters. I want powerful features and a nice syntax.
> Python doesn't have lexical closures like Perl's
What does that mean? Doesn't Python have lexical closures?
See the following link for an example and more details:
http://mail.python.org/pipermail/python-list/2004-July/27095...
Instead of list boxing, you can also use a function attribute to handle changing the variable in the correct scope.
It still won't be exactly the same, because Perl uses `my $foo` once to declare its lexicals; Python will require `nonlocal foo` declarations in all scopes that will use `foo`. The final result is the same, though: using variables from outer scopes.
The reluctance to break free from legacy technologies allows them to build more and more momentum. Unfortunately, this momentum delays innovation.