Three Tales of Second System Syndrome | Hacker News Reader

Three Tales of Second System Syndrome

blog.brentlaabs.com

207 pointscygx137 comments

    It just might be possible that Perl 6's crazy long development process can
    produce the best-adopted second system around

Unfortunately, I don't share the same optimism as the OP. In the real world, adoption is not only driven by how cool a language is, but the ecosystem of tools and developers around it. There are lots of Perl 5 codebases which won't be migrated because it's too expensive.

The true lesson learned for language developers is: Don't break backwards compatibility. Ever.

The Python community made this mistake, and they acknowledge it. Guido said this won't happen again. But at least for the case of Python 3, the number of breakages is relatively small, and the migration from 2 to 3 while tedious, it's not that hard. Even so, the migration of the whole ecosystem has been really really slow. But I think in the end the community will move.

The Perl case, on the other side, is much worse. Migrating a relatively complex project from 5 to 6 is extraordinarily hard. It's probably the same as a whole rewrite. Also, the community is divided, with some developers keeping the improvement of Perl 5, fixing bugs and adding new features. There is no incentive for migration.

If the migration has been so slow for Python 3, which had a much smaller barrier, I don't see how it could be better for Perl 6.

New languages are able to develop a new ecosystem when they introduce innovative features, e.g. Rust. But Perl 6 while being a nice language, doesn't seem to have any killer feature compared to mainstream languages.

I would never say it is likely that any language will be well-adopted. Fortune plays too big a role.

However, what Perl's long development cycle means is that its widespread adoption would depend on new users taking it up. It will be a new language from their standpoint and backward compatibility will be less an issue.

Widespread adoption will also require transitioning of a core group of existing Perl power users. After 15 years, all the emotionally inflamed debates, tirades and rants have probably run their course. Everyone knows where they stand and has had time to come to piece with it...and even the youngest among those who have been there since the decision are long out of short pants. Zealots on both sides have been living together a long time.

The more I think about it, the more I realize that Perl 6 was misnamed. It should have been treated as a new language with some Perl heritage, not just a new version of Perl.

The Perl 5 diehards aren't going to switch, and people who aren't already Perl hackers are going to think it's the same old Perl that scared them off ages ago.

It's a shame, because Perl 6 is a great language (and I say this as somebody who was put off by Perl 5), and the Parrot VM is particularly awesome.

It also has some of the best error reporting I've seen: it offers a number of possible solutions, and if you try to use a Perl 5-ism that's not supported anymore, it'll tell you what the Perl 6 equivalent is.

As someone that uses Perl daily I totally agree with you.

The name not only is bad for newcomers but also for Perl itself. Perl 5.0.0 was released in 1994 (!), today's Perl 5.20 is miles ahead but we can't bump the version because it would only make things even more confusing..

> Perl 6 was misnamed

Damian Conway agrees with you - but Larry Wall decides, and he insisted on calling it Perl 6. Maybe Conway has a point, but maybe so many people wouldn't have worked so hard on it for so long if it was called something else.

Perl unless true

> and people who aren't already Perl hackers are going to think it's the same old Perl that scared them off ages ago.

consire though, that perl6 is soon going to be 20 years old, there is a whole generation of devs which were never scared off by perl5 at all. I say this as someone who, ten years ago, didn't understand a lot of the complains about perl because they were mostly directed at perl4 problems :)

"The Perl case, on the other side, is much worse. Migrating a relatively complex project from 5 to 6 is extraordinarily hard. It's probably the same as a whole rewrite."

One of the features of Perl 6 discussed in the article is "Fully backwards compatible with the use of Inline::Perl5". So why is it extraordinarily hard to migrate? I admit it is non-trivial, but the designers made it a key point to be backward compatible. No need to rewrite, Perl5 and Perl6 can co-exist in the same codebase.

"There is no incentive for migration."

Have you looked at the features Perl6 provides[1]? They may not seem valuable to you but it cannot be denied that the new features are plentiful and powerful from a programming language perspective. [1]http://www.perlfoundation.org/perl6/index.cgi?the_long_perl_...

Inline::Perl5 is just some glue to communicate to a Perl 5 interpreter from Perl 6. It's a simple form of inter-process communication between Perl 5 and 6. It's not that you can continue writing your Perl 5 program/library in Perl 6.

This approach is not only really bad for performance, it's also very limited. For example, you can't just share data between both interpreters, however you can send some basic datatypes from one to the other.

This has been done for other languages as well, for example, check Inline::Python [1]. It's an interesting thing, but not very useful for production.

[1] http://search.cpan.org/~nine/Inline-Python-0.49/Python.pod

You do have a point, but your description sells Inline::Perl5 short: For example, it is possible for Perl6 classes to inherit from Perl5 ones.

Note that Inline::Python for Perl 6 allows one to do likewise with Python. [1]

"even PyQt's low level mingling finds all the information it needs and we can write programs using PyQt in Perl 6 :)"

[1] https://github.com/niner/Inline-Python/commit/82cab4181d98f0...

Yes, rewriting Perl 5 in Perl 6 would be quite an endeavor. However, that's why Inline::Perl5 was invented. It lets you use Perl 5 libraries in Perl 6 code:

  use Inline::Perl5;
  use MyProprietaryModule:from<Perl5>;
  use YAML:from<Perl5>;  # take advantage of p5 cpan

  my $foo = YAML::LoadFile('foo.yaml');
  my $bar = MyProprietaryModule.someMethod('baz');

So what this allows you to do is use P6 features where they might benefit your project, but still use all the P5 code you've invested in. There are some cases where Inline::Perl5 has problems, such as running P5 code multithreaded, but aside from that I've had few problems using it with stuff from CPAN and with my company's own proprietary code base.

There are also the modules Inline::Python and Inline::Java (for the JVM backend), but I haven't used them.

Inline::Perl5 is just a very thin layer to communicate with a Perl 5 process. This works well for small things but it's not practical for more complex ones. You can call code of any language from any language using this technique, that doesn't mean the languages are integrated.

Note that no actual IPC is involved: The Perl5 interpreter gets loaded from a shared library and afaict does not run in a separate process.

Have you tried it this year?

In his three minute lightning talk in October last year[1] nine demo'd the small things I think you're talking about (calling Python code from within Perl 5 code from within Perl 6 code and using XS-based Perl 5 modules from CPAN).

But that talk was merely a snapshot of his progress just a few hours after he began.

If you read the examples in the README of Inline::Perl5 as it is today I think you'll see that it is already a lot more capable than you are thinking it is. [2]

[1] https://www.youtube.com/watch?v=m_Y-lvQP6jI&list=UU7PuZDAIVM...

[2] https://github.com/niner/Inline-Perl5

I wonder if python3 design helped writing shorter and more robust programs.

I would think that correct unicode handling would satisfy that requirement.

That's what I was thinking about when typing my comment. I only wanted field reports, not only theoritical (even though clear) advantages.

It may be instructive to consider what exactly you mean by "correct unicode handling".

What would you say is the length of the Thai word ปฏิบัติจริง ?

What does Python 3 think is the length? What does Perl 6 think is the length? Can you explain the difference? And what do you think are the practical implications for code in Python 3 vs Perl 6 as it relates to "correct unicode handling", shorter code, and robustness?

I think he meant consistent more than correct. I've read that once decoded, the data will be stable, unlike python2 where transcoding (lossy one that is) may occur between different modules under your nose.

Right. Aiui both Python 3 and Perl 6 allow correct handling of Unicode byte buffers and whole strings.

Which will be great for applications that don't need to do anything at substring or character level.

Afaict, in Python 3, code that does things at substring/character level to Thai, Chinese, etc. text will be:

* buggy, even spuriously cutting characters in half if any string changes are made, if standard string/substring/character functions are used

* verbose and messy, if the reqired-for-correctness grapheme handling is introduced

If you're a Perl programmer in Thailand, drop me an email

> Migrating a relatively complex project from 5 to 6 is extraordinarily hard.

Are you willing to share any details of the project you migrated?

> The true lesson learned for language developers is: Don't break backwards compatibility. Ever.

Certainly true, and yet this is the situation that led to a lot of the hate for C++.

The reason for the numerous meanings of the "static" keyword for example is because if you introduce more keywords then you might break peoples code that used those proposed keywords as variable names.

And then of course a large percentage of C++'s baggage is due to backwards compatibility with C in the first place. Without that baggage C++ would be a much better language and yet you can't deny C++'s success from that decision.

So the moral of the story is that you either make a language nobody uses or you continue to cop flak for all your old decisions forever.

Yeah, life sucks. There's no good solution to all that.

The approach used for new keywords is to use lots of '__' on one end or the other. Anybody that used variables with those names was asking for trouble. Yet, then, the new keywords look like crap. Nobody wins.

My favorite, years ago, was the usage in the normal C header files in Linux of a linked-list variable with the name 'new'. It was obvious that nobody in Linux-land had ever compiled a C++ program.

New C keywords now use a leading underscore (_Atomic, _Bool, _Static_assert, ...) and come with a header file you can include to get a more friendly name (stdbool.h, stdatomic.h, assert.h, ....).

As to not keeping your C code C++ clean: Just use a dedicated C compiler and link the object files.

Then what header files do you compile your C++ with? That's all there was at the time.

If it occurs in public header files or if you use C++ keywords as external symbols, that is indeed a bit... inconsiderate, to say the least.

Header files intended for public comsumption should stick to the intersection of C and C++ in most cases.

"There are only two kinds of languages: the ones people complain about and the ones nobody uses." - Bjarne Stroustrup

>>In the real world, adoption is not only driven by how cool a language is, but the ecosystem of tools and developers around it.

If it gets even half close to where Perl 5 is we can only expect great things from Perl 6.

>>The true lesson learned for language developers is: Don't break backwards compatibility. Ever.

If you look at the whole grammar system and the compiler design of Perl 6 you will see this is precisely what they have done.

>>Guido said this won't happen again.

Not that easy. To compete with newer and powerful tools you have to add features that you need. You can't do that without bloat or breaking things. Again look up the Perl 6 compiler design.

>>Migrating a relatively complex project from 5 to 6 is extraordinarily hard.

This is like saying migrating from C to Java is hard. Of course it is, because 5 and 6 are two different languages.

>>There is no incentive for migration.

There never ever was and should never be, Perl 6 is for new projects. Perl 5 and Perl 6 are two different languages which will have their own evolution path.

>>But Perl 6 while being a nice language, doesn't seem to have any killer feature compared to mainstream languages.

Seriously?? Have you even looked at Perl 6.

> Seriously?? Have you even looked at Perl 6.

Yes I have. Many interesting features. Most of them not good enough to be considered killer, IMO. What are your suggestions?

Here's one I've started to focus on in the last few months:

    say "ปฏิบัติจริง".chars; # says 7 [1]

Perl 6 may be a game changer in years to come among those who need to munge text that's Thai, Chinese, etc.

http://www.reddit.com/r/perl/comments/2szv7m/beginner_progra...

I don't see how this is a language killer feature. It looks more like a library feature. And yes, Perl 6 has a superb unicode library. In other languages, you can add the functionality as libraries. This is an example in Python:

    import unicodedata

    def count_graphemes(string):
        count = 0
        for codepoint in string:
            if not unicodedata.category(codepoint).startswith('M'):
                count += 1
        return count

    print(count_graphemes("ปฏิบัติจริง")) # prints 7

This example is over simplified as grapheme clusters are trickier to detect, but proper support is coming to Python's standard library: http://bugs.python.org/issue18406

(Edited to say "regular folk" instead of just "folk" in the last paragraph to make it clear I'm talking about non-expert coders who would much prefer that stuff like converting mixed case text to all lower or upper case, or making sure some text visually fits in a fixed length field, be a reasonably simple operation, like it already is when handling most Western languages.)

----

Would you agree that you are saying that you think it's OK that:

* The appropriate long term solution is to use library code when doing character or substring level operations on arbitrary Unicode text;

* Language implementors have generally not yet written said library code;

* The built in character and substring operations in mainstream languages unpredictably misinterpret and corrupt international text such as Thai, Chinese, etc.;

* The current solution for correct, non-corrupting character and substring handling of arbitrary Unicode text is to hand build custom code to do things like get the length of a string or replace a substring.

If you disagree with the above, then I'm confused.

If you agree, then you're in agreement with most folk and I think you and they are blind to the needs of regular folk who need to handle, say, Chinese text, and these latter folk will find the simple and correct built in character and substring handling in Perl 6 to be one of its killer features.

I think the author is mischaracterizing all of these examples; none of them are "second systems" but major revisions of software with large install bases and no built-in update mechanism. They are reliant on distributions and third parties to provide any form of automatic update; oftentimes the upgrade path without distribution-level support is painful for an average user.

Certainly, in the case of PHP, it was a very painful process to get major adoption of 5 - especially as so many of the users are reliant on managed hosting. It took a long time for the install base to grow there; PHP 7 will similarly take a long time to get up there. At least the decision to scrap 6 (a decision also taken in light of all of the literature out there describing the features of 6) was a pragmatic one; PHP 7 is now just around the corner.

What the author is describing are two general problems for any software development where there is a large install-base: development cycles and release management. Just look at uptake numbers for operating systems for an example as to why this is absolutely not a problem confined to scripting languages or even OSS software: http://en.wikipedia.org/wiki/Usage_share_of_operating_system...

The author does mention Second Deployment Syndrome. And Perl6 most definitely is a 'second system'.

The transition from PHP3 to PHP4 and from PHP4 to PHP5 wasn't that painful. At the time, most PHP open source projects simply avoided OO concepts due to their slow implementation in PHP4 and to be downwards compatible with PHP3. It solved the transition as they only slowly adopted class based OO later in PHP5 era. Also the transition period of parallel version of PHP4 and 5 was between 2004 and 2008 - long enough so that most shared hosters were able to make the switch. PHP6 failed, and most planned features were backported to PHP5.3+. As several books were published in the months before the planned PHP6 release, the next PHP version will be named PHP7 to avoid any confusion. Nevertheless, PHP7 isn't a bigger change than let's say PHP 5.2 to 5.3. Overall I would say, PHP's developers and the community has done a great job. Modern PHP.net and HHVM are certainly better than their reputation. The same goes for MySQL and MariaDB.

PHP4->PHP5 required the gophp5 campaign in the end (domain now occupied by squatters) where several open source projects (iirc, Drupal, WordPress and phpBB among them) committed to dropping php4 support to pressure shared hosts into upgrading. And that wasn't even a backwards incompatible change (unless you were using things already deprecated and disabled by default at the end of php4, like register_globals)

Imagine if in 2-3 years, SQLAlchemy, Requests and SciPy for example dropped Python 2 which would be a similar Python attempt (not saying they would or should). That's some pretty forceful migration.

One small (afaik) correction: Wordpress was never part of the GoPHP5 initiative.

Wordpress has a terrible history of maintaining backwards compatibility over good design, and modernising their codebase.

They only started to add support for a modern DB driver (mysqli) a bit over a year ago. MySQLi and PDO have been in PHP since v5.0 and v5.1, respectively, that's 11 and 10 years ago.

If anything, my takeaway here is that PHP might have handled things the best of the three. PHP doesn't have the huge schism that Python 3 does, and the development cycle isn't nearly as crazy as Perl. Sometimes pragmatism is more important than idealism.

The process from PHP4 to PHP5 was long but there were clear advantages to PHP5 that it made it logical choice for all new projects. I'm still working with brand new Python 2.7 code.

We don't know that, because PHP 7 isn't out yet, but also PHP 7 doesn't change in breaking ways. All of the language warts that plagued 4 and 5 are still around.

> the development cycle isn't nearly as crazy as Perl

What regular releases every six months? With a standard mechanism for saying you are using a given new feature or expect a given versioned behaviour... yeah so cRaZzzY.

I'd say PHP's major upgrades were a success.

Adoption of PHP5 was a pain, but still it was quicker than adoption of Python 3. At least the community was all behind PHP5 and there was a major push towards it.

I think PHP has dodged the bullet by taking PHP6 apart and reintroducing features piecemeal in PHP 5.3+.

> Well, after a little over six years of development, we discovered that we were never going to see a PHP 6 at all. Having seen how long Perl 6 had taken, and how long PHP 6 was taking, the number 6 is associated with failure. So they cancelled PHP 6 and voted to change the name to PHP 7.

Uh, what? Did the author do the most basic of research?

PHP 6 was a real thing, a development branch. But it dragged on for years.

Due to delays, they made an interim release in the 5.x series, PHP 5.3. It backported some stuff from 6.

Eventually, PHP 6 was killed off because it just wasn't going to work. So they made another 5.x release, PHP 5.4, backporting more stuff from 6, and adding some other features.

After that, with PHP 6 dead and gone, work focussed on the PHP 5.x series again. So out came PHP 5.5 (2013) and PHP 5.6 (2014).

During this time, some people did some experiments and realised it was possible, by breaking internal (not userland) compatibility, to greatly improve PHP's performance. This project became phpng. Since it would break so many (internal) things, they decided that if it should be merged into PHP, it needed to go into a new major version.

It was after this that the ridiculous version number vote was held. I don't see why we ever had that debate, really. PHP 6 was a thing and was killed off, re-using the version number was silly. Anyway, the number chosen was PHP 7.

Fast forward to the present day: PHP 7 has gone into feature freeze and should be released in a few months. Alongside the performance improvements of phpng, it has some other new features developed independently, and includes a few deliberate backwards-compatibility breaks.

tl;dr: the time line goes: 5.2 release, work begins on 6.0, 5.3 release, 6.0 killed off, 5.4 release, 5.5 release, 5.6 release, 7.0 in development.

Edit:

> JIT to the Zend engine

...what? No, PHP 7 does not have that.

> Asynchronous IO and functions

...PHP 7 doesn't have that either?

> Standalone Multi-threading Web Server (HHVM)

...huh?

> phpclasses.org

Oh, so that's why the author is so confused.

PHP haters are notorious for using blatantly incorrect or horribly outdated information.

Correct PHP 7.0 changes: https://wiki.php.net/rfc#php_70

The problem with this is since they despise PHP they don't even bother to learn the correct information about it. Which means what they despise is entirely the product of their own imagination, and for anybody informed it sounds completely laughable. Unfortunately, they are not getting called out on it as frequently as they should be and the cycle of misinformation continues.

> Oh, so that's why the author is so confused.

So for anyone wondering about this, phpclasses was the answer to PEAR, which itself was trying and failing to be CPAN. PEAR ended up being laughable and underwhelming, while phpclasses ended up being a breeding ground for some of the worst PHP code on the planet.

Nobody should be using it and anyone pointing to anything on it as a good example should be laughed at.

Meanwhile, the article author probably needs to read this slide deck from the PHP6 team explaining exactly where the breakdown occurred:

http://www.slideshare.net/andreizm/the-good-the-bad-and-the-...

The postmortem starts on slide 60, but the rest is worth reading.

It had nothing to do with second system syndrome at all.

Sounds like a disaster. Also funny how anyone discussing PHP's flaws gets met with "oh THERE'S your problem, you decided to use this terrible awful piece of shit thing instead of not doing that". Well why in the hell was that an option in the first place?!

The golf course is full of landmines, and if you bitch about how shitty it is to step on one, nobody goes "yeah it sucks how this course is full of landmines", you're just an idiot because you stepped on one, like an idiot.

So you're suggesting that the PHP community at large, is somehow responsible for a single site, that gives bad advice, has bad information, etc.

What exactly is your solution? a SWAT raid to take down this travesty of technology?

There's nothing that will convince the Python 3 haters. So silly. I use nothing but Python 3 and I rarely if ever find I can't move ahead for lack of a library. The Python2 only libraries are just being replaced with more up to date alternatives. This article is total FUD and garbage from yet another Python 2 ideological holdout dogmatist.

This article would have you believe that if you tried to use Python 3 that within hours you would hit a library roadblock that would be a showstopper. Completely wrong, and you have to ask why the author wants to create this perception.

Python 3 works beautifully and is a pleasure to program.

If you WANT to hate Python 3 then you'll find no lack of ways to hate on it for the next 20 years (HEY! I found some package built by a fellow Python2 holdout! See, it simply is not possible to write the code to upgrade it! PYTHON 3 IS FINISHED! THE SKY IS FALLING!) etc etc ...

If you want to succeed with Python 3 then that's easily done, you just get on with it.

No one using Python 3? Flat out lie. Prove me wrong by scientifically explaining your assertion that no one uses Python 3. Lies, spoken to back the wishes of the Python2 holdouts to sabotage Python 3 - they hate Python 3 and will say any lie to bring it down and spread the FUD. I'm wrong? Prove it in detail with science. Alternatively make vague, handwavey statements about downloads, as though that defines how many people are writing code today with a language.

I've found exactly the same thing.

As someone writing applications using Python 3, I found the switch to be easy and I haven't had any problems with libraries. I've switched from some of the old ones I was using to more modern alternatives that are being actively developed.

It annoys me that there are people on Python 2 that haven't really tried to make the switch and stand on the sideline shouting insults at Python 3. It's a tremendously self destructive thing to do for all involved.

To anyone looking to get started with Python, there's no real reason to start with Python 2 these days.

Libraries are a problem - it's difficult to switch unless all your dependencies are Python 3 compatible. The only ways to fix it are to either use an alternative, or perhaps to fork or contribute patches to the project - both of which take up valuable time when your code is still functional under 2.7.

I'm currently creating 2.7/3.3+ compatible code in whatever I create, but I'm still deploying to 2.7 hosts. For most of my projects this is due to just one package which has some Python 3 related bugs, so hopefully I can just flick the switch soon and use 3.4.

It's only easy when your project depends on a few popular packages (eg. the top 200 where 86% are currently Python 3 compatible [1]), it's much more difficult if you've got a long list of dependencies.

[1] https://python3wos.appspot.com/

Yep, for me, distributions are the biggest thing holding things back right now, because the existing ones are out there, and will be deployed for a long time.

If deploying a .com website, you can control your dependencies completely, and it's much easier than if you are deploying a system that must interact with existing systems the user/customer might have (for instance, management or monitoring apps).

Thankfully that's not a huuuge class of systems, but it's what I typically work with :)

I suspect most people don't have to deal with that issue (fortunately, because that must suck). Is there not some way of wrapping the entire binary with your distribution? If I was working in that space I think I'd probably choose a different language, if possible.

As you say, it's not a problem for most of us as we get to dictate the platform.

If you're wondering how you can do above / why it's not such a big deal: Python 2.7 has everything you need to write forward compatible code. It gives you all the __future__ dunders, the only extra thing you need is a compat module [1]:

See kaleidoscope of examples:

- https://www.google.com/search?q=site%3Agithub.com+compat.py+...

- https://github.com/search?utf8=✓&q=extension%3Apy+"if+PY2"&t...

[1]: http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/

I'll get downvotes for this, but my reason for sticking with python2 isn't library support, it's that I prefer python2's strings. Sure, it made unicode handling less painful in cases where encodings are always known and everything should be in unicode, but it makes things much harder for people working with bytes interfaces that want to treat all non-ascii bytes agnostically.

But Python 2 strings are bytes. So in Python 3 just use bytes everywhere. Or am I misunderstanding something?

I was scared off of Python3 until recently for no good reason. When I started a new development effort I made the switch and have had almost no problems besides a few small configs I had to work through. Besides that and going over some new syntax and recommended techniques I spent maybe 1-2 days all in. It's seriously not that big of a deal.

Did you read a different article? The author only had positive things to say about python3 except its adoption (which has been noted to be an issue multiple times before this article existed). You are blowing the author's intent way out of proportion.

Which article did you read - I don't see any of the claims you are suggesting they made. I even did a search for the word, "library" - it never appears.

If anything, based on my read of the article (twice, I had to read it a second time after reading your comment) - I think it's actually written by a Perl advocate who is ruminating on the future of Perl 6, and just using Python as an example of a team that shipped code fast and limited the amount of stuff they broke.

Are you sure we're reading the same article?

The problem isn't that there are problems with going to Python 3. The problem is there isn't enough benefit to make any change worthwhile for most users.

Some current facts: ~17% of packages on PyPI support Python 3 and ~5% of packages downloaded from PyPI are downloaded for Python 3.

If you start a project today then yes, start with Python 3. But if you have a Python 2 codebase you're likely to have (1) one or more dependencies that haven't been ported yet, (2) no solid economical reasons to convince your employer to switch or (3) no sufficient technical reasons in switching at this point (which may change as more Python 3.x versions are released).

>~17% of packages on PyPI support Python 3

And how is that determined? Setuptools/distutils classifiers? Then is completely meaningless, most don't even bother including those.

And a big chunk of the packages will never be Python 3 compatible simply because they're abandoned and shouldn't be counted either in my opinion.

You're referring to packages that most people will never ever use, or have heard of.

What you should be referring to is http://py3readiness.org/ — which shows the opposite picture. Above 80% of the most popular packages are available on Python 3.

I have my problems with the Python 3 implementation. Unicode is great, but standard library APIs should understand each other without the need to call decode.

I like the way the change was rolled out, though:

1. Find a defined amount of cruft that needs to be fixed backwards incompatibly, so you do not take 15 years for it or fail completely. Release a new major version.

2. Make one or two minor releases to the older version to ease the upgrade.

3. Then STOP developing the older version, work with the community and perfect the new version to get everyone on board.

I think it is the perfect middle ground. If you do this constantly you will not have the need to change everything at once and will also not amass cruft.

The problem with the justification of the author for 15 years of Perl 6 development is, that Perl 6 will have issues, too. Are you going to change them all at once again and take another 15 years for Perl 7?

> Unicode is great, but standard library APIs should understand each other without the need to call decode.

If one "standard library API" generates integers and an other consumes strings, would you expect the second one to take the output of the first one without conversion?

No, I would not. To stay with your problematic analogy that differently encoded strings are to each other what integers are to strings:

Would you write a function, that takes either a string or an integer and returns therefor either a string or an integer, so you can not feed the output of this first function into a second function, that is often called after the first, but can only take integers?

Or would you rather say "Okay, we are working with numbers here, lets just take and return integers in this module."?

You can read more about this here:

http://lucumr.pocoo.org/2014/1/5/unicode-in-2-and-3/

Learning from experience, one of the design goals of Perl6 was the ability gracefully evolve. Thus you get an object system with meta-object protcol, and you can extend the syntax through user-defined grammars. This probably also factored into the decision to have Rakudo support multiple backends.

This post made me move to Python 3. I replaced all print statements, in the code I am currently working on, with print functions, and yum-installed python3 versions of the libraries.

The code ran correctly on first try. Guess I was lucky, should have made the shift earlier!

Does your code use division, and do you have unit tests for at least those parts? Because the same code will compile and run in either version, but you'll get different results, depending.

Thanks for the heads up. Fortunately I was aware of the kinks with division, so it caused no trouble.

Just be aware of the unicode differences between the two versions and you should be fine with Python 3.

This is ridiculous. The author makes it sound like perl6 is more of a success than python3 -- despite python3 being used for real projects for years now, and perl6 being "slow and about to have its first release" (just like it was for the last, I don't know, 7 years?). I mean, come on.

IMHO, the practical best-case scenario for Perl 6 at this point is a Lisp-like status where the language itself does not see much commercial use, but ideas from it fuel innovative attempts in existing and new languages. That would be the best-case though, and it might just bomb and be forgotten within months, which in fact seems to be the common expectation from many.

In any case, this was one of the most unabashedly biased articles I've seen on HN, with badly made attempts at revisionistic history. Perl is still my favourite scripting language, but even I have to admit that Perl 6 has little hope of adoption, and Perl 5 is dying a slow and painful death: due to the momentum, clarity and leadership and P6 took away from it. A few ported-back features gained can't possibly make up for the lack of a cohesive vision that resulted from nobody being sure of the status of the Perl 5 language. Subroutine arguments have long languished in interminable bikeshedding due to this (despite actually having a few enthusiastic implementors - a rare phenomenon in P5 world now!), and we have embarrassments like autoderef and smartmatch[1] which reduce people's trust in the language even more.

What makes the situation truly infuriating though, is the snarky and superior attitudes taken by most P6 folk, just like this article exhibits. Even if you begin with an attitude of collaboration, it soon begins to feel like you're talking to King Joffrey, with that ever-condescending smile of his.

[1] http://www.perlmonks.org/?node_id=961344

> it might just bomb and be forgotten within months

That seems unlikely for any project with over a decade of work behind it.

> What makes the situation truly infuriating though, is the snarky and superior attitudes taken by most P6 folk, just like this article exhibits. Even if you begin with an attitude of collaboration, it soon begins to feel like you're talking to King Joffrey, with that ever-condescending smile of his.

Do you have any examples, because I've dropped in their IRC channel occasionally over the years, and it's been nothing but welcoming. The attitude at YAPC (at least in 2013 when I went) was very positive from everyone I heard, from both P5 and P6.

So what is different about the Ruby approach to language evolution? The community has moved forward relatively painlessly from 1.8.x to 1.9.x to 2.x, all of which had breaking changes. Why has change been such an issue for e.g. Python?

Strings are, to a first approximation, the universal interface among software components. Lots of source code relies on strings in non-obvious non-trivial ways. Breaking string compatibility is a big deal. That's why C hasn't and C++ didn't despite the pit of hurt that null terminated strings turned out to be. Instead, Unicode abstracts over stings.

One could make a plausible argument that by opting for C-style strings, Python didn't get strings right. In fairness, Unicode was not mainstream when Python was on the drawing board and C-interop was and continues to be a way of achieving performance from interpreted languages. Anyway, by the time the problem was addressed, there was lots of Python in the wild. Then as now, most Python programs didn't need Unicode; and as is the case today, much of that code was written without thinking about the implementation details of strings.

Even the most non-benevolent dictator can change the economics of legacy code bases by fiat. Legacy code has inertia, and in the case of working Python 2 code it's Python 2 that's broken not the working code. Switching non-trivial code to Python 3 comes with the risk of subtle highly coupled bugs in exchange for little more than an 'atta-boy or two. There isn't a meaningful performance improvement at the end of a refactoring hell.

In the end, the business case for changing from Python 2 to Python 3 is weak for many users. Python 2 works fine for their needs. Python 3 may just generate more work: even setting up, running and managing two versions of Python is work [and for non-devops people, often non-trivial].

I think it's largely because ruby use is dominated by a single, pure ruby application domain: rails. Once rails supports the new version, it's easy to move. google "ruby programming" 3.5M hits and "ruby programming rails" 1.6M hits.

In contrast, python has many different application domains, each with their own dependencies that need to be updated, and native code extensions are much more common. "python programming" 38M hits, no single application is a significant percentage.

We are not a rails shop but moving to 1.9 was almost entirely painless. I think that is because Ruby never did any release which broke everything. 1.9 did not break that much.

In addition to brudgers point about strings, which might very well be true, in Ruby it was never an option to stay on the old version. The mindset of a really stable Ruby version does not exist. If someone considers Ruby, he is fast made aware of the fact that it will be necessary to adapt the code to the new ruby version, probably. But since that is true for everyone, all the frameworks and gems do the same anyway, which makes it easier.

But the breaks were not that big. And the new versions always had really cool features, so updating was really nice.

It might be a mindset thing of Ruby developers.

It's not just versioning that's "unstable". Inspired by Smalltalk, Ruby allows developers to make (nearly?) any object respond to any message by extending the language. A pining-for-C-Rubyist can change String to add characters and nobody who doesn't have to live with the consequences need ever know or care.

> The community has moved forward relatively painlessly from 1.8.x to 1.9.x to 2.x, all of which had breaking changes. Why has change been such an issue for e.g. Python?

1.8.x to 1.9.x was a pretty traumatic break, not that dissimilar to Python 2.x to Python 3.x; Ruby got over it quicker because there was less heavily-used-but-basically-maintenance-mode Ruby software around to be broken by the breaking changes (that Ruby hadn't had nearly as long of a stable period without breaking changes as Python 2.x probably contributes to that.)

The Ruby 1.8.x to 1.9.x conversion started sooner, too, and I think lots of people are viewing the past through rose-colored glasses on that. A lot of the discussions you still see with Python 2.x vs. Python 3.x -- including discussion of a desire for a long-term maintained fork of 1.8.x -- happened then. People just forget because Ruby got past it a while ago.

I wonder if Typescript will serve as a lesson to us all. It feels like the next generation of JavaScript, and (can) compile to ES(really old) without adding too much ugly Dart-like cruft.

Another interesting subject only slightly touched on here has been the degree to which Perl 6 features have been slowly rolled in to Perl 5; maybe Perl 5.30 will look a great deal like Perl 6.

Javascript seems destined to eventually dominate the "scripting" language space. It just has unstoppable momentum now. The pace of evolution of the language and the implementations puts Perl, Python, PHP and Ruby in the shadows. And I say that as someone that really likes Ruby and is happy enough writing Python.

Typescript may serve as another showcase for Microsoft tactics to influence and steer a language (JavaScript/ES6 & 7). http://en.wikipedia.org/wiki/Embrace,_extend_and_extinguish You know the Java vs. MS Java (later transformed to C#) - both MS Java/J++ and C# as well as Typescript were guided by Anders Hejlsberg (former Turbo Pascal fame).

I try to be as cautious as any free software loving programmer when it comes to Microsoft, but it's hard to see how they could get away with that strategy in TypeScript's case. The code uses the Apache License 2.0 (FSF approved, OSI approved and compatible with GPL v3) and it's been on github for a year with lots of forks.

You don't need to depend on anything from Microsoft for development, I've toyed with TypeScript using Node.js and Eclipse on Linux without issues. And as stated before TypeScript compiles directly to Javascript and they are trying to align their syntax with ES6.

Seems quite kosher. I think at this point if Microsoft tried something shady a community fork is very doable, or if it doesn't happen you can always compile to plain JS and continue working there.

It's really different from the situation you describe because developers and users won't tolerate a MS JS that only works with MS browsers, they have to remain compatible with the others or risk irrelevance.

I can run Fortran IV code on a modern compiler without change. 2015-1966=48 years. And counting. Ada95 was a very easy transition from Ada83. Pay attention to users ! Backwards compatibility is MANDATORY for a language.

Give me aggressive transition with breaking changes (and a major version), but the one that genuinely improves the language over attempts to maintain compatibility that lead to a stagnant language any day.

Well, that explains why Fortran and Ada are such wildly popular languages today!

In their communities, they are.

Parts of the scientific community still insist on Fortran, and the DOD still loves Ada.

Please consider the time it takes for a new language to be fully developed and to take hold. These major revisions are simply new languages, because they _break backward compatibility_.

Except, um... PHP.

Yes, and in the mean time: Go, Rust, Dart, Julia... maybe someone someday will get the sweet spot of performant code that is just plain fun to write. I guess that was the idea with PyPy, but after 15 (?) years that goal is still a ways away.

With regard to Python 3's print statement, I don't understand it. Or rather, I don't understand why they changed only the print statement. If nothing else, raise and assert should have had the same treatment.

And Python 3's unicode handling isn't sane. Or rather, it is saner internally, but not sane on the interface. I mean, recreating `cat` is bad enough. Python 3 made unicode handling slightly easier and byte handling gratuitously harder.

But this is probably bikeshedding.

Personally, the bigger problem is not creating a relatively frictionless upgrade path. If you can't run things on the new version without manual editing, you won't move over unless everything you depends on moves over. Catch-22. Python is slowly moving over, largely because a) it's not generally too difficult to port the code (and largely automated to boot), and b) you can relatively easily write code that works on both versions.

My main problem with Perl 6 is that it seems overly large and complex. It is very different from Perl 5, there is effectively no up to date tutorial, and I'm overwhelmed by Perl 6's apparent complexity.

> My main problem with Perl 6 is that it seems overly large and complex. It is very different from Perl 5

Perl 5 is also overly large and complex (with most features getting used only rarely and some being broken/having half-assed implementations), only not as scary because it doesn't get presented to the newbie with its radically new features first.

For example, did you know Perl 5 has a UNITCHECK compile phase keyword? Almost noone uses it.

Did you know that you can execute code inside a regular expression with (?{ CODE })? It's very useful but too bad that using other regular expressions in the code you use/call from there will break things silently.

It's such things that a) made the implementation of Perl 5 already overly complex (and progressing slowly) and b) made things harder for the Perl ecosystem to fully support the language (even parsing Perl to analyze it, like Perl::Critic does, is nearly impossible).

Instead of addressing these issues and slimming down the Perl 5 spec to a useful, easier to maintain, optimize and learn core functionality (and perhaps giving it a name that doesn't imply backwards compatibility), they built another large, incompatible mess. We'll see how well Perl 6 does out in the wild.

Perl 6 isn't as complex as people make it out to be. I have friends who are race car enthusiasts who could go in to excruciating detail about cars and if you don't know how to drive, it could seem daunting. The reality is that most of us get in to our car and just drive.

That's part of the issue with Perl 6: its proponents get very excited about it and write about all the really cool stuff you can do with it, but for those who don't know the language, it sounds intimidating.

Here's a talk I gave at FOSDEM explaining how day-to-day Perl 6 is fairly straightforward and easier than people think: https://www.youtube.com/watch?v=lpu-3UF_b48

> The reality is that most of us get in to our car and just drive.

I don't mean that the internals appear complicated (I wouldn't know anyway), I mean just learning it as a user appears that way.

> That's part of the issue with Perl 6: its proponents get very excited about it and write about all the really cool stuff you can do with it, but for those who don't know the language, it sounds intimidating.

Yes. But it's not just that; trying to read the Synopses is daunting. It's very easy to get stuck there, try for a while on IRC, then give up.

> Here's a talk I gave at FOSDEM ...

Thanks for the link, and for the talk.

The Synopses are intended for compiler writers.

The writing intended for users is at http://doc.perl6.org.

Unfortunately, even the docs at doc.perl6.org give me the impression that either the language is extremely complicated, or that the docs leave out too many essential details, or both. I don't wish to pick apart docs here though.

I think that what Perl 6 needs if it is to have a chance to succeed is a really good comprehensive and comprehensible tutorial.

Right. Imo Perl 6 ought to be fairly easy to learn -- the basics are simpler than Perl 5 for example -- but several parts, of which doc is one, are still weak enough that this simplicity is currently obscured.

If you're interested, I'd be willing to try to personally tutor you (for free of course). I'm raiphmellor at gmail if you want a private email exchange.

Perl, Python, and PHP: or why PHP just sucks because I can't figure out its benefits and Python 3 sucks because change and Perl sucks because Perl. Every language solves some problems well and some not so well. Why does everyone (figuratively) bash other languages simply because they aren't quite right for your particular use case??

Because we're instinctively tribal. See: friends don't let friends drive Fords.

This was a fascinating article to me, a python2 user.

I suspect there are three large subsets of readers, where the process described for "their" language is familiar, and those for the other two languages sound like tales of "inside baseball."

I am familiar with the progress and arguments related to python2/3. I am aware, from a distance, of similar issues with perl5/6 and php5/6/7.

It occurs to me that I haven't heard about similar breaking shifts for other currently active languages, including C and C++. I'm dimly aware that Java recently moved to 8(?) and 9(?) is starting(?). And I wonder about Fortran, which I'm dimly aware still has active users and some standards process.

Can anyone who is familiar with these languages' progress characterize them as breaking and angst-ridden, or smooth and relatively trouble free, and say why? What's different about the process and community from the three languages in the article?

For a possible initial point of contrast, my impression of the three languages' changes are that they are all but new languages that are easy to read for anyone familiar with previous versions.

C, C++ and Java generally don't break backwards compatibility in new language versions (or at least not as much as other languages), so upgrading to the next version is usually easier.

Sometimes I have the feeling that Scala is the "Second System" of Java. But it wasn't designed by the same people who did Java.

Scala is Java done even wronger; C# is Java done right.

Oh right! I guess that explains why C# wants to add all the features Scala shipped with since 5 years ago.

> Oh right! I guess that explains why C# wants to add all the features Scala shipped with since 5 years ago.

It ships them in a usable state, documented in an accessible way without patronizing language and useless neologisms.

Lol. No, it doesn't ship them at all yet. Why are you acting so butt-hurt?

I would argue that Groovy is Scala done right.

I guess that's why they just lost their commercial sponsors recently.

Popularity != quality.

Or would you argue that Honey Boo Boo was the best show on television?

No, but if even technically illiterate managers lose their hope in a product that says something.

Groovy devs have been unable to keep up with cleanups, improvements and requirements since years. There is just no point in using the language. Everything it could do is done better elsewhere.

> I would argue that Groovy is Scala done right

The creator of Groovy, James Strachan, said 6 yrs afterwards that if he'd known about Scala at the time, he'd never have created Groovy, so he's arguing the exact opposite viewpoint.

> Popularity != quality

The recent blog by Jochen Theodorou, the tech talent that practically built and maintained Groovy for 10 yrs, at http://blackdragsview.blogspot.com/2015/04/about-being-paid-... says about Groovy's quality...

> G2One management adding feature after feature without testing or documentation

...and identifies that G2One (SpringSource, Pivotal) management as...

> all of the paid Groovy guys... Cedric (developer), Guillaume (manager and evangelist) and me (developer) had to leave Pivotal

Groovy has two use cases:

* scripting Java (tests, Gradle scripts)

* providing a M.O.P. for Grails

If you want to actually build a system on the JVM or Android, use Java or Scala or some language created specifically for that purpose. Groovy was built as a dynamically typed language, with static typing features shunted on much later, and hardly anyone uses them. Groovy's for quick and dirty scripts, like those 30-line Gradle build scripts (unless you need readable stack traces then you're out of luck).

Okay, wrong how.

I think that Brooks' Second System Syndrome assumes that there is actually a compelling reason for the second system. A language is more subtle in that the value there is mostly in the convention of how to do things. In any case where you make a big change to a language there are always going to be people who either don't agree with the change or simply don't think the change is worth the bother.

There are zillions of proposals for new languages around at any particular moment in time. Very few ever become popular. If you significantly change a currently popular language then there is a good chance that you will fall off the side into unpopularity.

Except Perl 6 isn't a "Second System", but rather a "Third System"; Perl 4 (for whatever god-forsaken reason) is to this day still mentioned in the Perl documentation in various places, despite being about 2 1/2 decades old and having been effectively abandoned in favor of Perl 5 for nearly as long. Perl 4 and Perl 5 are pretty much entirely different languages, and Perl 6 is just an extension of that heritage.

The 5->6 transition might be more painful because of how long Perl 5 has been around and in active use, but this isn't exactly new to a whole lot of veteran Perl programmers.

What does Java do differently that it doesn't seem to have adoption problems?

90% of Java 1 code works on Java 8.

There are still enterprise apps stuck on Java 4, 5 and 6.

More importantly, bytecode produced by old Java compilers still runs on new VMs.

I believe that if Python 3 had been able to call into Python 2 modules (even by embedding CPython2) adoption would've been better.

I think byte code compatibility is gone now, its supports the current and previous versions only.

I worked on one such app that went to production a month ago. Java 6. :'(

It doesn't break compatibility. (Almost) ever. Meaning that (almost) all the code that used to compile on Java 1, or binary code compiled with the JDK 1.0, still works with the JDK 8.

(It's the main reason why generics in Java suck, btw: extreme backward compatiblity.)

Unfortunately, most of this post is just content-free snark, pissing on certain (phases of) languages with no effort whatsoever to provide any insight beyond the author's prejudice.

Apparently the "gratuitous negativity" guideline doesn't apply to content.

Edit: And let's not forget that 99% of the criticism levelled at those languages has nothing at all to do with Second System Syndrome. That very valid subject is apparently just an excuse to go on a rant.

He's attempting a general comparison, concluding that each language took a different gamble and he looks forward to seeing how it turns out for all three.

He doesn't piss on anybody. Honestly, to me your comment is more "gratuitous negativity" than the blog post is.

Too sad to see so many efforts on Python 3 although nobody is using it...

I've made the switch on the code I write at work from Python 2 to Python 3, and it was far less of a hassle than I feared. Unless you're using a package that doesn't work on Python 3 (which isn't the case for me; NumPy, SciPy, pandas, scikit-learn, etc, all work fine on 3) or worrying a lot about Unicode, it isn't that difficult (in my experience) to write code that is compatible with both 2 and 3.

I try to use it on new projects. I don't have any reason not to.

I really hope Perl 6 gets some adoption, it sounds like a great language.

One data point: the smartcard library https://pypi.python.org/pypi/pyscard is still not Python 3 compatible. Six and a half years and counting.

    Downloads (All Versions):
    0 downloads in the last day
    0 downloads in the last week
    0 downloads in the last month

That's what I saw, plus the description mentions Windows 2000, but if you go to the SourceForge page:

    206 Downloads (This Week)
    Last Update: 2014-12-13

It's probably time for a major rewrite anyway, seeing how Windows Vista introduced a new Crypto API and this library claims support only for 2000 and XP.

Is that multiplatform? Because PCSC has a multiplatform version in PCSC Lite http://pcsclite.alioth.debian.org/ and that's what that Python library wraps. I am unaware of any smartcard implementation for Linux.

Well, you shouldn't believe everything you read.

I imagine this would be a similar problem if C++ tried to get rid of all the crap that's been layered onto it over the last decade.

MySQL also started version 6.0 and then stopped developing it. They also tried to fix the unicode support (utf8mb4 by default).

Reply on news.ycombinator.com