RPerl – optimizing compiler for Perl 5 | Hacker News Reader

RPerl – optimizing compiler for Perl 5

github.com

93 pointsbane24 comments

Are there any benchmarks to be posted beyond this tidbit I found in the FAQ?

"A: Early tests show the potential to run RPerl apps about 7x faster than normal Perl 5 when using Perl data structures, and up to 170x-350x faster when using C++ data structures. This does not include any fancy optimization techniques which could add even more speed in the future."

On the damper side, I'd caution against too much "fast" (usually in the "as C" context) excitement. There have now been many different efforts to speed up the 1990s-dynamic-scripting languages now, most notably Javascript but they all have fairly similar issues when running at speed, and as near as I can tell, on general code all of them have plateaued at roughly 6-10x slower than C. "Just compile it as C++" has been done.

On the "get excited" side, it's about time somebody may have some success at getting Perl to this performance level. Perl's really been left out in the cold by its fellow 1990s-dynamic-scripting brethren lately, between PyPy, all the JS efforts, all the various PHP engines and Hack.... It and Ruby have been left out of all the action, and there's a ton of code out there that could use the improvement.

They have a bubble sort benchmark: http://rperl.org/performance_benchmarks.html .

""" The input to the RPerl compiler is low-magic Perl 5 source code. RPerl converts the low-magic Perl 5 source code into C++ source code using Perl and/or C++ data structures. Inline::CPP converts the C++ source code into XS source code. Perl's XS tools and a standard C++ compiler convert the XS source code into machine-readable binary code, which can be directly linked back into normal high-magic Perl 5 source code. """

That is some magic right there. Similar kinds of craziness (like the Inline module itself) is some of what gives Perl so much charm :)

Glad to see some new cool stuff about Perl on Hacker News. The community is still very active, and a lot of very smart people are doing quite interesting things with it.

A couple questions:

* The definition of "low-magic" (v. - presumably - "high-magic") seems to be unclear, even with the rules specified in the "Low-Magic Perl Commandments". It's also unclear if that's a hard restriction on what Perl code RPerl is able to understand or if it's a strong recommendation for performance/safety/security/cleanliness/sanity/etc. Are there any code samples floating around re: what a sample RPerl-friendly Perl program would look like?

* The FAQ states that RPerl is released under the same terms as Perl 5.22.0, which is subject to the GPL and the Artistic License. However, the FAQ states that selling binaries without accompanying source code is forbidden, yet doing so seems to be allowed by Section 4 of the Artistic License (particularly clauses 'a' and 'c'). Is this intentional? Did the RPerl folks mean for RPerl to be GPL-only? Am I just misinterpreting the text of the Artistic License?

re low-magic/optimizable:

  * The data must be all typed, 
  * there needs to exist a C++ mapping for this type, 
  * and there need to exist a C++ mapping for these typed ops.

Since rperl does this transparently it's a bit hard to tell which parts get optimized and which not. I hope we can create better tools or docs to see it.

re LICENSE: This additional selling clause is a violation of the ARTISTIC license, yes. But you can just fork rperl, delete this selling clause, and you should be fine, since rperl itself is under the dual perl license.

Larry Wall said at a Cluj.pm meeting earlier this year: "most of the Perl modules are released under 'the same license as Perl', but Perl itself is not bound under any kind of license". So, I don't think that RPerl is more restricted if it states 'released under same license as Perl'

Does he mean Perl (the language) or perl (the interpreter and - I'd imagine - standard library) in that remark? I know he tends to maintain distinct definitions of the two (e.g. when it's said (paraphrasing) that "Only perl can understand Perl").

Greetings, fellow Perl-loving netizens! I'm Will Braswell, creator of RPerl, and I'm here to answer any questions you may have about our new optimizing compiler. To start out with: yes, RPerl is real (despite our unfortunately-scant current documentation); yes, RPerl has been released and is installable from CPAN; and yes, RPerl is a high-performance optimizing compiler. (Reini Urban and I are good friends and programming partners, so even though we may disagree about the nuances of the term "optimizing compiler", we are only arguing about semantics, both literally and figuratively.) You will have to pardon my slow replies, I am currently at Scout Camp and have little-to-no Internet access. So, load up your questions and FIRE AWAY! :-)

Unfortunately, HN's poor integrated usability WRT comment notification will make this unlikely to ever be seen by the vast majority of those who viewed this post.

I only found it because I do a HN search for Perl stories and comments ordered by date every few days, to see if anything interesting was mentioned. But, you have an account now, so next time this drops on HN, you should jump on ASAP. People here love getting specific in-depth answers for their sometimes very involved questions. ;)

Is this something like http://cython.org/ for Python?

It's similar. The difference is that Cython is a super set of Python, while RPerl is subset of Perl. With Cython you can take fully dynamic Python and slowly convert it into some static equivalent (cdef) that runs as fast as C / C++. It also has some additional helpers for interacting with existing C/C++ code in an almost transparent way.

Interesting. As someone who works with biologists this might rekindle some interest in perl in bioinformatics which seems to be waning.

The "R" in the name is unfortunate as it might confuse a population that thinks of "R" as a language.

The "r" stands for restricted perl, the real name. Like in rpython.

The author is a bit confused what a optimizing compiler is. You have to forgive him. rperl is a restricted perl which maps typed data via Inline::CPP to C++, and if all ops and data in a function are properly typed and not-dynamic ("no magic"), then the function can be compiled to C++ and executed there. Thus the 200x performance win for certain static and typed parts.

And there exist "optimizing compilers" for perl 5, rperl is more a transparent perl - C++ mapping where gcc does the optimizations.

Given all of the restrictions needed for good performance it's hard to see where I would use this instead of simply rewriting my program in C++.

Pretty amazing work to put this all together in perl! My only worry is commandment #45 as a lot of the perl I've seen is mostly regular expressions.

Thou Shalt Not: 45. Use Regular Expressions, Smart Match, q Quoting Mechanisms, Unicode/UTF-8, Or String Interpolation

That sounds like stuff that isn't implemented yet, not impediments to speed. Quoting and interpolation are mostly compile-time issues. Smart matching is on its way out anyway...

And the performance of Perl's regular expressions is rarely something that slows down actual code. Using it in the first place (instead of more specific parsing) might be an item, though...

Yeah, what is Perl - Regex? Couldn't they manage to compile regex to pcre?

Yes. perl5 has the possibility to switch to better and faster regex engines, like russ cox' RE2 or pcre with jit support. It's just not done in rperl.

In my perl variant I'm planning to improve the inefficient and overly simple and troublesome Spencer two-pass compiler with backtracking support to use the faster Thompson NFA without backtracking (RE2), and/or switch to pcrejit when no pcre-incompatible ops are used, like unicode character classes. The first pass which calculates the space can be easily used for that.

But a better regex compiler should get away with one-pass and extend the ops dynamically.

Strangely enough I find myself increasingly curious about this. I've got all sorts of perl utility code sitting around that might gain from it.

Sweet, will it be added to CPAN?

According to http://rperl.org/ that's already done.

It's right here: http://search.cpan.org/~wbraswell/RPerl-1.000001/script/rper...

There is no documentation whatsoever.

Reply on news.ycombinator.com