Data munging in Perl 6 vs. Perl 5
perl6advent.wordpress.com
perl6advent.wordpress.com
Perl 6:
say " {.key}: {+.value} student{"s" if .value != 1}"
for %grade.classify(*.value.comb[0]).sort(*.key);
Ruby: grades.group_by { |name, grade| grade[0] }
.sort
.each do |grade, names|
puts " #{grade}: #{names.length} student#{'s' if names.length != 1}"
end
Or if it had to be as 'functional' as the Perl.. puts grades.group_by { |name, grade| grade[0] }
.map { |grade, names| " #{grade}: #{names.length} student#{'s' if names.length != 1}" }
.sort.join("\n")
Both can be improved, of course! :-) I'd be keen to see a Python version actually because I know so few of its idioms but it often does well with these sorts of things as well.. @globvals = glob("*"); # returns all matches to @globvals
$glob1 = glob("*"); # returns the first match and stores it in $glob1
$glob2 = glob("*.c"); # returns the second match of "*" and stores it in $glob2 @globs = glob("*") # all matches
$glob1 = glob("*") # the first match of "*"
$glob2 = glob("*.c") # the first match of "*.c"If you want the first element of the list, you should say exactly that to the compiler, i.e. force list context and take the first element:
($file1) = glob("*");I assume because you misread the example given there. If you note that $glob1 and $glob2 come from what should be different lists (all files in working dir and all files ending in .c in the working dir), then $glob1 and $glob2 should contain the first item each of their respective lists. That's exactly what the documentation says should happen.
> You would expect what it says in documentation, and it says it should iterate over the list.
Unfortunately it doesn't even do what it says in the documentation consistently. I have another comment here that outlines that fairly thoroughly. The behavior is very weird and specific to glob, and is not documented accurately.
[1]: http://modernperlbooks.com/books/modern_perl_2014/01-perl-philosophy.html#Q29udGV4dA
[2]: This very accessible, and free, book (updated very recently for 2014) is a must-read for any Perl developer.Not true.
$ touch 1 2 3 4.c 5.c 6.c 7 8
$ ls
1 2 3 4.c 5.c 6.c 7 8
$ perl -E '
@globvals = glob("*");
say "globvals = @globvals";
$glob1 = glob("*");
$glob2 = glob("*.c");
say "glob1 = $glob1\nglob2 = $glob2"
'
globvals = 1 2 3 4.c 5.c 6.c 7 8
glob1 = 1
glob2 = 4.c
As you can see, the third call to glob returns the expected value.Here's how glob is documented in the perlfunc document[1].
glob In list context, returns a (possibly empty) list of filename
expansions on the value of EXPR such as the standard Unix shell
/bin/csh would do. In scalar context, glob iterates through such
filename expansions, returning undef when the list is exhausted.
[1]: http://perldoc.perl.org/functions/glob.htmlglob appears to do something special, and unlike most other Perl functions where it's just a matter of knowing the context and the API the function exposes. In the versions I just tested (including 5.21.6), glob in fact doesn't quite do what it's documentation says, in the apparent effort to "do what you mean". It does not iterate through files in the result when used in scalar context, it iterates through files in the result when used in scalar context and in the exact same call-site in the program.
Here's all the files we'll test
# perl -E 'my @files = glob("*"); say for @files;
one.a
one.b
two.a
two.b
Glob acts like an iterator when used in scalar context. Here, each call returns another file. # perl -E 'while ( my $file = glob("*") ) { say $file; } '
one.a
one.b
two.a
two.b
Glob gives the first file each time it's called in scalar context if it's not the same exact point in the source. This doesn't follow the docs, which says it will act like an iterator in scalar context. # perl -E 'my $file1 = glob("*"); my $file2 = glob("*"); say $file1; say $file2;'
one.a
one.a
Again, we see it's not acting like an iterator. Each call is generating it's own list and returning the first file. # perl -E 'my $file1 = glob("*"); my $file2 = glob("two*"); say $file1; say $file2;'
one.a
two.a
Here, we can see that since the same point in the code is getting hit, glob is acting like an iterator. # perl -E 'sub myglob { my $mask = shift; glob($mask); } my $file1 = myglob("*"); my $file2 = myglob("*"); say $file1; say $file2;'
one.a
one.b
Here we see that since the same point in code is getting hit, glob is ignoring it's input and just acting like an iterator on the first input, which seems to be what you were trying to show in your example. # perl -E 'sub myglob { my $mask = shift; glob($mask); } my $file1 = myglob("*"); my $file2 = myglob("two*"); say $file1; say $file2;'
one.a
one.b
That is very weird behavior. I agree it's not consistent with the rest of the language. I'm not sure it's indicative of the language as a whole though, as it appears to be due to weird historical implementation details that are kept for backwards compatibility. The perldoc for glob says it's implemented using the the standard File::Glob module. The perldoc for File::Glob mentions that it implements the code glob in terms of bsd_glob (the FreeBSD glob(3) routine, a superset of POSIX glob), which is function that can also be exported. In fact, the bsd_glob function, when used, acts as we would wish, without weird iterator behavior (without iterator behavior at all, in fact). # perl -E 'use File::Glob qw/:bsd_glob/; my $file1 = bsd_glob("*"); my $file2 = bsd_glob("two*"); say $file1; say $file2;'
two.b
two.b
# perl -E 'use File::Glob qw/:bsd_glob/; sub myglob { my $mask = shift; bsd_glob($mask); } my $file1 = myglob("*"); my $file2 = myglob("one*"); say $file1; say $file2;'
two.b
one.b
Of course, this means the documentation that says the core glob routine is implemented using bsd_glob has some glaring omissions.So, congratulations, you picked an extremely good example and unearthed some crazy Perl arcana, and possibly a bug (there's some open, longstanding tickets regarding glob bugs[1][2], which seem to boil down to "we're doing what we can to make it better, but we're hampered by backwards compatibility and weird semantics"). I think the documentation is woefully inadequate to explain what's going on in this case though.
Note: that bsd_glob seems to return the last item when used in scalar context, not the first.
[1]: https://rt.perl.org/Ticket/Display.html?id=2707
[2]: https://rt.perl.org/Ticket/Display.html?id=2713In all honesty, does anyone even care about Perl 6 at this point? It's become the Duke Nukem Forever of programming languages as far as I can tell.
Back in the 90s I remember the first CGI scripts I wrote were in Perl but really... the world has moved on.
That said, I think Python has eaten Perl's lunch, and who knows how things will evolve with the proliferation of statically-typed, non-verbose, fast functional languages? Personally, I'm not going back to dynamic languages as my primary workhorse.
- grammars
- grapheme-based Unicode
- implicit parallelism
This is why Perl6 is still relevant today. Larry is hopeful that Rakudo can deliver sometime next year (not all of these features are quite there yet), but even if it doesn't, the language will remain of interest as long as there is no serious contender in sight.
Yes, Haskell is better known, but not proliferated in real-world usage (though jq was based on it). F# and Scala aren't seeing much adoption.
I do think type inference will be borrowed from them (and pr many other features), but not pure fp (it makes some tasks unnecessarily difficult). The growth of dynamic typed languages suggests to me that developers value ease of development over almost everthing.
Perl 6 does seem to have become a perpetual playground for language experimentation, but that hardly makes it Duke Nukem Forever -- you can use it for that today.
Perl 5 is alive and well, seeing regular updates, and seems to have a future, even if its glory days as far as mindshare are over.
If we're talking about the world having moved on to either Haskell, Erlang or Go, Rust, well, I don't really see that being as pervasive as it often appears on trend-seeking sites like this.
And I really hope we're not talking about JavaScript…
Plus, I've always noticed a tendency in Perl in the last few years to be interesting/entertaining... sort of like it was designed by science fiction writers.
Perl's weird, it just takes a bit of everything (and not in a C++ kind of "be" everything), tosses it in a blender and then suddenly everybody has or wants smoothies in their language.
A short list of what I think we'll see percolating over to other places:
- some of the better syntactic sugar ideas, like blocks and closures, comma as a list constructor, chained comparisons
- what will probably be best-in-class Unicode support
- autothreading
- further improvements to the way regexes are used
- grammars
- junctions could become the next "thing" in lots of languages
- elements of the module system, like using two versions at once without conflict
- the different parameter passing modes
- the current Perl culture around testing
- bags and sets
- the continued relevance and model of CPAN
Although, I actually think a lot of the stuff Perl does is really cool. I wouldn't want to have to maintain a codebase of it, but it never fails to be interesting. Like the whatever star. "*.value" basically saying, "Whatever value is here, gimme the value of it". It's different from a typical lambda declaration, but intuitive in its own way.
Isn't it pretty similar to Scala's use of "_"? I think _.value creates a one-parameter lambda in Scala.
(* * *)(2, 3)
will happily print 6. > .map: ** * * * **
===SORRY!=== Error while compiling <unknown file>
Multiple HyperWhatevers and Whatevers may not be used together(map #(/ % 2) [2 4 6 8]) => (1 2 3 4)
(map #(% 2) [#(- 2 %) #(* 2 %)]) => (0 4)
(#(+ %1 %2 %3) 2 4 6) => 12
The following are identical with and without whatever:
perl6> "ACAGATAAATTA".comb.map: * ~ * ~ * ;
ACA GAT AAA TTA
perl6> "ACAGATAAATTA".comb.map({$^first ~ $^second ~ $^third});
ACA GAT AAA TTA
It's worth pointing out we're lucky that 'first' is lexically before 'second' in the second one though. The $^ consumes parameters in the lexical order of the variable names. So it's used normally for $^a $^b in sorting. So .sort({$^a cmp $^b}) versus .sort({$^b cmp $^a}). But a general feature was made rather than an obscure exception. In general Perl6 is very consistent. It might also just be a bit too crazy at times, some of the stuff you can do like the (* * *)(2,3) below you obviously would just not do in real life.So an example where the lexical ordering is kind of neat:
perl6> "ACAGATAAATTA".comb.map({$^third ~ $^second ~ $^first});
ACA TAG AAA ATT
Contrived since you could just split the string by three characters and flip it :P But hopefully you get the idea! $freq{substr $grade{$_}, 0, 1}++ for keys %grade;
better: $freq{$_}++ for map { substr $_, 0, 1 } values %grade;
best: my @letters = map { substr $_, 0, 1 } values %grade;
$freq{$_}++ for @letters;
----------------------------and
open my $fh, '<:utf8', "grades.txt"
or die "Failed to open file: $!";
my %grade;
while (<$fh>) {
m/^(\w+) \s+ ([A-F][+-]?)$/x
or die "Can't parse line '$_'";
$grade{$1} = $2;
};
better: use 5.010;
use IO::All -binary, -utf8;
my $line_format = qr/
^ # line start
(?<student>\w+) # name
\s+ # gap
(?<grade>[A-F][+-]?) # grade
$ # line end
/x;
my $fh = io->file("grades.txt");
my %grade;
while (my $line = $io->getline) {
die "Can't parse line '$line'" if $line !~ $line_format;
$grade{$+{student}} = $+{grade};
} use strict; use warnings; use List::Util
qw/reduce/;
map { print } join "\n", "Distribution of grades by letter:",
map { sprintf( "\t%s: %s student%s", @$_, $_ -> [1] == 1 ? '' : 's' ) }
map { my $c = $_ ; sort { $a -> [0] cmp $b -> [0] }
map { [ $_, $c->{$_} ] } keys %$c } reduce { $a->{chr(ord($b->[1]))}++; $a } {},
map { printf( "List of students with failing grades:\n\t%s\n", join ', ' ,
map { $_->[0] } grep {ord $_->[1] >68} @$_); @$_} reduce {push(@$a, $b); $a} [],
map { printf( "${^A}'s grade: %s\n" , $_ -> [1] ) if $_->[0] eq $^A; $_ }
map { $^A="Zsófia"; [split(/\s+/)] } <STDIN>
The use of strict and warnings making this particularly Modern Perl, you could almost be forgiven for thinking you were reading Haskell...Why would you use Perl6 instead of ruby?
Those sorts of answers from other's perspectives is what i'm interested in for Perl6 (not 5) over ruby. I think I know many reasons to prefer perl5 over ruby (although I don't), but once you leave perl5 for perl6... I'm not sure why you wouldn't just ruby (or python). But could be just cause you're used to Perl and like Perl. Or could be because there's something about Perl6 you especially like over ruby/python. Etc. I think it's a reasonable question?
My $0.02 is you'd mostly want to learn Perl6 if you were after something like Ruby but perhaps with more room to express yourself and form a community and build new technologies. At least I really love the language and find it very expressive and intuitive to use and it has a lot of features useful for doing Bioinformatics built in (something I'm going to add to the P6 advent calendar on the 15th Dec). The extra level of detail ten years of design puts into a language like all the funky features working together and being consistent helps a lot. So much of the cooler expressive/functional stuff in Python feels out of place like lambda and list comprehensions. Maybe I'm wrong but I wouldn't write anything /that/ complicated or important using Python lambda or list comprehensions. They just don't play that well with the surrounding code you will have to write in a different style. I can't attest to Ruby since I don't know it well. I felt it was quite close to Perl and I'd be better placed learning Python instead, especially for a job writing code that wasn't for the web.
Maintainability, productivity, etc. of Perl 5 is on the same level with Ruby. And I don't think Perl 6 improves on that, so it's probably the same too.
Personally I don't see any benefit for anyone to switch between mentioned languages. They all have similar problems.
Here's a quick intro to supplies: http://perl6advent.wordpress.com/2013/12/19/perl-6-supplies-...
Here's a more in depth exploration of Perl 6's concurrency features: http://jnthn.net/papers/2014-nlpw-reactive.pdf
If you want to use a dynamic language (albeit also offering optional static typing) with powerful concurrency options that doesn't also involve s-expressions, I think P6 might be one of the few options available. I'm not familiar with Ruby, but from what I've heard the default runtime has some limitations when it comes to concurrency, and while it's been a while since I used Python, I recall hearing that it also has some problems with handling concurrency.
Wait, what? There are are specific use cases where this might still be true, but as a general rule, this hasn't been the case for some years now.
I don't know if it will be the same for perl6, but at least with perl5, backwards compatibility was rarely broken. Every time my debian-testing box upgraded a ruby version, it broke something. From 1.8,1.9,2.0, and I think 2.1 actually worked without any issues...
This is never an issue with perl5. And before you say "perl is dead it never changes", go look on the perl5porters mailing list. There are a ton of changes going on. I had every intent this year to join the dev team, but the commits come in so quick on that mailing list, I can never find a place to start. When perl5 updates to a new version (which has been happening once a year for like 5 years now), it is not just a few bugs that are fixed, but a whole lot of new features are added and plenty of new syntax to scare the heck out of anyone. So an argument of the ruby dev cycle is 'slower' probably won't hold up.
Onto more perl6 specifics, I think one of the original goals was perl6 being able to compile a script into a raw binary. Don't know if that feature will be implemented. Ruby can't do that, though it does have the JIT compile stuff. So I guess that could be one reason..
Perl6 is new, so I guess it is full of new "technology and stuff" right?
I personally never did much with ruby. I tried to use it as my first language to learn with. I signed up on some hacker forum wanting to learn to hack with my new found ruby skills... They just booted me with the message of "ruby aint really programming". This was back in 2007, but I never could get over that...
From then on I only used Ruby when I couldn't figure out how to do something in perl. So none of my ruby scripts have ever been seen by another pair of eyes. I keep them all hidden, knowing that they are cool and all in their own right, but knowing if I were just a little smarter... I could have written it in perl...
The short answer, is that the comparison isn't fair to either communities. Perl 6 has many killer features. Concurrency, Macros, Grammars, Regex goodness, functional programming features, OO features, Strong typing support, user defined operators, multiple VM backends, Continuations to name a few. But there are many many others.
Perhaps the most killer feature of Perl 6 is like lisp, you can use the language to grow itself without breaking backwards compatibility. This alone will ensure that the community can grow without worrying about breakage, regression and backwards compatibility.
Read more here: http://perlcabal.org/syn/
Ruby and Perl 5 definitely have more libraries, users and a far more mature ecosystem of things. But I guess the sheer awesomeness of Perl 6 will attract a lot of developers and that will have a healthy community of its own.
In short I think there are plenty of reason why one would chose Perl 6 above Ruby.
#!/usr/bin/env python3
with open('data.txt', 'r') as f:
data = dict(x.split() for x in f if x.strip())
print("Zsófia's grade:", data["Zsófia"])
print("List of students with a failing grade:\n ",end="")
print(*[x for x in data.keys() if data[x] == 'F'], sep=", ")
print("Distribution of grades by letter:")
for k,g in groupby(sorted(data.values()), lambda x: x[0]):
print(" {}:".format(k), sum(1 for _ in g)) #!/usr/bin/env python3
from collections import defaultdict
def invert_dict(data):
res = defaultdict(list)
for k, v in data.items():
res[v[0]].append(k)
return res
with open('data.txt', 'r') as f:
data = dict(x.split() for x in f if x.strip())
print("Zsófia's grade:", data["Zsófia"])
print("List of students with a failing grade:")
print(" " ,end="")
print(*[x for x in data.keys() if data[x] == 'F'], sep=", ")
print("Distribution of grades by letter:")
grouped = invert_dict(data)
for k, g in sorted(grouped.items()):
print(" {}: {} student{}".format(k, len(g), 's' if len(g) > 1 else '')) from collections import Counter
with open('data.txt', 'r') as datafile:
students = dict(line.rsplit(maxsplit=1) for line in datafile)
failing = [name for name, grade in students.items() if grade >= 'E']
grade_counts = Counter(grade[0] for grade in students.values())
print("Zsófia's grade:", students["Zsófia"])
print("List of students with a failing grade:")
print(" " + ", ".join(failing))
print("Distribution of grades by letter:")
for grade, count in sorted(grade_counts.items()):
print(" {}: {} student{}".format(grade, count, 's' * (count != 1)))Perhaps I'll clean it up using printf/format[0] later.
{-# LANGUAGE OverloadedStrings #-}
import Control.Monad (liftM)
import qualified Data.HashMap.Lazy as H
import Data.List (sort)
import Data.Maybe (fromMaybe)
import Data.Monoid ((<>))
import qualified Data.Text as T
import qualified Data.Text.IO as TIO
import Prelude
main :: IO ()
main = do
gradesMap <- liftM
(H.fromList . map ((\(x:y:_) -> (x,y)) . T.splitOn "\t") . T.lines)
(TIO.readFile "example.txt")
let failingNames = H.keys $ H.filter ((== F) . T.take 1) gradesMap
gradeTotals = foldl
(\m k -> H.insertWith (+) (T.take 1 k) 1 m)
H.empty
(H.elems gradesMap)
TIO.putStrLn $ "Zsófia's grades: " <>
fromMaybe "N/A" (H.lookup "Zsófia" gradesMap)
TIO.putStrLn "List of students with a failing grade:"
TIO.putStrLn $ " " <> T.intercalate ", " failingNames
TIO.putStrLn "Distribution of grades by letter:"
mapM_ (\(k,v) -> TIO.putStrLn $ " " <> k <> ": " <> (T.pack . show $ v) <> " students")
(sort . H.toList $ gradeTotals)
0: http://hackage.haskell.org/package/formatting... why? What was wrong with {} or [] , as most other languages do? Sigh.