Streams in Java 8 | Hacker News Reader

Streams in Java 8

download.java.net

94 pointsbra-ket89 comments

While lambdas and streams (along with default methods, or traits) are certainly the most important features of Java 8, I'm particularly interested in the pluggable type systems[1].

These are pluggable, orthogonal (intersection?) types, that can even be introduced or inferred for code that has already been compiled without them. I'm not a type systems expert by any means, but I haven't seen anything like it in any other language, certainly not a mainstream one.

There are currently Java 8 type systems for nullability, immutability and more.

[1]: http://types.cs.washington.edu/checker-framework/

    sorted(Comparator<? super T> comparator)

Eurgh. Why isn't this a projection to a comparable type (or better, a projection to a tuple of comparable types, for use with lexicographical ordering)?

Say you have class Country { int population; int name; }. You want the names of the top 3 countries by population:

    countries.orderByDesc(x -> x.population).limit(3).map(x -> x.name)

But instead, you must do something like:

    countries.sorted((x, y) -> Integer.compare(y.population, x.population).limit(3).map(x -> x.name)

It gets much uglier when you want a lexicographical ordering by several attributes.

Java's Comparator type is more general (you don't necessarily need to project the type for comparison) and an existing abstraction. It's not hard to introduce a combinator to transform the representation as a library, as in:

    countries.sorted(by(x -> x.population)).limit(3).map(x -> x.name)

where `by` looks something like:

    <S, T> Comparator<T> by(Projection<S, T> projection);

    interface Projection<S, T> {
       Comparable<S> project(T value);
    }

Sure; and you'd have to import the thing that contains 'by'.

It's still ugly, and would prevent e.g. a radix sort being used when projecting an integer as the sort key, or American flag sort for strings.

It already exists in the API. Your type Projection is a Function, and by is called comparing in interface Comparator.

so one can write: ``` contries.sorted(comparingInt(Country::getPopulation)). ... ```

>However, there are good reasons to prefer a reduce operation over a mutative accumulation such as the above. Not only is a reduction "more abstract" -- it operates on the stream as a whole rather than individual elements -- but a properly constructed reduce operation is inherently parallelizable, so long as the function(s) used to process the elements are associative and stateless.

I've read this kind of thing everywhere, but is there any language/runtime/compiler that DOES DO this, or is it mostly a case of "an adequately advanced compiler" syndrome?

Especially for examples as the simple accumulation (addition) above, it seems especially difficult for the runtime to judge how to break down, what the parallelization overhead would be etc.

If I haven't misunderstood you, Clojure does: http://clojure.com/blog/2012/05/08/reducers-a-library-and-mo...

Also, this talk that I found on HN some time ago explains some of the gory details behind the idea: http://vimeo.com/6624203

>* If I haven't misunderstood you, Clojure does*

Yes, that's an example of what I was talking about, and I know you can also do it with Apple's Grand Central Dispatch API.

But is Clojure's stuff implicit or do you explicitly ask for the parallel thing to happen?

What I'm asking is essentially: sure, a reduce/map/filter/etc can potentially run in parallel. But does that ever happen automatically when using common API's for map/reduce/etc, or do you have to explicitly tell it to do so?

And if it happens automatically, how does the scheduller knows it's a good idea to run stuff in parallel? For some tasks the overhead might even make the time needed worse.

also https://blog.codecentric.de/en/2013/10/java-8-first-steps-la...

Thank you.

This hurts my eyes, it doesn't look like Java, what would this look like in Java:

     int sum = widgets.stream()
                      .filter(b -> b.getColor() == RED)
                      .mapToInt(b -> b.getWeight())
                      .sum();

    List<Widget> filtered = filter(widgets, new Filter<Widget>() {
        @Override public boolean keep(Widget w) {
            return w.getColor() == RED;
        }
    });

    List<Integer> weights = mapToInt(filtered, new Mapper<Widget, Integer>() {
        @Override public Integer map(Widget w) {
            return w.getWeight();
        }
    });

    int sum = sum(weights);

Nice, thanks! Now _that_ I get.

Seems too short. Maybe you could split it into 5 files? You don't even have a factory. Much less a factory factory. This code will never cut it in the enterprise world.

It turns out there's a good blog article that explains why the less enterprisey way is better since it remove the iteration loop of the collection and lets the computer [figure out how to express the sum of weights of the red objects in a collection of blocks]:

http://blog.hartveld.com/2013/03/jdk-8-33-stream-api.html

I've been around too long to trust the computer to figure out how best to do something. If I'm wrong then Java 8 leads the way to the end of programmers, similar to how robotics mostly ended how cars were manufactured.

  int sum = 0;
  for (w : widgets) {
    if (w.getColor() == RED)
      sum += w.getWeight();
  }

The one feature that Python has that regularly makes me jealous when I'm using other languages is the generator / iterator mechanism. It's possible to write code that is composable and modular in a functional way, and automatically scales to massive data with minimal memory use because the streaming mechanism is built in to the language at the ground level. I hope this is bringing the same to Java for real and that this is not just syntactic sugar on top of regular collection operations.

From the docs: »•Possibly unbounded. While collections have a finite size, streams need not. Short-circuiting operations such as limit(n) or findFirst() can allow computations on infinite streams to complete in finite time.»

So they can't just take collections here and pretend they're streams (but you can obtain streams from collections).

But nice to see that the days of writing explicit loops just to filter a collection in Java are over. That being said, I still like Python's or C#'s syntactic enhancements to write generators/IEnumerables. The Spliterator in Java doesn't sound as easy to use and seems more aimed at low-level library code instead of user code. In the latter case you'd probably just get the stream from a collection again instead of creating it yourself. (You can apparently base it on a normal iterator, though, but the docs warn about it having bad performance in parallel scenarios.)

This is really cool, that they are making parallelization easier but does anyone else feel like every time they add new syntax to java it becomes harder and harder to read?

I think the lambda syntax makes it considerably easier to read.

It makes reading java more like reading lisp, which is a different way of thinking entirely. Personally I think it's a mistake to put functional programming into an OO language - it makes code harder to maintain as there are too many ways to do the same thing. C++ has become virtually unusable as a result of all the different languages they've mashed into it, and I'd hate to see java go down the same road.

"C++ has become virtually unusable..."

Geez, the C++ hate is unbelievable. I use it every day, and it's a great language. No way is it "virtually unusable". Some of the criticisms are justified, but the way people carry on about it is just absolutely beyond overblown.

Streams are among my favorite Java 8 additions.

More about the goodies in Java 8: http://www.techempower.com/blog/2013/03/26/everything-about-...

I have been experimenting with lambdas and streams for several months - great additions to the language.

The one thing that I really hope for a future release of Java is a better literal syntax for dealing with maps and JSON.

Yay. They finally caught up to C# 6 years later.

[Edit: to the haters -- where am I wrong other than that I'm making fun of your favorite language?]

(I did not downvote you)

Such jingoism adds nothing to the discourse. Someone can as easily find something in Java 1.5 that C# still doesn't have. So what does that prove? Nothing.

> Someone can as easily find something in Java 1.5 that C# still doesn't have.

Like what?

static imports ?

You could add an extension method to object, if you want the same effect.

(Although: A workaround for java's bizarre stance that no function can exist without an object doesn't really strike me as a "feature". [Granted: this criticism is equally true of C#])

It's in C# 6 :)

checked exceptions :), sorry just kidding.

mainly wildcards, you can design an interface which is covariant and contravariant, and real interference when calling a generic method.

The lack of wildcard types in C# consistently bothers me when I use it.

It looks to me like C# supports everything from http://docs.oracle.com/javase/tutorial/extra/generics/wildca... using generic type constraints: http://msdn.microsoft.com/en-us/library/bb384067.aspx

What am I missing?

They added Covariance and Contravariance of generic type parameters in 4. http://msdn.microsoft.com/en-us/library/dd799517%28v=vs.110%...

Stream<T>, IntStream, LongStream, DoubleStream?

You can't make this stuff up.

EDIT: what I'm trying to say is: why his remark is inflammatory and unnecessary, it's also wrong. This API is perhaps the single worse example of why java makes API's look worse than C#, and there is nothing to be done about it until the JVM stops erasing types.

Java's type erasure, while once considered a disadvantage, is now being recognized as one of the reasons other statically typed languages can be ported to the JVM. If Java reified types, it would make interoperability with other type systems extremely hard.

Yet somehow they made F# and scala run on the CLR. So: I'm calling bullshit.

F# team ask for modifications of the CLR (covariant delegates), porting a language on the CLR if you are not Microsoft is hard.

Scala never really run on the CLR, it's a vaporware, sorry a work in progress.

Yep, scala's .net back-end is discontinued, and one of the main challenges (I worked on it a bit) was indeed generating the proper generic signatures. (I hear people are using ikvm these days.)

For more context on why F# does in fact run into problems, see https://visualstudio.uservoice.com/forums/121579-visual-stud.... To my understanding, F# only allows type variables to vary, not type constructors (all generic variables must be escorted by a concrete type), and to go a step further, it doesn't allow type constructors to be used as higher-kinded type variables, which is useful sometimes. I see no reason why these couldn't be implemented on top of the JVM, though.

I've never understood why Java programmers dislike type erasure. In my mind, runtime type inspection is a bad idea for a supposedly statically typed language. It just seems like people are complaining that they can't do something they probably shouldn't be. If there's something you can't do without it then that means Java is poorly designed, not that type erasure is a bad idea.

To be fair, erased generics force autoboxing of primitives. Anyway, Java's combination of static and runtime types, though not pure by any measure, has actually proven quite useful.

> is now being recognized as one of the reasons other statically typed languages can be ported to the JVM

No, I really don't think so.

I see this claim fairly often, but it seems like wishful thinking to me. There's nothing stopping a language targeting .NET from erasing its own generics to `List<object>`, `Map<object>`, `T<object>` if necessary, which would achieve almost exactly what you'd get with Java's erasure. And in the meantime, Java's approach requires any generic class containing arrays to box small value types, wasting large amounts of memory when those arrays are large.

Can you clarify your comment? I don't understand how those four types could be avoided with reified generics.

Because then you'd only have Stream<T>; IntStream would be just Stream<int>. The problem with reified generics is that they bake a particular type system into the runtime. While once a desired feature, I don't think the wide JVM community wants that any more.

To be fair, I don't think this is a problem with reified generics. The problem is that primitives cannot be erased to Objects in Java. Stream<int> only works in C# because int is an alias of Int32. Not because of reified generics.

The above should no longer be an issue when/if we remove the primitive types in Java 10(?) as we can safely use Stream<Integer> and Predicate<Integer> (or Stream<int>).

The runtime type erasure would have to be quite clever to realize that a ArrayList<Integer> should be backed by a primitive array of ints? One alternative is to erase ArrayList<T> into all variants (One for objects and one for each primitive type).

(In C# System.Int32 is a value type, and an "int" is equivalent). A List<Int32> is equivalent to a List<int> and is backed by an array of ints.)

Type erasure has ups as well as downs. For one, it means you can write runtime polymorphic code when you don't care about static types - something that requires extra effort and a non-generic base type in .net. Any time you pass an object through a runtime polymorphic "hole" (e.g. a location of type java.lang.Object / System.Object), you lose the ability to pass along type parameters. In .net, if you don't have that abstract base class / interface, your only fallback is reflection. In Java, you've got List<?> or whatever it is.

According to TIOBE, it looks like C# stands no chance of ever catching Java. (End of troll feeding)

What I find really interesting about your comment is that I always thought C# was the closest equivalent to Java there was, and that I'd be happy using it if I was developing in the Microsoft eco-system (I gave up on QuickBasic/VisualBasic). It would make more sense if you were sniping at a language you felt was inferior if that language was also signicantly different than Java.

P.S. I didn't downvote you either

Preface: I am not a microsoft fan, or even really a C# fan, but: having used both languages C# is nothing like java except in the most superficial of ways. I'm just amused that the caretakers of java took half a decade to adopt a lot of proven useful features. I wish they'd steal more.

C# is more similar to Java than any other language.

(Because you said so)

(Don't worry about, you know, backing up your point or anything silly like that. That might require that you know what you're talking about.)

Gosh, aren't you a pleasant fellow to deal with? Try to assume good faith.

I'll bite. C# is syntacticly and idiomatically more similar to Java than any other language. I'm sure how I can prove this, beyond saying "look at them!" – is there a counter example I'm missing?

They still need to steal the properties so we can remove the getter/setter and the local type inference from c#.

I must admit that except for this two missing features, I'm quite happy with the java 8 release in term of features.

It's still just too much boilerplate to define and use immutable types. Named parameters would help a lot. Something like Scala's case classes would help even more, and along with them, some kind of pattern matching.

Take a look at Project Lombok: http://projectlombok.org/

A bit hacky, but it works.

To answer your question, your comment is just snark that doesn't contribute to the conversation. That's probably where you went wrong.

You can accuse me of trolling, but I think the fact that java took this long to adopt these pretty basic features isn't besides the point, it IS the point.

In my opinion, lambdas were not introduced in Java before now because Java as already anonymous classes that are at a high level conceptually the same things even if a lambda is more lightweight syntactically and at runtime than an anonymous class

Java certainly moves very slowly. I recall reading debates on how to support Lambda long before Java 7.

The reasons, IMO, are twofolds: 1) The huge install base in enterprise: changing language feature is a big deal, and like it or not, enterprises are conservative in this aspect. One only needs to look at all the garbages included with Windows in the name of backward compatibility. 2) Design by committee: Many vendors have stake in Java eco systems, all with their own agendas. To get them to agree on something take time.

In some ways, Java is victim of its own success. On the other hand, JVM is a very solid platform. Those impatient can go with Scala or Clojure. I could be snarky like you were and said Scala was light year ahead of C# in terms of feature evolving, but of course I know C# is a fantastic language which serves its users very well.

Sure, but you can make that point in such a way that it encourages dialogue. A good point won't need an overly edgy delivery.

I could, but I don't care to.

Then you don't really need the crocodile tears of "Why the hate, guis?". Just set phasers to Such Edgy and blast away!

Context doesn't always come across on the internet, but there were no crocodile tears. I actually thought it was amusing that what I thought was a relatively tame comment got such a reaction. (If anything I'm more happy getting negative votes. I get really uncomfortable when I say shit that gets upvoted to hell because it feels like I'm just shouting into the echo chamber)

since when are lambdas a basic feature? since scheme in the 70s? since smalltalk-80 in 1980? since ruby in 93? since Haskell 98? since c# in 2006?

if I got in my delorean, hit 88 miles per hour, went to Sun Microsystems in the early 90's and convinced Guy Steele that yes, we should include Guy's currently 15 year old research of lexically scoped, first class functions into this new language which is designed to drag C++ programmers halfway to lisp, then java would have not been widely adopted, and this run on sentence would never have been written.

If we assume C++ and Lisp are local maxima, dragging C++ developers halfway to Lisp may not be the best thing to do.

I mean, given your first paragraph I would say: yes. To like all of those. It IS basic. So yeah. It's been a basic feature since the 70s. I guess you can give a demerit to C++ for it being the one feature they didn't steal from elsewhere, but then again you can give C++ a black eye for a lot of things.

(And still: C++ managed to get lambdas before java did. How messed up is that? Extending Java is about 10x easier than trying to add something to the crufty minefield that is the C++ spec, and they still beat them to the punch.)

Does C#/.NET have a fully supported runtime for any operating system that can delete (or rename) an open file? ;-)

Quick advice: if you have a valid point, you should state it in a form more conducive to debate. I agree with you Java has some quirks "more modern" languages don't, including C#, which started its life, more or less, as a replacement to Microsoft J++ after Sun successfully made a painful point Microsoft could not "embrace, extend and suffocate" Java just because they wanted to.

> Does C#/.NET have a fully supported runtime for any operating system that can delete (or rename) an open file? ;-)

Yes.

Thank you for the quick advice, I'll be sure to ignore it. Because I don't care.

So cool.

Java has come a long way. Why not just merge with Scala?

Because Scala has a very different (I would say completely opposite) philosophy than Java. In a paper outlining Java philosophy[1], the very first thing James Gosling has to say about the language is this: "It was not an academic research project studying programming languages: Doing language research was actively an antigoal."

When describing the design of language, again, the very first thing Gosling says is, "Java is a blue collar language. It's not PhD thesis material but a language for a job. Java feels very familiar to many different programmers because I had a very strong tendency to prefer things that had been used a lot over things that just sounded like a good idea."

So: no research, no PhD, only proven stuff and nothing new. Regardless of your opinion of Scala (I don't like it), I think it's not controversial to say that Scala is on the exact opposite side of the spectrum.

[1]: http://dfjug.org/thefeelofjava.pdf

Except your argument is invalid because java has continued to add features and keywords to address its flaws.

It is moving towards scala, just at an incredibly slow rate.

But moving at an incredibly slow rate is exactly what the Gosling document says: Java will only implement features that are familiar and proven, namely years after they've been introduced. Also, Java's philosophy says that no feature is to be adopted, no matter how nice, unless it solves a very, very, very painful problem. Many people, myself included, think it's a very wise stance for such a widely adopted language.

Scala is first and foremost a research language. It has some very good features and some terrible features. Also it is completely unopinionated. But that's how it's supposed to be: as a research language, its main goal is to figure out what works and what doesn't, so that mainstream languages years from now will know what works and what doesn't.

If java was taking things out as it added new things in, then we'd have a good debate on our hands. Alas, they don't seem to be taking this stance, and instead are adding in more and more (just slowly) without removing any of the cruft.

Despite reading your many "scala" rants on HN, I still don't know which features you think are 'terrible', just that you think there are some that confuse you and/or are allow you to something multiple ways (abstract classes vs. interfaces with concrete implementations say what???).

Care to actually articulate the 'terrible' features this time?

Java can't take stuff out, at least not anything major. Java's adoption is two or three orders of magnitude higher than most languages out there (except C/C++); you can't make so many millions of developers go back and change 10 or 15 year-old code. One of the reasons it's this popular is that it's consistently backwards compatible.

I don't want to be dragged into the Scala discussion again (after all, these are just opinions; I realize some people really like Scala, but for some reason, Scala people just find it hard to accept the very real fact that some people don't like it, and it's not because they haven't tried it or don't understand it).

Sure, Scala is used for research, it came from research, but I don't think it's accurate anymore to say it's a research language. Some of our clients are doing some really cutting edge stuff, but I wouldn't call them research institutes. I'm of course biased as well, but I'd say Scala's been maturing nicely with 2.10.x and (soon!) 2.11.0.

For example, which of the official new features in Scala 2.11.0 is research-y, would you say? We worked on faster incremental compilation, started modularizing the standard library, improved our infrastructure for better integration builds with SBT & Scala IDE. We did add experimental support for SAM types -- is it "research" to prepare for Java 8 interop? The main experimental features are of course reflection and macros, developed at EPFL. I'd say we've been pretty clear about labeling these experimental bits "experimental".

FYI: The comment was put there to stimulate discussion. Thanks for the perspective and reading material provided.

And the one major non-proven thing that it included - checked exceptions - is perhaps the most problematic feature.

It's not the only one. Concurrency support ('synchronized') may be another one and I would see that one even more problematic.

Just recently I was hit by a runtime exception from a library which apparently decided that using undocumented unchecked exceptions was a good idea. The vast majority of open source libraries are not or poor documented, even ones which are used heavily (as in my case). In these, checked exceptions are a godsend as they MUST appear in the method signature.

For many programmers, exceptions just get in their way and they handle them poorly which is imho one reason for the bad reputation of checked exceptions.

Reply on news.ycombinator.com