Java 8: Replace traditional for loops with IntStreams | Hacker News Reader

Java 8: Replace traditional for loops with IntStreams

deadcoderising.com

95 pointsmariushe117 comments

Maybe a heretical opinion here, but I like writing loops much more than using higher order functions like map/fold/filter, even though I have quite a bit of experience with ML family languages by now. These are my reasons:

1) The automatic parallelization opportunities of HOFs don't happen in practice nearly as often as you'd expect. Sorry, but it's true. Haskell's "map" doesn't parallelize automatically.

2) Loops are a single construct you learn once and apply everywhere, while HOFs are a huge and bewildering zoo.

3) Loops can do many things that HOFs cannot. You can do break/continue/return in the loop body. You can iterate over two collections at once, skipping elements of one collection depending on what you see in the other. And so on.

4) Loops scale to from simple use cases to complicated ones gradually and continuously. With HOFs, when your use case changes slightly (e.g. you add a counter), you often need to go all the way up and use a different HOF.

5) Loops make the time and space complexity much more obvious. With HOFs, you often get nasty surprises. For example, see the tail-recursive vs non-tail-recursive implementations of "map" in OCaml, or the foldr vs foldl vs foldl' situation in Haskell.

6) Loops are much easier to understand from a machine point of view. For HOFs, you need a language with closures, and in some cases GC as well. Loops, on the other hand, can be done in C.

In a nutshell, loops are easier to learn, easier to read, easier to write, and easier to execute.

Loops can do many things that HOFs cannot.

That's an advantage of HOFs; when you see one, you know exactly what it does, compared to loops where anything can happen either intentionally or accidentally. It's a form of the principle of least power.

For HOFs, you need a language with closures, and in some cases GC as well. Loops, on the other hand, can be done in C.

C uses HOFs in many places, like the compare function in qsort.

> when you see one, you know exactly what it does, compared to loops where anything can happen either intentionally or accidentally.

That is debatable, which is shown by the parent comment decrying the "bewildering zoo" of HOFs. I could flip this argument around and say that for loops have all complexity explicitly written out using fairly simple low-level constructs, whereas with HOFs I have to remember the semantics of each of them. And it would be equally unconvincing.

The problem I have with loops is that they are like fly-paper. Once you have a loop iterating over a collection it's tempting to add more and more to it, mixing responsibilities in run-on code. This isn't theoretical, I see code like this all of the time. It's nearly impossible to abuse HOFs that way.

Yeah, I agree. "Fly-paper" sounds a bit judgmental though. For a more neutral metaphor, we could say that HOFs are like Lego bricks, and loops are like clay. Both have their benefits and drawbacks.

I've been fascinated by HOFs for many years, but my personal projects are filled with "loop abuse" and mixed responsibilities, all soft and malleable, just the way I like it. If I had to dismantle half the thing just to change the color of one brick, I'd give up pretty quickly.

That sort of change is easy - at least you have bricks. Clay just oozes all over itself.

Whatever works for you. If it's just your code, you have enough control to keep loops from going bad. Large multi-programmer projects seem far more prone to that problem.

I second gp comment. HOF and FP in general constraints programming so much it leads to decoupled logic. Sometimes too much, but it's a good way to balance imperative statements allowing too much state.

Flypaper accumulates more bugs than clay.

Yeah, I see this all the time. Giant 600 line functions made of nested for loops, with all sorts of shit going on at each layer.

The author will insist it's basically impossible to abstract any of it, that the for-loops are the only way to capture the essence of the problem.

Then you spend two days tracking down a bug in there and discover that it's all actually some combination of map, reduce and a fold with a closure at heart.

Maybe the solution is to break the large function into several smaller ones, even if they still make use of for loops.

No doubt. But once it gets to a certain point the costs of doing that are much larger than having started in a compositional style.

If you have Martin Fowler's Refactoring book, check out the intro example. It's a 20 line function and the amount of refactoring work he does on it to make it sensible is quite large.

> 1) The automatic parallelization opportunities of HOFs don't happen in practice nearly as often as you'd expect. Sorry, but it's true. Haskell's "map" doesn't parallelize automatically.

Haskell generally won't parallelize unless you tell it to. HOFs are not specifically for parallelization, however the cost associated with switching from a serial HOF to a parallel HOF can be very, very low. The same is absolutely not true for loops. There are some interesting things available with SIMD macros, but that's not parallelization in the way you're referring to it.

> 2) Loops are a single construct you learn once and apply everywhere, while HOFs are a huge and bewildering zoo.

This is sort of true, but as noted in a sibling comment, this is not necessarily a good thing. Each loop construct can behave differently. The invariant could change, the initial index might be different, the increment is arbitrary. None of those are things you are supposed to do, but they are all stepps taken used to obtain a behavior. When you are using HOF's, each one has a specific purpose and behavior and they can be understood in isolation and then in as many or few compositions as desired

> 3) Loops can do many things that HOFs cannot. You can do break/continue/return in the loop body. You can iterate over two collections at once, skipping elements of one collection depending on what you see in the other. And so on.

I'll agree on this point, mostly. They are strictly more powerful when you want to do something complicated. You do pay for the power with complexity. My one quibble is that it isn't necessarily that hard to do co-iteration, and let's be honest, most co-iteration is done 1 for 1.

> 4) Loops scale to from simple use cases to complicated ones gradually and continuously. With HOFs, when your use case changes slightly (e.g. you add a counter), you often need to go all the way up and use a different HOF.

I don't see this as even a minor issue. In fact, it's kind of the point. If you need to do something different, create a new composition of functions and as long as they have the same type signatures and the operation is what you now want, there's no problem at all.

> 5) Loops make the time and space complexity much more obvious. With HOFs, you often get nasty surprises. For example, see the tail-recursive vs non-tail-recursive implementations of "map" in OCaml, or the foldr vs foldl vs foldl' situation in Haskell.

Except when they don't, as in the case where you are doing manipulations of the index or loop invariants; one of the things you cited as a benefit of loops.

I don't know about OCaml, but foldr and foldl are strictly different behaviors. One folds from the left, the other folds from the right, which is seriously important if you're using a non-associative operation. Now, maybe you want to say this is backing your point up, but write a for loop that does a non-associative left fold over a single linked list and we'll talk about obvious complexity.

To make an aside, are you maybe thinking about this in terms of Haskell's lists vs C arrays? The fact that the data structures are so, so different makes a huge impact on how you work with them.

foldl vs foldl' has nothing to do with HOF in particular, but Haskell's non-strict evaluation semantics. If you want to argue about strictness vs non-strictness though, that's a totally valid thing where I don't think there's any single dominant viewpoint. Non-strictness has certain benefits, but it's not necessarily worth it, especially if you can ask for limited non-strictness with otherwise strict semantics.

6) Loops are much easier to understand from a machine point of view. For HOFs, you need a language with closures, and in some cases GC as well. Loops, on the other hand, can be done in C.

You don't need a language with closures. Closures are definitely a great thing to have in general, but all you need is first class functions. You can write HOF's in C if you want to. It's a pain in the ass because function pointers are annoying as all hell, but you can absolutely do it.

Overall, I'll say that there's probably a point to be made that there's a bit of overhead in learning all the different and varied HOF's that work with a function, but I don't really think it's that much larger than becoming familiar with any given pattern you can implement in a loop construct. The difference being that you implement the loop construct every time you want to use it. With a HOF, you just refer to the one you want to use.

>1) The automatic parallelization opportunities of HOFs don't happen in practice nearly as often as you'd expect. Sorry, but it's true. Haskell's "map" doesn't parallelize automatically.

Maybe in your world, but not in Scala. Happens all the time in scala, if you're using a HOF on any monadic context it's trivial to add additional context to allow for parallelism without changing the underlying structure of any subsequent code.

My hobby: probabilistically down-voting jargon-laden comments on HN. Mere jargon gets a 50% chance of downvote (I flip a coin); that shoots to 100% when someone uses jargon and the word "trivial".

So, in addition to knowingly violating HN etiquette you are bragging about your ignorance? Tell me why what I said isn't 'trivial'? If you aren't familiar with the concepts, then how could you even know if what I said was simple or not?

>if you're using a HOF on any monadic context it's trivial to add additional context to allow for parallelism without changing the underlying structure of any subsequent code.

If that's not head-up-your-ass jargon, then I don't know what is. And I'll continue to down-vote stuff like that.

Writing is about communicating, not demonstrating how smart you are. Abusing me as ignorant tells me you have little interest in communicating, and confirms you were posturing.

And how do you know what I am or am not familiar with? Jargon is orthogonal to content.

>To me, docker can be thought of as a process wrapper. The executable is called an image, and the running process called a container.

That's from a comment of yours. It's meaningless to me because I don't know much about deployment/dev-ops etc.

Now I could lash out out of some insecurity and say that you're just using "jargon", or I could take some responsibility and look up those words.

You don't know what "process" and "executable" means? You've got more important things to do than troll me, then.

Personally, I like both for loops and higher-order functions over sequences.

Loops can be more powerful, but they force the reader into an imperative mindset where they have to mentally step through the code to understand what it's doing. That makes sense when the imperative model is the natural way to understand the code, but it can be overkill when there is always a simple high-level way to understand the operation.

Higher-order functions give you that. If I'm filtering a sequence "filter()" makes that fast and easy to read. But once I start chaining more than a few together, or start passing large lambdas to them, they lose that higher level reasoning and aren't worth it.

One way to look at it is, "How will the reader most easily think about this code?" If it's as a series of high-level transformations, use HOFs. If it's as imperative code, use a loop.

Another way is, "Does this code need to do control flow?" If so, use a statement. That's what they're good at. Otherwise, HOFs work fine.

I do find myself converting code from one form to another as it changes over time. I wish that transition cost was lower but I don't find it to be a big deal.

> The automatic parallelization opportunities of HOFs don't happen in practice nearly as often as you'd expect. Sorry, but it's true. Haskell's "map" doesn't parallelize automatically.

It's just a s/map/parMap/g away. Sometimes parallelization isn't worth it though. Are you talking about automatic parallelization as something that parallelizes anything possible or that also analyzes whether it's a good idea?

The functional style eliminates types of bugs that can happen with for loops and makes the code easier to read. For example, consider the task of removing names that start with "B" from a list. This is the for loop way:

    List<String> names = Lists.newArrayList("Alice", "Tom", "Bob", "Brandon", "John", "James", "Ben");
    List<String> bNames = new ArrayList<>();
    for (String name : names) {
        if (name.startsWith("B")) {
            bNames.add(name);
        }
    }
    names.removeAll(bNames);

And the functional way:

    names = names.stream().filter(name -> !name.startsWith("B")).collect(Collectors.toList());

The basic problem with the for loop is that you need create a temporary list to do the work. It's low level coding that requires telling the computer exactly where to put the temporary results, then what to do with it. I've seen bugs where developers accidentally return the wrong list (such as bNames in this example) or later modify the wrong one. When the code becomes more complicated, there's often a lot of these temporary variables that greatly reduce code readability and allow subtle bugs to occur.

What about iterating from the end of the list in this particular case. This way you don't need a temporary list

    for (int i = names.size()-1; i >= 0; i--) {
        String name = names.get(i);
        if (name.startsWith("B")) names.remove(i);
    }

Admittedly, needing to use an explicit index counter is not as nice (and more prone to errors) as using the other for syntax. But one could imagine a language with e.g. macros that made the backwards-looping syntax more intuitive (I assume a single-threaded situation and an ArrayList).

Your general point still stands though.

Both implementations are incorrect as they needlessly mutate the input list:

  List<String> namesNotStartingWithB = new ArrayList<>();
  for (String name : names) {
      if (!name.startsWith("B")) {
          namesNotStartingWithB.add(name);
      }
  }

Imagine a scenario where you're filtering one of the arguments to a method: the input will be mutated with no indication to the caller (or in the method signature), causing bugs, iterator invalidation etc.

Good additional point that demands repetition: Don't mutate arguments to a (public) method if not absolutely necessary.

Just to be clear, that does not make the approaches "incorrect". The list "names" is not necessarily an argument to a method. It might be a local, intermediate result that does not have the risk of "mutation at a distance".

I know what you're trying to say, but the problem definition is to mutate the list, removing items that start with B.

How about:

  Iterator<String> it = names.iterator();
  while (it.hasNext())
  {
    if (it.next().startsWith("B"))
    {
      it.remove();
    }
  }

Yes, much better. For anyone interested, here's a link with background information: http://stackoverflow.com/questions/1196586/calling-remove-in...

Personally, I find functional styles to be easier to understand. Transforming data by composing functions is easier for me to understand because each step generally stands by itself, with no state to keep track of across steps. This generally leads to less buggy code.

Also, I find that manually managing control flow leads to a lot of bugs. If I have tricky for-loop with lots of continues and breaks, the complexity leads to subtle bugs. I try to avoid those features as much as possible, deferring control flow to functional composition. In general, there are less edge cases to manage with a functional style (control flow, null collections, etc.).

There are definitely performance reasons to use traditional for-loops, but the vast majority of the time it's appropriate to sacrifice performance for stability and readability.

The next step is to realize that you can use streams to communicate between processes. Stream parallelization is used not just to speed things up but as an easier way to write massively parallel programs.

Instead of bug inducing mutexes and semaphores, you write a series of threads who are only able to communicate between themselves through streams. Each thread possesses input streams to receive data from other threads and output streams to talk to other threads. Each thread is idle until it receives an input from one of the streams. It then performs some processing which can include sending messages to output streams before going back to sleep. A system may possess a large number of threads and

I wrote a pacman this way where interactive object was a thread: the pacman, ghosts, bonuses,... Even the score was in its own thread. Since the game was real time there was a special clock process to generate the ticks to advance through each frame of the game.

Except for the initial wiring up and flow control (when the emitter of the stream is faster than the receiver), the system is easy to reason with and debug. By looking at the stream you can get a high level of visualization.

I guess the next step after that is to realize that you can apply this to the entire system and replace messy one-to-one ESB RPC calls with something like Event Sourcing.

sounds alot like the golang channel "phylosophy":)

Isn't that node.js?

Standard unix programming with streams and pipes is quite functional-ish. The auto-pausing of pipes and lazy infinite streams it makes possible are quite useful.

I was thinking "this is cool" -- a more functional style of iteration for Java. But then it occurred to me that now you will need to know yet another style of writing the same thing. That brought up (bad) memories of Perl, where the problem of "there's more than one way to do it" was that you had to be comfortable with _reading_ all of them. (And, much as I love Lisps, that's the problem with macros in Lisp -- they create a semantic burden on new readers of the code.)

One of the hotly debated aspects of java is that even with java 8 additions the language is fairly simple and there really aren't that many ways to express the same thing.

Hmm... Java's reference manual is about 800 pages long. That is close to Ada's reference manual, and Ada is quite a hard to grok language, and its reference very exhaustive, by design. It is bigger than C++14's ISO reference, which is fun considering Java was marketed as "much simpler than C++".

Overall, Java is not an arcane language, but it has many, many gotchas and mixing new concepts with old design decisions only increases the number of gotchas.

The spec is long because it's very, very detailed (probably more than any other language spec out there). Also, Java doesn't allow any undefined behavior under any circumstance. It is much, much simpler than C++.

There are also lots of awkward edge cases where Java specifies something just as complex as a more general language feature, that's then only used in one tiny case. E.g. Java has a specification of type inference - that's used only for anonymous classes. And another, more limited form of inference that's applied to generics. Java has a syntax for union types - that can only be used in catch clauses - and generic type bounds have their own parallel syntax and rules. Language constructs are inconsistent (braces are mandatory around the body of a try or a function, but not an if or while). A lot of parts of the specification need special-casing for primitive types and/or arrays, which most languages handle in a more unified way (e.g. accessing the length of an array via reflection is unlike any other method or field access).

So what? A language's complexity is measured (by developers) as the mental effort required to learn and read it -- not by the regularity of the grammar. Java is a relatively very simple language (for a statically typed one) -- even with its corner cases. It's certainly more complex than C and a little more than Go, but it's much simpler than C++, C#, Scala, Ada, Rust, Haskell, OCaml and pretty much every other statically typed language out there with more than 1000 users.

These aren't just theoretical grammar issues. Every beginner struggles with when you use == and when you use .equals(), or the differences between casting primitives and casting objects. I've witnessed even very experienced developers get confused reading generic bounds, or fail to take advantage of multi-catch or generics inference. All I can say for your comparisons is that I disagree with many of them.

> Every beginner struggles with when you use == and when you use .equals(), or the differences between casting primitives and casting objects.

I actually agree with those points (though Java is still very simple), but there's an obvious reason: that's the exact same behavior as in C++ (unless you start playing with operator overloading), and any other behavior would have looked surprising or strange to people coming over from C/C++, which was basically everyone who learned Java in its first 10 years. The same goes for the fallthrough switch statement.

Generic bounds indeed completely go against the language's design goals, but failing to take advantage of multicatch or generic inference (or lambda inference) is not a problem[1]. Java is a language designed for ease of reading (it says so right in its design documents), and for large teams/project, so code is made to look uniform, with all "advanced" features being local and obvious to anyone reading them. The goal was to make every new developer on your team (people working on large projects move around a lot) immediately able to read the code and not have to learn the team's DSL or coding style. Those were the problems that plagued C++ (alas, they only became apparent years after C++ had been in widespread use, and cost the industry billions), and Java very successfully avoided. Go has adopted the same design philosophy.

[1]: Not that it's important, but every Java IDE would automatically suggest the shortened syntax. Java IDEs even automatically convert loops to streams.

C++ reference would be longer if it had to define behavior for all the currently undefined cases.

Sometimes you use a macro which creates a lot of things and allows more descriptive programming. The user only needs to understand the descriptive part.

Without it, you'll see lots of machinery and maybe meta-machinery. That can be a much higher semantic burden for code readers.

I agree that macros can hide the gray wall of machinery, but initially, even more descriptive terms can be confusing and unexpected. A longer-term win, but the initial burden goes up.

Interesting approach. It would be nice to see some benchmarks of IntStream vs. traditional loop.

I would expect that IntStreams are slower if it does not use the parallel execution, see: https://stackoverflow.com/questions/22658322/java-8-performa...

And even parallel execution might be tricky: http://zeroturnaround.com/rebellabs/java-parallel-streams-ar...

A coworker and I tested this out when the announcement was originally made, at least in the use case of iterating a massive array. (Insert 100M trues and one false at the end, and find me the false)

Code is here: https://gist.github.com/Karunamon/abc6483ac1d08f6cc137.

The result was that streams were roughly 4x slower.

Still, I really like some of the new constructs that Java is getting - they make the language a bit more expressive, lack of which has always been my main gripe with the language.

Poo. Nice analysis, btw.

Write code to be as clear and expressive as possible, then optimize for performance when you know performance is a problem. This is why I don't mind the performance cost of new language features like this. 90% of the time it won't matter, and you can optimize the other 10.

Sometimes it is the loop that brings most overhead to the processing time (imagine big array of ints, and some simple transformations). In such case refactoring would be easy, but anyway...

That's the mindset that eventually makes everything slow and you can't easily see that 10% anymore because it gets more spread out the more abstractions you introduce.

When loops stop looking like loops, it makes it rather harder to find them.

Aka "flat profile". Using these things in full force requires really banking on JIT doing the right thing (if perf is a concern), which is optimistic. A simple loop is just as readable as these one liners, and carries less risk of not being compiled as tightly.

These constructs don't always make code any easier to understand / maintain. I have played a lot with NetBeans' feature that automatically translate loops with their "functional" equivalent. Sometimes the code was so clever I couldn't understand what was happening.

Sure, code is more compact, but compactness is not an end in itself.

Compactness is more of a side effect and can be a detrimental one. Personally I find that this kind of code is more declarative of intent than the more imperative for loop. By using particular functional tools for the job you're avoiding any possibility of a bug in your looping construct and making your purpose explicit.

I also like how it removes boiler plate to handle different container types making it easier to switch to different ones.

This reminds me a lot of Resharper's refactoring common things to LINQ in Visual Studio. A lot of times I just ran it thinking "oh wow that's a neat way to do it" and then reverted it back because it wasn't a common idiom or it made it much harder to understand.

The same thing is going to happen with Java 8 until the feature becomes less shiny.

I'm really surprised that it isn't a zero cost abstraction, doesn't the Java JIT inline?

I think there's some other secret sauce happening in there, especially since making the process parallelized is as easy as changing .stream to .parallelStream.

HotSpot (the OpenJDK JVM) does, and it should in this case, too, but this usage suffers from "the inlining problem"[1] and/or the profile pollution problem[2]. These are problems that are continuously addressed and improved with each release, but have not yet been satisfactorily resolved.

[1]: http://www.azulsystems.com/blog/cliff/2011-04-04-fixing-the-...

[2]: https://wiki.openjdk.java.net/display/HotSpot/MethodData

I think Graal does more aggressive inlining than Hotspot.

Looking forward to the day it will be in the reference JDK.

I don't think the 100M entries example suffers from inlining or profile pollution. It's just that the manual loop is going to be as tight as you can get it and it's likely the stream version leaves artifacts behind that are noticeable when the loop kernel is dead simple like this.

-XX:UnlockDiagnosticVMOptions -XX:+PrintInlining will tell you whether this inlined, and dumping the JIT asm can be done to see what was actually generated.

Did you see that they're planning on making Graal a plugin of the standard OpenJDK build in Java 9? It's right there: http://openjdk.java.net/jeps/243

I know, but I imagine it will only be fully merged if Project Sumatra turns out to be integrated.

But you'll be able to use it as a 100% Java plugin regardless.

Sumatra is dead, AFAIK. But why would graal depend on sumatra for integration? Bigger challenge is how to bootstrap graal itself such that compilation time and thus time to peak perf isn't degraded substantially.

Really?!

> But why would graal depend on sumatra for integration?

The other way around. Since Sumatra depends on Graal, it being integrated, means Graal also has to be.

Pity, since I learned about Maxime and JikesRVM back in the day, I have looked forward to the day the reference JVM would be meta-circular.

But why do you think Graal wouldn't be integrated? They've already started the process with Sumatra or without.

Maybe with Sumatra the integration would be deeper than just an API, e.g. replacing HotSpot completely and also add SubtrateVM into it.

I don't know, lets see how it turns out.

I don't think you want a production/industrial JVM written in java :).

It's not the whole JVM -- just the JIT. What difference does it make what language the JIT is written in? It is my understanding that if Graal proves itself, it will replace C2 (if not C1 as well).

I was replying to pjmlp's comment:

>Maybe with Sumatra the integration would be deeper than just an API, e.g. replacing HotSpot completely and also add SubtrateVM into it.

There are a few commercial ones. :)

Which ones?

Graal is not a HotSpot replacement. It's a JIT for HotSpot or an AOT compiler for SubstrateVM which is a separate JVM altogether. If Graal matures and proves itself, it will become HotSpot's JIT. And Substrate may or may not become a product regardless.

And project Sumatra -- while cool -- was never a big influence over OpenJDK's plans. Being able to run streams on GPUs is absolutely awesome, but not the number one priority for the majority of Java users. My point is that Sumatra wouldn't have played a significant role in the decision of when to make Graal HotSpot's default JIT.

BTW, you don't even need Graal to be the default JIT in order to support Sumatra, anyway. Graal as a plugin (JEP 243) is good enough for that.

Sumatra was about GPU offload, not so much metacircular JVM. If you look at sumatra dev mailing list archive, you'll see the last email there states it's not in active development. The project appeared to have been driven by AMD, but they may have re-prioritized things.

I also don't think it's currently possible to write the bulk of the JVM in java, if you want comparable performance and memory footprint to Hotspot.

Sumatra was using Graal.

> I also don't think it's currently possible to write the bulk of the JVM in java, if you want comparable performance and memory footprint to Hotspot.

Better check Graal and JikesRVM research papers then.

One reason why reference JDK JIT doesn't get rewritten is the ROI.

Just check how long has taken to rewrite C# and VB.NET compilers while keeping the new compilers 1:1 compatible or the new RyuJIT and the multiple AOT compiler iterations in .NET land.

As already mentioned, Graal is just the JIT compiler, it's not the entire VM. JikesRVM is a research VM, which has different needs/characteristics from production JVMs.

I think Java 9 paves the way to (or actually includes) JIT caching.

Don't think there's anything in 9 for JIT caching unless I missed it - do you have a reference? JIT caching is non trivial problem for Hotspot due to the nature of speculative optimization, so it may take quite some time for that to appear. Having said that, Azul has some form of it in its ReadyNow feature, but I don't know the details.

I think likely some form of AOT will be needed.

Yes there is, the JEP 197 - Segmented Code Cache.

http://openjdk.java.net/jeps/197

This is just the early work to be improved in later versions.

However there are commercial JVMs, like J9 that already do JIT caching.

Segmented Code Cache has nothing to do with JIT caching. Or rather, it has as much to do with it as the existing homogenous code cache.

It's related to Project Jigsaw, but I don't know if it's actually scheduled for Java 9. I assumed the idea is to cache C1 output, not C2. I think I saw it in one of Paul Sandoz's videos. I'll look for it, and if I can't find it, I'll ask Paul.

It depends on which JIT you mean, there are plenty to choose from.

In any case, at least JITWatch, Solaris Studio and Intel Intel Amplifier do allow to look at the generated assembly.

>(Insert 100M trues and one false at the end, and find me the false)

That sounds like arrays benefited heavily from the easy branch prediction.

that's not a good benchmark, it doesn't take JIT warmup into account.

Should have used JMH instead.

http://openjdk.java.net/projects/code-tools/jmh/

Echoing everyone else, Java microbenchmarking is annoying. You should proably use Caliper or whatever anyone else posts.

Of course since I've never used any of them, here's my terrible example with stream beating for loop.

https://gist.github.com/anonymous/2395fb0728e491bc54f5

Warmup Warming up done 9262288 # for loop 6156414 # stream Done

edit: tuning WARMUP_RUNS to something lower (like 500 vs 10k) and for loop wins consistently vs warmup runs at 10k.

If I put on -XX:+PrintCompilation I see some extra compilation output but I don't really understand the output. I assume some of this contributes but there's way more output then I expected tbh

     4929  263       3       java.lang.invoke.LambdaForm$DMH/1581781576::invokeStatic_L_L (14 bytes)   made not entrant
   4929  339       4         java.util.function.Predicate::isEqual (20 bytes)

Yes your example is bad. If JIT inlines through, the code is trivially dead and can be removed. So yes, please use proper bench harness (JMH).

Ya I thought it might mark it as dead but even if I append the result to something like a List and print the list at the end (so it can't just not run the code?), Stream wins. Anyway I'm off to work for today, maybe I'll post in evening.

edit: https://gist.github.com/anonymous/ed0d8f4a5c6553fe8435

Is there some way in this example it could not actually run the code here?

Well, you're not really testing for loops since 5there are other artifacts here:

1) forEach driver method is receiving multiple types, it's not monomorphic 2) you may be hitting OSR compilations 3) for loop may hit range checks on each get() 4) for loop version warms the cache for the stream version and this benchmark is mem ref heavy

So, please try to use JMH to get more accurate picture. And, as mentioned, this isn't really testing for loop vs streams.

If you want new language constructs, why bother writing 90% of your Java code in pure Java to begin with anyway? Use something that compiles to bytecode like Groovy. Profile your app, and write whatever is performance critical in pure Java.

You chose a scripting language as your example. The profiler would probably tell you to rewrite the entire app in Java, which can be tricky because although the Groovy code uses the Java syntax, the semantics are often different. If you use a language for building systems, the profiler would probably tell you nothing needs rewriting.

Iterators are an interesting addition to Java 8, but a bigger change is what they had to do to enable iterators to exist.

What they did was allow interfaces to provide default methods. They then added stream() and parallelStream() default methods to the Collection interface to generate streams and start all of the iterator goodness.

The result is that in Java 8, an interface now behaves like a Ruby mixin. It is a way to do multiple inheritance in a language that officially does single inheritance.

I'm sure there will be some disasters before the Java community settles on best practices for this feature. :-)

The idea that you can replace nearly every loop with functional constructs really clicked for me when I saw that you can use zip in Haskell and each_cons in Ruby to see each element and its successor at the same time.

[10,11,2,3,4,5].each_cons(2).map {|curr,succ| succ - curr } => [1,-9,1,1,1]

That's cool. I once made a Common Lisp macro that iterated and gave you the pred and succ of each list element, but it was a lot more work than this. Although I think implementing each_cons was most of the work.

Or just:

  (-)':(10 11 2 3 4 5)

...in K.

Or:

  (-) prior (10 11 2 3 4 5)

..in Q, if you feel more comfortable with words than symbols.

Just write functional programs in a legible manner for understandability. I've seen people write five functions on a single line and takes me a while to figure out what is going on. They should be on separate lines with a possible tail comment.

I know backward compatibility with Java 5 is important for the clojure world, but I would love to see streams and invoke dynamic implemented in a "Clojure 2.0".

I would love to do it myself, if someone would pay me to do it :)

Streams are an API built on top of the Spliterator[1] abstraction, which basically gives a collection the option of partitioning an iterator into two, allowing parallel traversal of both partitions and thus parallelization via fork/join. Clojure already does something very similar with reducers[2], which are pretty much the same as Java streams, but implementing the reducer protocol to employ spliterators should be rather simple.

[1]: https://docs.oracle.com/javase/8/docs/api/java/util/Splitera...

[2]: http://clojure.org/reducers

    I know backward compatibility with Java 5 is
    important for the clojure world...

Why is that? (I googled but didn't find anything relevant.)

Since Clojure 1.6, Java 6 is the minimum requirement.

I'm sure there is a discussion somewhere, but I expect that there are quite a few people who run Java on commercial J2EE app servers and the like which don't officially support newer JDKs.

Liking that java is getting nice now. But they need to get type inference next too.

It has type inference already. Are you talking about omitting type when declaring a variable, e.g. let/var x = 5 type of thing?

There are some instances of type inference i Java.

1) Diamond operator for generics 2) Type inference for lambdas

Naturally would be good to get even more of it.

Seems to be an unpopular opinion, but I don't like or want type inference. All that happens is instead of seeing

  int result = someFunc()

you see

  val result = someFunc()

Is anything really gained other than saving a couple obvious (and compiler checked) keystrokes? And you've lost the ability to see the type of each variable locally in the function. It's never seemed very useful to me.

Well, what about something like:

    ArrayBlockingQueue<Work> q = new ArrayBlockingQueue<Work>(10)

Would be more like

  BlockingQueue<Work> q = new ArrayBlockingQueue<>(10)

right? You'd use the interface type that you'd want to be working on, so you could replace the ArrayBlockingQueue if needed and be sure you're not using any specific functions.

Still seems better than having a 'val' in there, since it signals exactly what aspect of the object you're using. Plus now the compiler will complain if you change ArrayBlockingQueue to something that isn't a queue.

I just don't really see the benefit - so what if it saves a couple keystrokes? I don't think I've ever run out of keystrokes before... Meanwhile if it saves a single logic error or helps the compiler detect a mistake, then it's saved a lot of real effort.

The problem with excessive verbosity is not the extra keystrokes when writing, it's the increased mental load when reading.

Java could use type alias/typedef, but I doubt Oracle will add it.

For your example, sure.

But it's not uncommon (in my opinion) to write stuff like

    EnumMap<FooEnum, BarClass> map = new EnumMap<FooEnum, BarClass>(FooEnum.class);

which to me ought to read something like

    var map = new EnumMap<FooEnum, BarClass>();

Of course, I don't do a lot of Java programming these days and perhaps my first example is simply broken due to ignorance.

When I wrote something very similiar to that last week, I failed to figure out how to avoid repeating the type signature, and was annoyed.

It's not quite as short as you'd like, but since Java 7 came out in 2011, it's been possible to write:

  EnumMap<FooEnum, BarClass> map = new EnumMap<>(FooEnum.class);

using the "diamond operator" [1]. In fact, my IDE even provides automatic hints to make use of it!

With that said, I do think java has too much pro-long-method-and-variable-name culture.

[1] http://www.javaworld.com/article/2074080/core-java/jdk-7--th...

You can already do something similar, instead you write

Map<SomeComplexTypes> map = new HashMap<>();

When one has functions named "someFunc()", there are bigger problems than being afraid of type inference.

I know this is just an example, but if you actually came up with something more realistic, you would have seen that your issue is actually a non-issue.

People like to bring this example up in .NET world. The usual recommendation is to actually have a type on the left hand side, if the right hand side is a function call.

int result = doSomething();

but

var result = new Dictionary<string, int>();

Type inference really shines once you get into evolved lambdas and generics.

It's payback for saddling JavaScript with algol-style syntax. The wheel turns.

Agreed.

We had a developer here that used 'val' from lombok all over the place and it was irritating. The team made a collective decision to go through and eliminate this from the code base after he left.

I don't want any of this in our code.

Actually, I don't think that can happen. I can't look for source now, but subtype polymorphism (like subclassing) is not compatible with type inference, unlike parametric polymophism (like SML polymophism or Java generics).

cool article! parallel reminds me of C#'s Parallel.For/ForEach loop. Does anyone have experience of the performance of streams vs conventional for loops?

Performance has been briefly mentioned in this thread; if you google you'll also see some empirical results. But really the best you can hope for, all else equal, is same perf as for loop. The JIT compilers have been taught about loop optos for a long time now, and the best you can hope for is the JIT removes the extra abstraction when using streams, but it can be brittle.

Java needs to come out of the Ruby closet.

Not only do I wish for for loops to remain I would like to see the magical looking lambda crap removed.

Reply on news.ycombinator.com