Java JEP 461: Stream Gatherers

Ok, so then what is this discussion about? Java Streams have existed basically forever, as a 3rd party library, but that doesn't matter here in our conversation about Java language features, where 3rd party libraries somehow changes what the language is...

If it doesn't ship with the default distribution of the language, then there's the risk that multiple mutually incompatible libraries emerge or that the adoption by the ecosystem is spotty. Best example: the async runtime mess in the Rust ecosystem.

> Whether it ships with it or not is, however, irrelevant

When talking about projects in the wild, sure. But if we're specifically talking about language features, then "something being a part of the language" is wildly different than "installable 3rd party library".

In the entire time I worked at a web shop with a large Java backend I never found the parallelization to be necessary, and in many cases was outright dangerous. We just had it disabled on high-traffic services (thread limit set at 1) to prevent foot injuries.

Don't mean to detract from the point :-). I love how neat and elegant the feature is.

OpenMP is not a language? You can take any language and add libraries to achieve a specific goal, my point is that Java streams are first-class citizens compared to other matured programming languages that also enable leveraging other aspects of the Java ecosystem without running into the rough edges you will with OpenMP.

Also, that is definitely not the rule? If your problem is parallelizable by nature, parallelism provides massive speedups. Overhead from parallelizing a stream is minimal (effectively nonexistent) now if you leverage Java virtual threads (JDK 21). Additionally, most problems are not parallelizable, yes, but most solutions consist of individual steps and there is a high likelihood that at least one of those steps will benefit from parallelism. Java streams can switch between .parallel() and .sequential() execution on the go. You almost certainly don't want to leverage stream parallelism for most I/O operations, unless you leverage Java's Completeable Futures and Managed Blocking (but any gains here are probably minimal anyways) but the point is that you can leverage them, because streams in Java are a first class citizen.

#pragma omp target parallel for map(to:v1,v2) map(from:v3)

is sure as hell not part of the C specification.

One of the strengths of Scala is also a weakness. They aren't afraid to break things and make changes that are not backwards compatible. The Scala devs have been much more careful about breakage than in the past, so I think they have reached a good compromise between stability and evolution. The masterful execution of the Scala 3 upgrade is an example.

Java by contrast will never clean up the bad, inconsistent or obsolete cruft in the language. If it is really important for you to run ancient JARs on Java 21, then the Java approach is superior.

Yea, I don't imagine native stream parallelism will help when CPUs are already loaded. Presumably you're using Spring or Rx, in which case you probably can leverage reactive streams and/or Managed Blockers, but thats really just taking advantage of async patterns and not necessarily parallelism. The only case I could envision having a concrete benefit is if you used your own fork-join pool leveraging virtual threads instead of the global fork-join pool to prevent platform threads from hogging CPU, and then used reactive streams that leveraged the virtual threads. Although this would (theoretically) raise responsiveness, it would almost certainly come at the cost of throughput. All that is to say, parallelism generally only provides as much value as you have idle CPU cores.

It is useful, but it also has weaknesses. For example, I’ve lost count of the number of times I’ve seen someone forget to close() an IO-based stream (eg File.lines). But probably for 99% of cases you could get away with slurping the entire file into memory and returning a list. The streams API is optimised for the 1% of cases that need the additional complexity, at the expense of worse ergonomics for the common case.

I get why it ended up that way. I think it would have been difficult at that time to get traction for adding FP features just as a convenience. So they needed to do all the parallel stuff to justify it. But, as I said, anyone I see doing serious parallelism in Java is not using the streams API.

When I was introducing a team to Scala I wrote everything in a more verbose style just to make it clear, e.g.

  val num = words.count(word => word.nonEmpty)

The "bird droppings" (particularly the clever uses of _ found across Scala) were hard for me to grasp when I first started working in the language. Now they're second nature, but I try to remain cognizant of future maintainers who may be coming in to make contributions or bug fixes without first marinating in the language.

! contradicts a boolean. _.isEmpty() is a lambda, applying the isEmpty function on its string parameter. It’s just the two things together.

The nice thing is you can just write it like `x => !x.isEmpty` if you want

What is hard to understand about it? _ is universally used as a placeholder, it also usable in pattern matching. Scala can be used to make some perl-y <!! operators, but it is not like that at all, and makes for very readable, understandable code.

Firstly, I put multiple bug reports there to show that 'real programmers' are running into real bugs in the wild. This is necessary because otherwise when I point out bugs, I'm dismissed as a "PL fan" whose ivory tower views don't line up with the mainstream. See elsewhere in the comments.

Secondly, if I click into my first link (2015), it says resolved 2015. And by 'resolved' it means 'duplicate but not fixed'. Because when I clicked into its duplicate, it wasn't 'resolved' until 2018. What did they resolve if tickets are still being filed in 2019 and 2021?

The last three are open, open and won't fix. Not so promising for the flatMap method.

> Yes, some aspects require fixing (e.g. the interaction of lambdas with checked exceptions; there's much we can do there, but we need to design this carefully)

I do like the Java platform in general, but this sort of argument from hypothetical future Java fixes strikes my as incredibly disingenuous. People aren't programming in a hypothetical future Java.

I can literally pose no argument against your imagined solution to this problem without very specific details. (Implication being, it's a straw man at best.)

And saying saying stuff like (in an adjacent thread):

> You're assuming that deconstructing and exhaustive patterns will continue to be restricted to ADTs only. That may or may not be the case.

Is just complete fiction. (EDIT: Are you going to solve the halting problem, or are you going to provide details before just promising "it'll be ok"? I assume the Halting Problem is out of the question, but what's the plan, then? Details, pls)

Either point to extant JEPs or explain in detail what the exact plan is, please.

Either way it feels like optionals are going to be deprecated real quick now we have sealed interfaces and record matching. They let you get rid of so much of the unpleasantness in dealing with this API. Instead you can

    public sealed interface Maybe<T> {

        <T2> Maybe<T2> map(Function<T, T2> mapper);
        <T2> Maybe<T2> flatMap(Function<T, Maybe<T2>> mapper);
        T elseGet(T fallback);

        record Just<T>(T value) implements Maybe<T> {
            public <T2> Maybe<T2> map(Function<T, T2> mapper) {
                return new Just<T2>(mapper.apply(value));
            }
            public <T2> Maybe<T2> flatMap(Function<T, Maybe<T2>> mapper) {
                return mapper.apply(value);
            }
            public T elseGet(T fallback) {
                return value;
            }
        }

        record Empty<T>() implements Maybe<T> {
            public <T2> Maybe<T2>  map(Function<T, T2> mapper) {
                return new Empty<>();
            }
            public <T2> Maybe<T2>  flatMap(Function<T, Maybe<T2>> mapper) {
                return new Empty<>();
            }
            public T elseGet(T fallback) {
                return fallback;
            }
        }

    }

    public void demo() {
        Maybe<String> foo = new Maybe.Empty<String>();

        System.out.println(
            switch (foo) {
                case Maybe.Just(String val) -> "Hello " + val;
                case Maybe.Empty() -> "Fine, leave me hanging";
            }
        );

    };

But Java is a mainstream language, and so we generally try to adopt features that a large majority of programmers are ready for, which means, pretty much by definition, that we adopts things later than most languages. Just to give you an example, we only recently added algebraic data types, but in some ways we still added them too early because far too many programmers don't yet know how to work with ADTs, but we felt we had to add them because we thought they're the best way to work with simple data in a language like Java, especially since we didn't want to add property or Kotlin-like data classes, as we think these are bad features. This emphasises another point, and that is that we're not "catching up". Most features don't prove their worth and will never find their way to Java, because we try to pick the minimal set of features that will give the most bang, and that reduces the growth of language complexity, which can truly harm adoption. In this way we also managed to avoid async/await.

Another example that people often ask about is named and default parameters. It looks like lots of languages have them, but the problem is that it's usually implemented in a way that harms those languages. For example, C#, Kotlin, and Swift all generally try to support separate compilation and binary compatibility, but their named & default parameters break that. We'll only add the feature to Java once we know how to do it right (something that I think no other language that cares about separate compilation has done yet), and we have some ideas.

And I disagree that FP features are awkward to use in Java (well, some pure-functional theorems don't work because of null, but they also don't work because of reflection, anyway, and they don't work in most non-pure-FP languages, either).

Since Java 9, new features are finally coming again. However, the ecosystem had gotten reliant on certain JDK internals being accessible and stable, while in fact they were neither. That will require some more years to properly transition over.

The only feature that really got obsoleted is the Security Manager and the underscore variable name. Primitive wrapper class constructors are on the chopping block next. Most other things are merely being restricted to harden the platform, with ample time to prepare.

Edit: LTS hopping is probably an unwise strategy as one will harvest only disadvantages: few updates, no performance improvements, no new features, and complicated migrations when it is time to upgrade. One best upgrades as soon as the new version becomes available (as people did before Java 8) or stays on LTS until the software is retired.

The latest Mockito works with Java 21; you'll just see a warning about the usage of the bytebuddy agent. In the future, you'll need to explicitly approve the loading of agents (you can call it "magic"), which will improve the security of your programs.

> Unfortunately, since Java 9 the rate of change seems to have increased a lot.

Since Java 9, they don't introduce changes, they introduce new features which is a bit different.

IT is a field that (should) attract people that are for fast paced environments, not old school slow-to-innovate industries.