Background: how we got the generics we have (2020)
cr.openjdk.java.net
cr.openjdk.java.net
Decades later, we're still wondering as we get to more and more complex use cases, why we're stuck with passing the class object to all our generic types just to know who we are. It's really a rare instance where Java gave an engineer's solution when they should have given us something more, like a compile flag, a fork, a GenericArrayList<Something> that would be reifed, anything, rather than excuses about ecosystem, backward compatibility perfection, etc.
I understand why they took that decision, I don't understand how it helps all of us do meaningful generics :(
Another way to state this requirement is: it was not considered acceptable to orphan all the code out there that could have been generified, or make developers choose between generics and retaining the investment they’ve already made in existing code. By making generification a compatible operation, the investment in that code could be retained, rather than invalidated.
It's great to see such care for the users, while at the same time improving the platform to make it one of the top performing ones.
There’s a bunch of ways to do this, but I’ll just leave this one trick here.
Instead of:
List<Integer> x = new ArrayList<Integer>();
write this: List<Integer> x = new ArrayList<Integer>(){};
Now you’ll be able to ask at runtime and determine it is a subclass of List<Integer>.So, instead of doing that, they have decided to devalue Java code in long term, by deliberately making language and runtime less useful. Interesting choice.
It will allow (almost) direct control over data layout. At least make it so that it is cache/prefetch/SIMD friendly.
Here is the JEP: https://openjdk.java.net/jeps/401
I have a List<Integer>. Maybe I want to replace List with something else, call it L.
Can I write L<Integer> in Java? No. Rust? No. C#? No Kotlin? No.
Does Go have generics yet? Once it does, I'm sure it'll be a No as well.
Haskell and Scala got it right.
def process[L[_]](input: L[Int]): Unit = ???
Where `L[_]` could be any type that has a type parameter, e.g. `List[_]`, `Option[_]`, `Future[_]` or `Either[String, *]`. This opens the door to a new level of abstraction where other languages must resort to code generation or similar ad hoc-like solutions. <L> void process(final L<Integer> input) {
}
------------------------------------------
<L> void process(final L<Integer> input) {
^
required: class
found: type parameter L
where L is a type-variable:
L extends Object declared in method <L>process(<any>)public <T extends Collection<Integer>> T process(T collectionOfInteger) { return collectionOfInteger; }
I dream about "implicit" interfaces in Java like in Go. So everything that has "flatMap" would implicitly implement "FlatMappable" or something like that.
S<U> map(S<B> b, (B -> U) func) { //stuff}
Which allows you to do things like: Optional<Integer> i = 4;
(Integer -> Float) func = (i \* 2.5);
Optional<Float> f = map(i, func);
This ability becomes much more powerful when you're parsing some file. Imagine you're parsing some JSON of people, and you're using https://schema.org/Person as the type. So the JSON contains things like additionalName, affiliation, birthPlace, email, gender, image, etc. and each of those are typed respectively - a Name class, Affiliation Class, Location class, Email class, Gender class, Image class, etc. Now imagine that the service you're calling may fail to return certain values.In java, it's idiomatic for libraries like Gson to return null on the fields that aren't returned - so if you try to get a person who doesn't have an additionalName, the additionalName field in the Person object will be null. It would be nice if it could be an Optional<AdditionalName> instead of just an AdditionalName to force the null check, but then the library becomes extremely painful to use. You would end up having to call additionalName.get() whenever you just want the name. If you have HKTs, then you can operate on the fields and ignore the fact that they're technically Optionals.
It's also generally recommended in Java to not use Optionals in fields in classes, or to express the optionality of a value in a constructor. If you do, you have to wrap values in Optionals to make the object, which harms readability.
HKTs can become even more powerful with types other than Optional/Maybe. Either, Monad, Future, Comonad, IO, Functor, Applicative, Task, etc. You would be surprised how many design patterns in Java end up being just a simple combination of these types that are much more ergonomic with HKTs.
Something like the following Scala 3 code is closer to the actual intend, I think:
@main def entryPoint =
trait ConsoleLogger[W[_]]:
extension[E] (w: W[E]) def log(): Unit
given ConsoleLogger[List] with
extension[A] (list: List[A]) def log() = println(list)
given ConsoleLogger[Some] with
extension[A] (some: Some[A]) def log() = println(some)
def process[E, L[_] : ConsoleLogger](input: L[E]): Unit =
input.log()
process(List(23))
process(Some("'process' is generic!"))
//process(None) // Does not compile — even Some and None are interchangeable under subtype polymorphism
Here process(input) can work with any wrapped input as long as a ConsoleLogger instance is given for a specific wrapper type. The point is: There doesn't need to be any relation between the wrapper types (like in the example with List and Option.Some), only the "shape" needs to match W[_] (which could be actually also adapted with type lambdas in case it doesn't match, but not going into this here).For example, there is a typeclass Foldable[F[_]] which provides the ability to "fold" (i.e. recurse over) some type F[_] which takes a type parameter. Then you have typeclass definitions for Foldable[Set], Foldable[List], Foldable[Tree], Foldable[Option], etc.
Then if you have an operation and you want it to work on any collection that can be folded, then you can just define it in terms of a generic type that has a Foldable typeclass. For instance to generically sum a collection of ints, you might do
def sum[F[_] : Foldable](ints: F[Int]): Int = ...
Now you can call sum(List(1,2,3)), sum(Set(1,2,3)), sum(Option(1)), etc...
You want to fold over Options? Make a FoldableOption class implementing Foldable<Option> and pass "new FoldableOption(myOptionValue)" to whatever places you need. Haskell does pretty much the same (unless they moved away from dictionary passing).
public <T extends Collection<Integer>> T process(T collectionOfInteger) { return collectionOfInteger; }
And basically your T will be as wide as the interface/class it extends in the generic parameter.
I guess the most Java way to do that would be using Iterable:
public <Z, T extends Iterable<Z>> T process(T iterable) { return iterable; }
public <Z, T extends Iterable<Z>, R> T<R> process(T, Function<Z,R>)
change the type parameter of T<Z> to T<R> and return an instance of T<R>, but the best I can do is return an instance of Iterable<R> because I cannot define T to have a type parameter.
public <A, B, T extends Iterable<A>, R extends Iterable<B>> R process(T input, Function<? super A, ? extends B> transformer,
Function<Iterable<B>, R> helper) {
return helper.apply(StreamSupport.stream(input.spliterator(), false)
.map(transformer)
.collect(toList()));
}
@Test
public void testProcess() {
Iterable<Integer> ints = process(Arrays.asList("1", "2", "3"), Integer::parseInt, identity());
System.out.println(ints);
}
Outputs nicely integers nicely converted from strings:
[1, 2, 3]:-P
`identity()` is a static import of `Function.identity()`
class SpecialIterator<R> extends Iterator<R>{}
SpecialIterator<String> strings =...;
SpecialIterator<Integer> ints = process(string, magicFunc);
The Iterator example starts to break down, so here's a more concrete example:
I have 3 classes: AbstractTester, StringTester, and ListTester
class AbstractTester<R, T extends AbstractTester<R, ? super T>>
class StringTester<R> extends AbstractTester<R, StringTester<R>>
class ListTester<R, T extends AbstractTester<R, ? super T>>
I want to make a method in AbstractListTest like this: public <Z> T<Z> convert(Z z)
I have supporting code in my classes to create an return an instance of type T<Z>, but I cannot express the type T<Z> in the type system. The type parameter T can extend something with a type parameter, and be passed a type with a type parameter, but it cannot be defined to have a type parameter itself.
This is for a fluent api framework, where I need the compiler to know the type T<Z> without casting
I can do something like this: public <Z, Y extends AbstractTester<Z, ? super T>> Y convert(Z z);
but that will return an AbstractTester<Z> which can be cast to a StringTester<Z>, but the compiler does not know it is a StringTester<Z>
A subclass of AbstractTester is not required to have any type parameters itself. Converting a T to a T<Z> would not be legal for a T without a type parameter. To do what I want, Java would need to add a new type bounds operator to restrict a type parameter to subclasses with a particular type parameter.
Java would need to allow me to define ListTester like this to restrict T to only subclasses with a type parameter:
class ListTester<R, T<_> extends AbstractTester<_, ? super T<_>>
I'd love to be wrong, and would be very grateful to find a way to do this.
baz(bar(foo(t, f)));
T<A> should go into foo(..) and come out (e.g. as T<B>) such that I can then call bar(..) on it.But in this case it leaves foo(..) as R extends Fooable, which is not something I can pass into bar(..).
foo.doSomething() // returns Iterable.
.doSomethingElse(); // Iterable doesn't implement doSomethingElse();I'm curious what your were working on where you ran into this?
* At the same job, a coworker expressed a desire to rewrite it with Observables.
* At a different job, a different coworker rewrote a chunk of code from Observables into futures.
* At my current job, we use micronaut, which has announced it's switching from rxjava2 to project reactor.
In the above cases, having interfaces would really help in changing the code piece-by-piece. And the rewrites wouldn't be necessary if they were behind interfaces.
In the general case I just want to program to the interface, and not pin my business logic to any particular implementation.
public <Z, T extends Collection<Z>> List<Z> process(T collectionOfInteger) { return collectionOfInteger.stream() .collect(toList()); }
These are the three implementations I want to hide behind a single abstraction:
// java.util.Optional
public <U> Optional<U> flatMap(Function<? super T, ? extends Optional<? extends U>> mapper)
// java.util.concurrent.CompletionStage:
public <U> CompletionStage<U> thenCompose(Function<? super T, ? extends CompletionStage<U>> fn);
// java.util.stream.Stream:
<R> Stream<R> flatMap(Function<? super T, ? extends Stream<? extends R>> mapper);
Any suggestions? public <T, Z, F extends Function<? super T, Z>> Z apply(Function<F, Z> target, F fn) {
return target.apply(fn);
}
@Test
public void test() throws Exception {
Stream<String> stream = Stream.of("foo", "bar", "baz");
stream = apply(stream::flatMap, s -> Stream.of(s + "_a", s + "_b"));
System.out.println(stream.collect(toList()));
Optional<String> bean = Optional.of("bean");
bean = apply(bean::flatMap, b -> Optional.of("soy"));
System.out.println(bean);
CompletableFuture<String> future1 = CompletableFuture.supplyAsync(() -> "hello");
CompletableFuture<String> future2 = apply(future1::thenCompose, string -> CompletableFuture.supplyAsync(() -> string + " world"));
System.out.println(future2.get());
}
Output:[foo_a, foo_b, bar_a, bar_b, baz_a, baz_b]
Optional[soy]
hello world
I'm trying to figure out how to refer to flatMap directly from the abstract function, e.g.
flatMapTwice(x, fun) {
return x.flatMap(fun).flatMap(fun);
}
or chain(x, fun1, fun2) {
return x.flatMap(fun1).flatMap(fun2);
}
It should also preserve the type information (just like you did above). I don't want the caller to need to downcast from FlatMappable.It doesn't need to be x.flatMap(f) - I'd be interested seeing a flatMap(x, f) version.
public interface FlatMappable<T> {
<U> Optional<U> flatMap(Function<? super T, ? extends Optional<? extends U>> mapper);
}
public class FlatMappableOption<T> implements FlatMappable<T> {
private final Option<T> value;
public FlatMappableOption<T>(Option<T> value) {
this.value = value;
}
<U> Optional<U> flatMap(Function<? super T, ? extends Optional<? extends U>> mapper) {
return value.flatMap(mapper);
}
}
// similar wrappers for CompletionStage and Stream
Now whenever you want to pass a Stream/CompletionStage/Option to something that accepts FlatMappable, you either a) already has it wrapped in FlatMappable, so pass that wrapper in; or b) has a naked value of known type, so wrap it in the propper wrapper class and pass the resulting wrapper in.Of course, it would be nice to have this wrapping done automatically instead of manually, but that's pretty much how it's actually implemented behind the scenes (dictionary passing, or look at Go's way to wrap things inside interface{}).
[1] https://cr.openjdk.java.net/~jrose/values/parametric-vm.pdf
And the VM can specialize on any values, not just types.
[0] https://projectlombok.org/ [1] https://github.com/manifold-systems/manifold
For example, if you want to automatically create getters and setters to all private variables, why not make them public in the first place?
I hate when tools mess with IDE. What about guys using Emacs (or any other editor that does not understand Lombok)? I think generating code from IDE is fine as well as having code generated from build system distributed along with code, but having that code generated through magic of annotation processor is not as nice. I also suspect that Lombok messes my IDE on a deeper level but I have not yet a good concrete proof. Recently I am more and more frequently finding situations where Idea messes up understanding correct types of things and it takes for it some time to catch up (and sometimes reloading the file).
On the other hand I love small things like creating builder or logger. Which is to say these can be just as easily generated from a template, no annotation processor needs to be involved.
I think main feature of Java for many years were its IDEs which would allow to run accurate refactorings or always understand who is using particular piece of code, what is exact call hierarchy, etc. I feel loosing that is going to make Java much less useful for me when I try to analyze large project or run huge refactorings.
If you're in an old school "OOP" style java codebase, where each class is its own little island of encapsulated state and behavior, the coupling of the two means that strictly controlling access via getters is usually required.
Even in more "modern" java, where you do most of your programing against plain data, having Lombok attach getters still gives you some quality of Life benefits because you can use method reference rather than anonymous functions for high-order access (e.g. `things.map(MyClass::getName)` rather than `things.map(myclass -> myclass.name)`)
I'm with you on mixed Lombok feelings, though. It's great right up until its not. It fails or doesn't work in really unexpected or weird ways, and is the source endless pain when you've got other annotation based things (like dagger) which then imposes annotation processing order failures.
That I agree. My comment was meant to be cheeky a little bit as it is actually pretty difficult to find trully OOP application, at least in area of backend apps I am working in.
Obviously, in OOP you are not supposed to expose your internal state but rather accept messages to run behaviour.
If you are working on a truly OOP application then Lombok is useless. Your public class interface is all that matters then and you would spend more time fighting Lombok than just writing it manually.
Lombok is only good at automating boilerplate which, if you have a lot of, is a sign of some other problem.
> It's great right up until its not. It fails or doesn't work in really unexpected or weird ways
That is exactly the point. Magic is fun until you find out that it leaks horribly and causes unintended side effects with all sorts of stuff.
What's the point of replacing simple problem (just really use templating mechanism you have in your IDE) with complex problem (dealing with magic failing on you).
Because in future I may want to add some validation there, or processing, like sending events on set of a value, etc.
Properties in C# was a genius move that solves it correctly.
But origin of it is in EJB specification, it is not something that would be interently needed elsewhere. It just "feels wrong" to programmers (including me) at this point.
https://stackoverflow.com/questions/42644923/eclipse-with-lo...
It is one of those things that does not much useful, but takes away only little bit so there is no reason to fight it too much.
If I would want to use something like Java for a project, I would always pick C# which is a better Java in all aspects.
Java with IntelliJ is awesomely productive. You barely have to type anything beyond new variable / field names, everything else is generated or tab completed. Refactoring is a breeze.
That, times 'worse is better' - it's the language and toolset I already know for the types of things it's good at. I have no doubt C# is comparable or better at most things, but probably not so much so that it is worth switching.
They have a C# IDE (Rider) now, based on IntelliJ. While it was originally the only non-Windows C# IDE, I and others have switched to it from Visual Studio (which is almost as good) even on Windows.
> I have no doubt C# is comparable or better at most things, but probably not so much so that it is worth switching.
I’d say you are right. Now. 10 years or so ago (6/7/8 times) I’d have disagreed, C# avoided many of the mistakes* Java made and was able to become far more pleasurable to use, it took some time for Java to catch up again.
*: Mistakes only in hindsight
Rust is better than C
C# is better than Java
You can talk about tooling, environments, how many people use it, what you can use it for, but when it really boils down to the language there are languages that are better than others while trying to achieve the same.
I mean I use Elixir for other things than Swift or Rust or F#, Elixir is the best language for highly concurrent "message passing" systems. Some features of Elixir that I don't immediately like like it's type system unlock possibilities that no other languages have. But I don't see anything like that in Java, unless I really look around in 1993 and the only competitors are C++ and nobody-uses-it-yet Python.
My little pitch when starting a migration project in a distressed no-type language team is to make them read the code and explain it, while asking about the type of objects they use and show them when they read it's their brain parsing it with high effort. If they de-optimize and just write the types more often, it becomes much clearer longer.
You can type usually in those language but often people want to use the "proper way" as shown in ultra simplistic examples in manuals for sales pitches and end up thinking the right way to code is to write as little as possible (combined with the no-comment belief they have but don't map to "no comment because clear enough code").
That's not the case for Kotlin, that has the full weight of JetBrains behind it.
I guess that's what I see as one of the advantages of Groovy. You can do ultra rapid implementation of something with minimal code and then as it becomes more critical you increase its type safety by adding first Groovy type hints, but further along that curve, convert it fully to Java or another statically typed language. So what you describe as a problem I see as sort of, optimal workflow, and the real problem is being forced to universally choose one or the other in other languages.
If you were trying to write portable C++ code in 1996 you would get it, specially when using each OS specific C++ compiler provided by the platform vendor.
#ifdef _AIX
...
#else ifdef SGI
...
#else ifdef WINDOWS
...
#else // everything else
...
#endif
Shudder.But the other thing that Java had is the library. Compared to C++'s standard library in 1996, it was a revelation.
Things like GC, reflection, runtime loading of code, that were very familiar from more academic languages at the time.
I also did a lot of C# to help on a GUI optimization project (because doing low latency if the GUI freezes every 10 minutes isn't great), and I was really surprised by how good it was so I wouldn't disagree it works nice.
BUT I like that Java works simply in linux, I like that it didn't yet got contaminated by all the crazy constructs from elsewhere (Groovy grr), I love the tradition around the syntax (I can't stand C# quirks sometimes). I do HATE with passion the meaningless generics, and I wish we'd ditch backward compat to steal it all form C++ templates.
I enjoy more than the language syntax itself that is close to most languages I like (Javascript, C#, C++), the team spirit around it. Believe me, it's much easier to use and understand maven (or maybe gradle who starts to get dangerous in real world projects where you can't afford 2 weeks of reverse engineering a madman's build script) than it was nugget, it's easier to use IntelliJ than to receive yet another stupid email from compliance about a Visual Studio license missing, it's nicer to build a jar and run it than to go through all the kinks of exe files... I dunno I may have started from Java and got to C# and done the opposite of your journey, but while I agree C# has superior features in some aspects (but sometimes they chose mess vs safety) I just wasn't convinced it could work for something more critical where you must do things as simple as you can, as clear as you can.
And let's not even start on C++ who's superior in every conceivable way except that it encourage, or doesn't discourage, every team I ever meet to create unbearable messes they can't fix, so focused they are fixing problems we can't even imagine in Java for reasons we can't comprehend our company even accepts to finance :D
- primitives or any custom value types/structs can be used. No need for boxing or method duplication for primitives. - type information available at runtime, no need to pass 'Class<T>' parameters. - type constraint 'new' available, which means it must have a default constructor so you can do 'new T()'. No need to use reflection or pass a constructor parameter. - type constraint 'notnull' available - class-scoped compiler enforced covariance and contravariance, though I think you can get similar results with java's "? extends T" and "? super T" constraints
return new T();
is possible in C#, not in java.
Definitely not pleasant though.
I'm sure it's me but do you use it yourself and why ? Don't you get lost in the indirections in big projects "oh shit where is that thing again that auto inject itself here" ?