infoq.com
I also like the easy parallelization functions, though as the article indicates they're not suitable for every use case.
slf4j is the current solution for this kind of problem, a common interface to various different logging solutions.
"For me, starting a new project was a lot worse than just "disheartening". The SLF4J vote was just the straw that broke the camel's back. After putting many many hours of work into log4j, it became increasingly painful to waste time in arguments, where opinions got asserted by the one writing the longest email. Not fun."
- http://mail-archives.apache.org/mod_mbox/logging-log4j-dev/2...
Serious question: isn't this something the JVM could abstract away?
Returning to the concept of parallel streams, it's important to note that parallelism is not free. It's not free from a performance standpoint, and you can't simply swap out a sequential stream for a parallel one and expect the results to be identical without further thought. There are properties to consider about your stream, its operations, and the destination for its data before you can (or should) parallelize a stream. For instance: Does encounter order matter to me? Are my functions stateless? Is my stream large enough and are my operations complex enough to make parallelism worthwhile?
The author of the linked InfoQ article (OP) cites that same dilemma by explaining the potential for context-switching overhead to counter the advantage of splitting the work.
Abstracting it away with rough heuristics might be possible, but doing so with consistent success could be challenging. In other words, you could elect to use the serial algorithm for small collections, or when the CPU contention at the start of the sort operation is low. But if the comparison operator is expensive, CPU contention is volatile, or if operating on a stream of unknown length, the abstraction may choose poorly. Ultimately, I like the option to choose for myself, but like you, I wouldn't mind having a third option that defers that choice to some heuristic.
[1] http://www.techempower.com/blog/2013/03/26/everything-about-...
Arrays.sort(arr, Concurrency.LEVEL)
GPU should be interruptible the same way the CPU is, so if the GC decides to move the memory it can actually do so. The memory can be pinned instead, though. The latter poses some side effects with the GC. If the GPU is not on the same die it will have to virtually copy the array as the L1/L2 caches won't be accessible.
Arrays.sort(somePrimitive[]) would be too much of an edge case to optimize for. Overall hard nut to crack. Java8 streams and direct buffers, however, could be a good starting point to perform various operations via the GPU.
Disclaimer: I am really not well versed in the GPU tech.
> This continual creation/termination/destruction of threads is done so often that I wonder where the idea came from. I presume some poisonous textbook is responsible. Sometimes, it seems that the whole SO is riddled with threads that add two integers and then terminate, just so that the 'main' thread can wait with 'join'. God help us :(
I hope parallelSort() just calls sort() if the array is smaller than some threshold.
ForkJoinPool forkJoinPool = new ForkJoinPool(2); forkJoinPool.submit(() -> // write your parallel query here ).get();
If you did it often, I guess the JIT could work out what was best the last time few times. Trial and error. :-)
Unfortunately the legal fight between Oracle & Google has really made me doubt that the cool new language features will be coming any time soon to Android.
My beef with Go is weak web-programming and prototyping support and immature libraries. I'm surprised there's been so much uptake in the Ruby/Python communities, which are very strong in those areas, but I guess Ruby/Python are now being used outside their original domains and those use-cases are pretty ripe for a language like Go.
That would be a step backward, we really need generics and a reasonable support for exceptions on Android (or any large scale system, for that matter).
Go is a fine language at the system level or to replace Ruby/Python scripts but that's about it.
Optional is another nice addition, I've been creating one myself in all Java projects I've created since I first encountered it in Rust.
If you want code that won't compile due to @Nullable violations I think the Checker framework can give you that, or you can use an IDE that flags violations like IntelliJ and just treat any static analysis warning in that category like a compile failure for your own purposes. The nice think about nullity annotations is that the newest JVM languages like Ceylon and Kotlin are building nullity into their type systems, so if you annotate an API in this way, code written in these new languages will know automatically that the reference can be null and the compiler won't let you access it at all until you tested or asserted away the nullness. The upgrade path for Kotlin especially is looking like it could be quite strong, so I think I'll be sticking with @Nullable for now in the hope that later on we get "real" type system integration via newer languages.
public ImmutableMap<String, Long> getSessionTime(HttpServletRequest request) {
return Optional.ofNullable(request.getSession(false))
.map(s -> ImmutableMap.of("lastAccessedTime", s.getLastAccessedTime()))
.orElseThrow(IllegalStateException::new);
}
Session s;
if ((s = request.getSession(false) != null)
return ImmutableMap.of(....);
else
throw new IllegalStateException();
public ImmutableMap<String, Long> getSessionTime(HttpServletRequest request) {
return request.getSession(false)
.map(s -> ImmutableMap.of("lastAccessedTime", s.getLastAccessedTime()))
}
Of course this is a trivial example where the programmer is likely expecting null, but many times null can be returned and it is not always clear.
[1] https://code.google.com/p/guava-libraries/wiki/UsingAndAvoid...
Due to backwards compatibility concerns, no existing Java APIs that may currently return null can ever be changed to return Optionals instead, so nulls need to be dealt with regardless. Also, there's technically nothing stopping an Optional value from actually being null itself, or from developers calling Optional.get() without checking isPresent(), just like they might currently use a reference without checking for null.
I personally really wish jsr305 had been adopted in Java8, and their existing APIs retrofitted with these annotations.
http://www.oracle.com/technetwork/articles/java/java8-option... describes Optional, and compares it to how in Groovy, the safe navigation and elvis operators allow you to write code like this:
String version = computer?.getSoundcard()?.getUSB()?.getVersion() ?: "UNKNOWN";
String version = computer.flatMap(Computer::getSoundcard)
.flatMap(Soundcard::getUSB)
.map(USB::getVersion)
.orElse("UNKNOWN");
New languages like Kotlin solve this much better in my opinion by introducing nullable vs nonnullable types. While Java can unfortunately never adopt this due to backwards compatibility issues, @Nullable and @Nonnull will do most of the time, and hopefully we'll see operators like ?. and ?: in future versions of Java.
With option, you clearly declare what is safe, and what isn't, and the compiler won't let you screw up. You can decide which layer of your code handles the empty case, and lots of unnecessary ifs go away from the bytecode itself, so the app will run faster, and you know when you have to even consider the null case.
Now, doing that in Groovy is rather silly, because your language is dynamic and your types are optional, so all of this compile time safety would not provide any value anyway. Option is just the way you'd solve the problem in the strongly typed way. It's how you handle it in Scala, for instance, and how Haskell would deal with it.
As far as using flatmaps and maps to work with optionals, yes, it's a chore. I'd argue it's borderline abuse of the stream operators, as treating Option as a collection is ugly. That said, putting yourself in a situation where you chain 3 optional getters is also object design from hell, so one should avoid putting themselves in that position altogether.
In Scala, instead of flatmapping single optionals, we often getOrElse(), or use pattern matching as an extractor. Now that's a feature I would like to see Java 'steal' from the Scalas and Erlangs of the world, but I do not see that happening.
The elvis op is called the null coalescing op in other languages. [1] Groovy's promoter chose the elvis name to fit in with marketing the Groovy and G-String names.
PHP also uses the ?: symbol but other languages use different ones, e.g. C# uses ?? and Perl uses //
I think you can define the types to get around it, but that is messy and a PITA. For example, class OptionalA extends Optional<A>.
The major pain point with generics is that they don't work with primitives.
Is there a particular use case?
There's no equivalent of scala's orElse(returning a second optional if the first one is null) and since default methods while useful don't allow adding things to types you don't own you can't use an method like notation for the best you can get is static importing and wrapping a function around it). ofNullable(creating an empty Optional from a null instead of throwing) isn't the default(of which throws is). Also Optional isn't an Iterator which is disappointing since using it with a for loop can be useful if you want to avoid nested closures.
But worst of all it's like they haven't started slowly deprecating most of the standard library(anything that makes sense to pass null to or that it makes sense to return null in some cases according to the api. Option types are arguably the best solution for the null problem at least for statically typed languages but having null still be possible while having Option is arguably the worst of both worlds, in my understanding scala deals with this by almost never using Null except for java compatibility and tries to create Options as soon as possible if calling a java function that may return null.
Basically I think they need to make some sort of standard Nullable and NotNullable annotations, add it to everything in the std library, and have some sort of package annotation that tells the ides to check every call to a Nullable and bug you to wrap it in Optional.ofNullable. Then deprecate those methods with Nullables by java 10 or 11 with strong warnings for ths who call them(or possible forcing a special compiler flag/package annotation to use them).
Even not doing that they are still adding methods that return null(such as Hashtable::computeIfAbsent) instead of Optional. Why for fcks sake?
For most people StampedLock will remain quite a mystery as they are harder to use and the vact majority of Java developers doesn't actually write (or even use) low-level concurrency primitives.
Isn't that the jsr that designed StampedLock and decided to add it to Java? If so, I would hope it was talked about extensively, but I wouldn't see such talk as a counterexample.
My company is running it on production with no issues.
It could check for null in its setValue method, at write-time - more useful than discovering it read-time (though I'm not sure it's worth the abstraction).
http://www.amazon.com/Effective-Java-Edition-Joshua-Bloch/dp...
Is it no one is talking about new Java features is because Java isn't the hot new web stack or JS library? Or because most big Java users move to new things with the speed of continental drift?
We're quite a bit slower at migrating existing applications (except for security issues), but because we code defensively, we are generally backwards compatible.
If I get this right, the source code of the signature of many (many!) methods will now be twice the size of before, unless the names of your types and variables were already considerably bigger than "Optional<...>" (10 chars).
I hope it's worth this cost.
X.getA().getB().getC().getD()
X.flatMap(X::getA)
.flatMap(X::getB)
.map(X::getC)
.orElse("UNKNOWN"); //[1]
[1] http://www.oracle.com/technetwork/articles/java/java8-option...
String result = "UNKNOWN";
T a = X.getA();
if (a != null) {
T b = a.getB();
if (b != null) {
String c = b.getC();
if (c != null) {
result = c;
}
}
}
Wow man Java's still all in with it's terrible threading mechanism all the while C# has had async, await and parallelism for years, Go is well past version 1 to great acclaim, node.js and libuv are taking over, and Akka has gained enormous popularity and yet real coroutines in Java aren't even on the horizon. Even python 3 is getting coroutines.
Low level threads in Java are so goddamned awful and unavoidable that it makes me want to pull my hair out. And I otherwise love Java. Java would be so awesome with real, native coroutines. Like that should be only thing they should be working on right now.
Edit:
I'm guessing the people downvoting this have no idea what I'm talking about and don't know what coroutines are.
You know I'm starting to think there's maybe an entire generation of Java engineers who have no idea how much easier it is to write concurrent code with other tools and Java has ruined them. I love Java, but expand your horizons a bit.
https://en.wikipedia.org/wiki/Coroutine
https://en.wikipedia.org/wiki/Coroutine#Implementations_for_...
Just because there are libraries that provide higher level abstractions around concurrent programming doesn't mean that lower level primitives aren't necessary. In fact, on the JVM due to it's abstraction away from the underlying machine, these sorts of primitives are more needed.
By definition, you aren't doing concurrent programming if you aren't doing shared writes.
I don't understand what you mean but I like Robert Pike's take on concurrency:
"concurrency is the composition of independently executing processes"[1]
As a specific example if you don't want code to block while you're waiting for HTTP requests to finish you're going to be writing concurrent code. I don't understand how that involves "shared writes" but maybe you can explain further? I can write concurrent code that shares no memory, and I can print log statements that show me it's executing concurrently, so I think you may be mistaken.
Yes, and that implies shared writes.
> As a specific example if you don't want code to block while you're waiting for HTTP requests to finish you're going to be writing concurrent code. I don't understand how that involves "shared writes" but maybe you can explain further?
That's not really concurrent code, because there's a clear happens-before relationship between making the request and executing the callback "onComplete", so the original request and the ensuing onComplete continuation is serial.
However, this particular example uses concurrency under the hood to work. You don't know when the request will be ready and you need to execute this onComplete and the next specified onComplete (if multiple callbacks are specified), so under the hood you need a shared atomic reference and synchronization by means of one or multiple CAS instructions, which also imply memory barriers and so on.
Concurrency with "independently executing processes" are not strictly independent. There must be some communication between the processes, otherwise they cannot coordinate to achieve the same task. A typical mechanism is to have a shared queue between the processes as the only point of communication - use of that queue will involve "shared writes".
People build abstractions on top of such mechanisms which hide these shared writes, but they are still there. And that is part of bad_user's point: even though you, yourself, are not actually writing the code for a "shared write", you must call code that eventually performs one.
That is when having good primitives around compare and swap becomes important. Adding these primitives makes implementing those higher level abstractions on the JVM possible for people who are not implementing the JVM, that is as libraries.
Yup, the C# library is called Interlocked and it provides a variety of atomic operations. C# also has ref, which allows variables to be passed by reference, which greatly increases the power of the library.
I find Interlocked absolutely essential for getting maximum performance -- like in implementing lock-free data structures.
There is something like that for Java
http://docs.paralleluniverse.co/quasar/
Android has async-like abstractions in its API. But I like threads better anyway.
> Wow man Java's still all in with it's terrible threading mechanism all the while C# has had async
What is so bad about threads? Why do you need co-routines?
You aren't forced into sharing memory across parts of your program that have no business sharing memory. In Java you can easily end up in a situation where you can look at a program and not know what thread one line is executing in versus another line in the same file. The same instance of a class may be referencing one of its property in one thread or another, and that's sharing memory across threads. A simple if statement may fail you when you share memory across threads because after the if statement checking shared memory evaluates the underlying value for that property it may have changed before the next lines after your if statement executes.
Coroutines as they exist in Go or C# help programmers write code that don't share memory across threads. Akka does this, try it, it's amazing. Java needs native support. Threads are shit. Sharing memory across threads is a nightmare.
Edit: If you're going to downvote, explain why I'm wrong.
I think you are confused (but I didn't downvote you, someone did and then also downvoted my post, was it you ;-) )
Anyway. Unless you use Erlang or fork OS processes you will be sharing memory between your concurrency units.
So learn Erlang it will do you good
http://learnyousomeerlang.com/content
You can build java threads with queues and that works fine. You can shoot yourself in the foot with Go or node.js (probably more so).
[1]: http://blog.paralleluniverse.co/2014/05/01/modern-java/
For Scala developers, there's Scala Async, which is exactly what C# "async" is, implemented as a library: https://github.com/scala/async - plus Scala's `Future[T]` has a better design than C#'s Task.
> node.js and libuv are taking over
And both suck in comparison with Java's NIO and the libraries that have been built on top of it.
> Low level threads in Java are so goddamned awful and unavoidable that it makes me want to pull my hair out
Low level concurrency primitives are necessary for building higher-level abstractions on top. For example I need low level concurrency primitives for implementing a Reactive Extensions (Rx) implementation that does back-pressure: https://github.com/alexandru/monifu/
> the people downvoting this have no idea what I'm talking about and don't know what coroutines are
You're assuming too much.
Why? The one issue I have with Task is the aggravating SynchronizationContext/continueOnCapturedContext. ConfigureAwait(false) really should have been the default. Otherwise, my experience with both seems about equivalent.
Regarding the technical points, having low-level mechanisms for fine-grained synchronization between threads (such as these) does not prevent you from also providing high-level mechanisms for parallelism and concurrency. In fact, it enables others to develop such abstractions in that language in question, without relying on the language itself to provide them.
Aggression that's really the result of thread frustration in Java. I hadn't read about middlebrow dismissal before, I can see how my post came off wrong. I'm glad I posted what I said however, because I the responses taught me new things.
My point was that a programmer shouldn't need to share memory to write concurrent code. That statement is not wrong.
Yes it is. Unless you use Erlang or OS processes you are sharing memory. Or rather you not sharing it any more or less than all the other technologies you listed.
But I agree with your sentiment in general that shared memory and concurrency are not working well. That is why it is worthwhile learning Erlang if you want to be reliable, fault tolerant concurrent systems.
Also nothings stops your from using queues and threads so data is local to each thread and gets copied over the queue.
Actually, even actor-based systems share memory. If two actors A and B send a message to an actor C and expect a response from it, they are sharing memory: what's in C's state. Which can be different depending on whether C received A's message first or not.
Ok in that respect there is just one big pile of shared memory in the whole world, isn't it (maybe except for military air-gaped system). It is the equivalent of saying if A makes an HTTP post to server C the it shares memory. Well ok, I am not sure what you mean by "shared memory", usually it means living in the same heap. So can access it via a pointer or reference.
If we didn't have concurrency primitives, the only high level concurrency idioms we would have would be the ones that made it into the JVM, which is a very slow process.
On the other topic: async/coroutines/friends etc have nothing to do with "adders". The latter are a low-level concurrency primitives that enable fast counters and the like. Feel challenged to write a good a simple counter (like page hit) with co-routines or sync.