Close Encounters of the Java Memory Model Kind | Hacker News Reader

Close Encounters of the Java Memory Model Kind

shipilev.net

109 pointssigna1165 comments

I'm not done reading this yet, but I have a "does this have a name/does this exist"-type question after reading the jcstress example (section 1.1):

It seems like it would be possible to enumerate all interesting orderings of several threads running at once at the threading-engine level, and therefore do this kind of probabilistic test deterministically.

Of course, the number of possible orderings of two n-length programs is very high (2n choose n, since 2n instructions are run in total and you can place the n instructions in one thread first). But the language definition also provides some concept of 'commuting' operations. For example, unrelated writes to two different variables can't have their orderings matter. That should cut down the number of orderings a great deal. And it's possible useful even if it it's only practical for small programs with few total orderings.

Is there a language or framework that does something like this?

I don't know if this answers your question.

I feel like it is about enumerating the possible states your program can be in. You could model it in TLA+, which is about Temporal Logic of Actions; a good fit for modeling the happens-before. There are some nice visualizations associated with a checked model:

https://www3.hhu.de/stups/prob/index.php/State_space_visuali...

Sounds like you're describing Microsoft Research's CHESS (2007), and a host of related projects (Google's ThreadWeaver for Java).

See http://research.microsoft.com/en-us/projects/chess/ and https://code.google.com/archive/p/thread-weaver/wikis/UsersG...

I made blogpost about thread weaver some time ago. Not very useful http://www.mapdb.org/news/thread_weaver/

Wow. That made my head hurt. And, I'm more convinced than ever before that threads remain a TERRIBLE idea. Maybe Java 13 will have something like Erlang tasks with isolated memory and message queues (only) or something...

Somehow I expected more on memory allocation, arrangement or whatnot, rather than concurrency access. Glad I clicked on it for the nightmarish skim, though.

This isn't what a normal programmer using threads is supposed to do. Most people using threads are expected to use locks, concurrent data structures and higher level tools.

This is all about covering behaviour for very low level programs and language implementations which don't use these tools.

The current situation with just few cores is temporary. In future CPU with massive numbers of cores will be a norm and Erlang model will be very natural. Also things like transactional memory should make life easier.

Complicated things are not inherently terrible ideas: your CPU is complex, but you have compilers to abstract you from it. Likewise the ways your multi-core machine shares memory is complicated, and the JMM partly abstracts it, but really you're meant to use much higher level constructs. The JMM matters to people writing lock libraries and compilers, not ordinary users.

It's a common logical fallacy that you can simplify a complex solution by simply redefining the problem it solves as not a problem. The "Erlang model" you discuss is in fact the model the entire computing world used before threads were invented and there were only processes. But threads weren't invented for shits and giggles, they were invented for good reasons!

well said (written)!

I seem to remember first seeing threads used to run Novell and OS/2 on the 80286. (they must of been invented long before that, but that was my initial "OMG!" exposure to them)

But threads aren't just complicated, they are dangerous. E.g. - if somebody forgets and updates something in an instance variable in a Java servlet, rather than local variables, hilarity/calamity ensues.

By the time you are using thread pools, vs "spawn at will", how much better off are you than using a pool of processes safely separated by explicit interprocess communication?

> But threads weren't invented for shits and giggles, they were invented for good reasons!

I remain unconvinced ;-) It always seemed to me, that if threads were such a great idea, operating systems should expose threads, and not processes. In a way NT and Linux took different paths here - on Windows processes are expensive, on Linux threads and processes are pretty much the same in terms of overhead (but threads obviously allow sharing memory, for better or worse).

I've long suspected that if threads really could "be done right", they would've been preferred to processes. But just like people worked really hard to get hardware protected memory to be fast enough to be usable, I think working hard to get processes to be "fast enough" is more viable than "getting threads right". Amusingly, I suppose Erlang is actually an example of how mixing the two ideas can be a great idea: Don't let programmers know their "process" is a scheduled thread, force messaging over shared memory - and reap the benefit of light weight processes and strong(-ish) protection between threads...

my #1 rule of multi-threaded programming: don't do it.

failing that my #2 rule is to make sure you know exactly what you're doing and even then exercise extreme care. #3 is have no mutable shared memory state. #4 is to use queues of immutable messages if needed to communicate between the threads.

And then you call something that, unbeknownst to you, updated some shared state like a singleton.

I like your rule #1.

(But I do Java server programming so I can put the kids through school, so, "Damn the torpedoes! Full speed ahead!")

> And then you call something that, unbeknownst to you, updated some shared state like a singleton.

that would mean one has violated rule 1 and 2. the challenge is that it seems to be rare to find programmers who can nail rule 2 downward. (though they do exist.) thus... rule 1. agreed. :-)

flashbacks to fixing dozens of race conditions and thread leaks back at a major travel ecommerce website...

I've had the same "flashback" for a financial account site I inherited. Offshore work done in a hurry...

In fairness, though, it's hard to constantly be on your guard NOT to EVER update something that might be visible to another concurrent request. It's interesting that the MS web app model for .NET was to make a new instance of the service code for a page for each request, to at least avoid this problem the moment you "walked in the front door".

TL;DR:

    shared.setState( "Oops!" );

i barely know some java, and i have heard of concurrency and locks, and probably understand the idea of thrrads. yet, i found this to be a thoroughly enjoyable and mysterious read!

Aleksey Shipilev is a great developer and writer. His articles are must read!

I recently talked with a Microsoft engineer and his observation was that a lot of MS customers have troubles running their C# code on ARM. As with Java C# does not guarantee much beyond stating that word-sized writes are atomic allowing a lot of reordering. However, Intel CPU are nice to developers allowing a lot of buggy code to work, but ARM is not forgiving resulting in a really hard to track bugs.

I wonder whether Java will get displaced by Swift on the server at some point. Anyone want to place bets?

No garbage collection, so no GC performance hiccups. No explicit threading for easier concurrency.[1] Optionals for protection from null pointer exceptions. These are 3 of the biggest problems in server-side Java programming, I'd say. That's before we even get to the elegance of the syntax.

[1] https://developer.ibm.com/swift/2016/02/22/talking-about-swi...

> No garbage collection, so no GC performance hiccups.

If your models are naturally highly cyclic, dropping a GC can vary from painful to near impossible. I think a better solution for most server-side code (from both an ease of use and performance standpoint) is to use a hybrid GC/arena approach, centered around tasks.

If most of your work and memory churn centers around discrete and relatively short-lived requests, you can allocate all objects whose lifetimes are bounded by the duration of the transaction in a thread-local arena. You can't beat unsynchronized bump-pointer allocation, and you can simply throw the memory away at the end.

I've been really disappointed with the lack of innovation in the memory management space. Rust is the only recent language I've seen that did something more significant than choose between GC and reference-counting.

Painful in terms of what?

Jumping through hoops and hoping you put enough weak pointers. Such structures, if they are getting mutated, also tend to take significant performance hits even if you get it right, because of all the synchronized increment/decrementing.

Another place where this can really hurt is "reactive" approaches, where cycles are very hard to avoid. Weak delegates (like what WinRT uses for events) can help here, but they raise a number of complications. The biggest is that they compose poorly: if you have an intermediary in your event chain, you need to make sure the subscription is the only place where a reference is being held.

FWIW I will try to summarize the Swift architect's comments on this:[1]

a) Declaring weak pointers does take some more reasoning about your object graph. A silver lining is it makes the intention of your code more explicit which helps with maintenance.

b) Swift emphasizes value types which reduces the need for pointers vis-a-vis Java.

c) Most garbage collectors also add overhead when updating object references.

d) Swift is massively more memory efficient. Fast GC requires 3-4X the memory.

[1] https://www.quora.com/Why-doesnt-Apple-Swift-adopt-the-memor...

a. This one smacks of sour grapes. It doesn't help with maintenance when you're dealing with a dense graph, and every time you add a relation you have to check if you need to make something weak. Reasoning about reference graphs is often not intuitive, due to it being hard to see indirect references. Also note that some cyclical structures cannot be broken by statically using weak references, and require special action like specialized manual reference counting (I recently had to implement such a system ).

b. Very true, but these are the exact opposite of the cases that I'm talking about.

c. Two things:

First, memory has gotten incredibly cheap and dense for servers. Server GCs are generally more cache friendly than reference counting (due to both allocation strategies/compaction and the lack of need to touch reference counts), which is important because CPU performance gains are slowing down.

Second, the graph he shows (I've seen it before) is looking at older GCs, and more importantly GCs where memory usage outside of the working set was not optimized for. In a paged system, it is of little benefit to ensure memory outside your working set is released.

Obviously on mobile the situation is different, but there are also GCs that have far lower memory usage because they have been optimized for constrained scenarios. Android's new ART collector is sort of an example of this, but their efforts were massively undermined by the fact that they had to adapt to all of the existing Android library/app code which uses tons of object pooling. Manual object pooling with GCed objects creates all kinds of problems for GCs, particularly generational ones.

I agree with the decision to go with ARC for Swift from a lot of perspectives: Swift's target market is not usually dealing with complex interconnected models, using a GC would make ObjC interop difficult, etc. But it is far from obvious based on this information that it is beneficial to have a language stick with only a GC or only reference counting.

Without knowing specifics it seems like you're focusing on complicated "dense graph" scenarios. Perhaps Swift is a bit harder for those.[1] However your initial claim was that dropping GC was bad for "most server-side code" (emphasis mine). Does most server-side code involve dense graphs?

As a point of logic, optimizing to make 95% of cases a lot less "painful" would be worth a tradeoff of a bit more pain in the already difficult 5%. Thus if Swift fits that description it stands to reason that it could displace server-side Java.

[1] (Although one could argue those are exactly the scenarios where reasoning more carefully about your object graph is helpful.)

I focused on that somewhat in this thread because that was what you initially asked about, but note that only my first response to Swift's creator's justifications even mentions that. I do think servers moving from GC to RC would not be very advantageous, but not for that reason (or at least, like you said that reason only applies to a minority).

The biggest reason is that while some server side applications are very latency sensitive, many (most?) are not. For most websites, significant (i.e. collections of older generations) GC pauses are going to be relatively infrequent and having a few requests slow down just a bit for less than a second is not a huge problem. In these cases (which if they aren't the majority, they're still close) throughput is more important that latency, which is where a GC shines. And ease of programming/maintenance is likely to be more important than both.

As far as "helping you reason about your object graph", I don't see the argument about how forcing you to RC helps you do that at all. All of the extra reasoning is centered around something that would be otherwise irrelevant. Even in cases where you do something that is not amenable to GC collection (e.g. events/observers that keep objects alive which you didn't intend to), it seems of little benefit to have to constantly think about all of your other references at the same time.

> GC pauses are going to be relatively infrequent and having a few requests slow down just a bit for less than a second is not a huge problem

I respectfully and strongly disagree. Outlier performance issues are a huge problem when scaling businesses.

Additionally "ease of programming/maintenance" is exactly an area where, overall, Swift may have some major advantages over Java. Without even further debating the cost-vs-benefit of declaring dense graph relationships, I believe you've conceded that this a small part of the overall picture.

> I respectfully and strongly disagree. Outlier performance issues are a huge problem when scaling businesses.

For server-side GCs, a major collection is well within the bounds of incidental network latency, and happens far less frequently. Outlier performance is a huge problem, but from what I can tell outlier performance specifically due to GC pauses is an issue in far fewer cases (one that comes to mind is servers for certain financial services).

> Additionally "ease of programming/maintenance" is exactly an area where, overall, Swift may have some major advantages over Java.

That's true, but these advantages are uncorrelated to reference counting and they diminish if you choose to compare with a language that has aged better (C# comes to mind). If you're really concerned about high-performance, low-latency server programming and you are willing to switch to a less common language you'll be much better served by something like Erlang, which is specifically designed for that.

Erlang's memory model is fairly similar to what I have described here, but still runs into some issues that I feel could be solved by giving more control over where exactly things are allocated. It also resists any kind of mutable state; I agree that mutable state can complicate concurrency heavily, but having an option to use it sparingly would help (process dictionaries are too limited).

My point is basically that just like GCs aren't the solution to every problem, ARC isn't either. I think a language that allows for both will beat both Java and Swift handily when it comes to making both easy things easy and hard things possible.

Working with cycles under reference counting is not impossible. The key is good memory reporting tools. Yes, sometimes even with tools identifying a memory leak through cycle with reference counting can be painful. However, the fix itself is typically very local and easy to test. Compare that with a C#/Go/Java application after it turned out that GC-induced latency is unacceptable. The fix could be a massive rewrite that makes the resulting code maintenance nightmare. And if in the case of Java there are at least some knobs to tune the GC that could be enough to fix the problem, with Go or C# the code rewrite is the only solution.

As for local object pools, they are nice as long as their size is smaller than CPU cache. Without the control on the pull size it is very to easy to accumulate a lot of local garbage that it starts to hurt the performance.

The problem with both approaches is how binary they are. GC latency is an issue if the active part of the heap gets too big and/or any strategies you have for dealing with short lived garbage (e.g. generations) get overwhelmed. The simple solution is to give flexibility in how allocations are done. C# does this a bit but makes you pick per type, instead of per instance. Go goes a bit further, but it still only leaves you with the options of stack allocated/containment and GC. What would be ideal is having stack allocated, ARC, and GC all be options in the same program.

Another advantage of the task-focused approach is that you can optionally collect the arena if it has gotten too big without halting the whole program (just the running task). In this case, you may have latency on one request but it need not effect latency on other requests.

> As for local object pools, they are nice as long as their size is smaller than CPU cache. Without the control on the pull size it is very to easy to accumulate a lot of local garbage that it starts to hurt the performance.

How is this different from the ARC case? If your working set fits in the cache, great, if it doesn't, then you'll have problems. If your allocation/release patterns are such that the allocator keeps handing you the same exact pieces of memory, then you're basically doing stack allocation. Not to mention that RC doesn't help the cache-unfriendliness that comes from heap fragmentation.

I really wish arenas with optional GC would be more supported in main stream languages. Erlang got it so right. That and using reference counting for global structures where the type system or API guarantee absence of cycles should allow for responsive and maintainable applications without the nightmare of manual memory management.

With ARC you can still have hiccups resulting from cascading deallocations.

(Nit: ARC is a form of GC)

That seems like a much smaller scale problem than global GC for typical request-based server apps.

(And yes you're right, by "GC" I mean mark-and-sweep GC.)

> I wonder whether Java will get displaced by Swift on the server at some point. Anyone want to place bets?

Last time I checked developing with swift requires xcode, which requires osx, which requires spending excess $1k on what should be commodity x86 hardware, to have it labeled "a Mac".

So a big fat no chance there.

Swift runs on Linux, and Swift's vim/emacs/atom packages work just fine.

Not that I think any enterprise that has tons of Java code will switch to anything ever, much less Swift. But "needs a Mac" is not the reason.

In that case, O'Reilly better get their act together and stop spreading disinformation in their Swift books[1].

[1] http://www.oreilly.com/programming/free/swift-pocket-referen...

Where does it say it only runs on iOS and OS X?

Page 5: getting started with swift 2

It says xcode 7 or later is required. How am I going to run that on Linux? How am I going to CI that on Linux?

It doesn't say it's required. Relax, Swift was just open sourced in December so some of the books might still focus on Macs.

Watch someone code and run Swift on Linux: https://developer.apple.com/videos/play/wwdc2016/415/

Are you trolling me? Playing that video requires fucking Safari or a Apple WWDC app...

http://imgur.com/BUrkMVc

This is Apple and regard for other platforms in a nutshell. And it sorts of kills any interest I may have had in Swift as well.

Relax, click on Resources tab under the video, there are links to HD and SD versions you can download. PDF slides are there too.

Nope. No links to video on non-Mac systems. PDFs only.

Do I need to pay the apple-tax just to use their web-pages? That's basically a huge middle-finger to web standards. Absolutely useless crap, borderlining offensive.

Apple is doing some pro-grade trolling here. Not even Microsoft is doing stuff like this anymore.

I just downloaded the video on Windows 10, in Edge no less. Click on the resources tab like he said.

Not sure who's trolling here...

edit: Oh wow, I didn't even notice but the video can stream in Edge! Doesn't do it in Chrome on OS X though XD

Firefox running on Fedora: http://imgur.com/akxUq2Y

I tried that page from Firefox on android. Wth would they remove the downloads?

Anyway.. their web pages and developer resources are clearly disfunctional garbage, and their cross platform commitment almost non-existent.

I think it's still fair to say swift won't displace java anytime soon.

They probably think that you are not going to download 1GB+ video on mobile. I don't know, you must ask them.

Anyway, there is no conspiracy to deny the WWDC videos to non-Safari users or to piss them off. It's just how the Apple is, they have protocol they were pushing for a long time (HLS), they are certainly not going to use competitor (MMS or DASH).

In Firefox on Android, check Request Desktop Site and you will get the download links. Or use Chrome for Android, it supports HLS streaming. Meanwhile, you can file bug to Firefox bugzilla and ask for HLS.

I don't think Swift would replace Java anytime soon either. From JVM languages Kotlin has better chances, from non-JVM languages probably Go. (Sorry Rust fans, a typical enterprise programmer is not going to learn how to appease the borrow checker).

I have links to download a >1GB video file on Debian/Chrome. That feels like a pretty big middle finger to web standards as far as I'm concerned. No thanks.

Apple uses HLS, because that's their protocol, competitor to DASH (strictly speaking, neither one is a web standard, though DASH is an ISO standard). It does not work for Firefox or Chrome for OSX users either.

As others already said, Swift is opensource and runs on Linux. With Swift 3, also the standard library (Foundation) should be available on Linux too.

However, Swift for Linux is not yet "productized" in the same way, as Swift for Darwin is. You can download a tarball for Ubuntu only, unpack it somewhere and call it a day. It will take a while before you can use 'yum install swift' from EPEL on your shiny Centos/RHEL workstation.

How would you write the code (and JCStressTest tool) corresponding to the article in Swift?

To answer my own question: You cannot.

To answer your question: Very unlikely. Swift has a lot of irrelevant complexity from its Apple/Objective-C interop, poor reflection support, poor IDEs (Xcode) compared to established platforms such as Java 8 or alternatives such as Kotlin, Scala and so.

I don't know the exact answer, but Swift takes a higher-level task/queue approach to concurrency, letting the OS handle more of the underlying thread management. So if you wanted to write a concurrency stress test you'd probably write it on top of those constructs.[1]

I'm not sure what you mean by "irrelevant complexity". Maybe you could cite some examples. Limited reflection/dynamism was an intentional design choice for greater safety, and there are complaints, but it's not without benefit. And it's a bit hasty to criticize lack of IDE support when it was just open sourced 7 months ago. That's like criticizing Scala for lack of IDE support.. in 2005.

[1] see https://developer.ibm.com/swift/2016/02/22/talking-about-swi...

While the constructs are higher level, the compiler still needs to deal with all of this complexity. Normal developers can ignore this. But there will be a memory model and a modern language should specify this. If swift does not have an equivalent idea of what its memory model should be, the ARM/X86 splits will lead to interesting bugs.

Ha. Maybe if we are talking about geological timescales. Nobody is going to rewrite all those trillions of lines of Java code out there in the wild any time soon. Also with all the tooling built around the JVM, a non-JVM replacement is probably a non-starter.

Java is the new COBOL - and it'll be lurking in the guts of big enterprise systems for decades.

If your time frame is less than 50 years I'll take that bet.

But your comment leads me to believe you haven't used Java in some time.

The GC has 100s of man years worth of research and optimization built in to it, Java provides a lot of higher level concurrency like executor services, fork/join, promises/futures, and with Quasar it has Go like goroutines called fibers. Optional is built in to the language as well (and to be honest find them sort of ho-hum, certainly a bizarre reason to stop using a language.)

Speaking of Go... would Swift really have a chance against Go on server?

It's early days yet, but one advantage of having the same language on the client and the server is that you get a lot of benefit in code and team sharing.

I think it's either 5-10 years or never. I am a huge Java user and fan btw, but I am looking to the next 20 years here.

Despite much optimization, GC still struggles with memory efficiency and predictable performance in a diversity of scenarios. You have to tune it.

I guess you could simplify my comment to two advantages: No GC, and elegantly baking constructs that make programming safer into the language as opposed to things you can opt-in to. While it may be possible to get these things with Java, if it's a lot simpler to get consistent performance, efficient memory use, protection from null pointers, and safe concurrency.. then there may be an opportunity for Swift.

> No garbage collection, so no GC performance hiccups.

Swift has ARC(https://en.wikipedia.org/wiki/Automatic_Reference_Counting) which is a GC strategy. And I don't think it can compete with the performance of the JVM's G1 (currently).

> I wonder whether Java will get displaced by Swift on the server at some point. Anyone want to place bets?

I'd bet on Scala(http://scala-lang.org/) which has far more benefits compared to Swift - OOP+FP, higher kinds, typeclasses, implicits, (hygienic) macros etc. dotty(https://github.com/lampepfl/dotty) will have union- and intersection types with other useful stuff. There is a JS backend(https://www.scala-js.org/) and there'll be an LLVM(https://github.com/scala-native/scala-native) backend too.

Just to be clear: reference counting isn't a garbage collection strategy. It is, however, an automatic memory management strategy.

Reference counting is a form of garbage collection.

[1] https://news.ycombinator.com/item?id=11659863#up_11662144

[2] https://en.wikipedia.org/wiki/Reference_counting#Use_in_garb...

[3] https://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf

I have a task coming up about reducing the memory of some WebLogic Java applications, I'm not very familiar with Java.

Does anyone suggest a book / site / tool I can use?

Currently they have Min/Max/ 4 GB and Object 2 GB for the Java args and I want to delve into it to see more about actual memory use etc.

VisualVM can do that. It helps to have some background on the different spaces and JVM options. Ken Sipes' "Tuning the JVM" and the book he recommends are good starting material. It is a highly nonintuitive skill.

thank you very much!

That zalgo toward the end made me smile.

Reply on news.ycombinator.com