Java Optimized Processor (2005) | Hacker News Reader

Java Optimized Processor (2005)

jopdesign.com

69 pointsKoshkin60 comments

Azul Systems (who today are mainly a JVM vendor) started by building large multicore boxes (Vega) to do Java workloads - there is an interesting tech talk about it:

https://www.youtube.com/watch?v=5uljtqyBLxI (2009)

They built a "really good JIT target" rather than direct bytecode for reasons explained in the video. Curious to hear from anyone who ever used one...

Yes, was part of Team which used Azul Zing and had very minimal issues.The Primary use-case was to minimize the application downtime during Stop-the-World GC and Zing is almost pause-free.

I wonder how it compares to Shenandoah 2.0, which has been spectacular for me.

https://youtu.be/3l2oi-X8P38?t=129

I.... I think there's been a mistake.

Not quite the same, but this brings back memories of both Transmeta and PicoJava:

https://en.wikipedia.org/wiki/Transmeta https://en.wikipedia.org/wiki/PicoJava

Transmeta's processor was able to run Java bytecode through built in emulation layer.

The JOP served me well for years of research work, but only little commercial success. This project is kind of stalled. JOP’s strength was worst-case execution time, not average case speed. No Java processor beats JIT on a modern Intel processor.

I remember following and very much enjoying the development of your project years ago. I admire your project for its daringness to cross so many boundaries.

Are you still working with FPGAs? Have you played with the Open Source tooling symbiflow? If you re-designed JOP what would you change? If you could change the FPGA fabric to better support a processor like JOP, what would it be?

https://hackaday.com/tag/symbiflow/

edit, I see have continued publishing, https://www.jopdesign.com/publications.html and I think many of my questions are answered there. Specifically, https://www.jopdesign.com/doc/icap-ctrl.pdf

So many questions ;-)

Yes, I have still quite some fun with FPGAs. I am interested into open source tooling, e.g., using Chisel and want to look into yosys.

Redesign of JOP: 1. switch from VHDL to Chisel, 2. change the cache to be able to read one bytecode per clock cycle, 3. maybe some bytecode fusion (executing more than one per clock cycle), 4. time-predictable branch prediction.

Mmh, looks like there is some open work ;-)

JOP is pretty neat and a good reminder that FPGAs enables us to make unconventional machines. Another impressive one is J16 which is an amazingly small FORTH machine. I'm surprised that I haven't found a Lisp machine, but at least there's Reduceron for lazy function languages (complete with hardware GC).

There have been a few Lisp processors made with FPGAs:

http://www.aviduratas.de/lisp/lispmfpga/

https://opencores.org/projects/igor also https://www.flickr.com/photos/kaitorge/sets/7215760944571932...

https://www.mail-archive.com/picolisp@software-lab.de/msg048... also https://www.mail-archive.com/picolisp@software-lab.de/msg048...

> Another impressive one is J16 which is an amazingly small FORTH machine

Did you mean the J1?

https://www.excamera.com/sphinx/fpga-j1.html

yes, thanks

Do you think it would be possible to have hardware acceleration for JIT compilation that could beat a pure software JIT (ignoring special purpose domains like real-time)? If so, how?

I don’t see any reason why you couldn’t code a JIT to run on an FPGA, though it might not be faster than 3ghz of the excellent branch prediction/Instruction level parallelism in a modern x86 processor.

Faster has two important dimensions in JIT compilers: faster compilation and (more important): faster resulting executables. Both are related, in that the quality of optimisation is traded-off against how long the optimiser can run.

https://riscv.org/wp-content/uploads/2017/05/Wed1545-HybridD...

This presentation uses a NIOS II processor to translate RISC-V instructions to a VLIW. Then it replaces the three most time critical parts of the software with hardware blocks.

Years ago there was the Jazelle project on ARM processors that tried to run Java on hardware:

https://en.wikipedia.org/wiki/Jazelle

I wonder what were the pros and cons of this, and why it didn't end up being integrated in later ARM CPUs.

ARM requires precise exceptions, which meant translation of bytecode was on instruction by instruction basis in the fetch/decode stage ... which is not good for performance. Moreover, Jazelle could only do simple bytecode in hardware, the complex instructions were done in software, which lead to massive exception-emulate-return overhead. Also, Jazelle was Java only, so no support for other byte codes, which limited Jazelle's potential. Within a few years, everybody had a 'null implementation'

Jazelle was intended for mobile phones to support Java apps on the phone. I had a phone with Jazelle and it run Java quite fast those days. But since Apple killed Java on the phone with the iPhone with native apps from the App Store and no Java support, this was the end for Jazelle.

Because like all the other language attempts, there are hardly performance improvement vs a general purpose CPU and a more flexible runtime written in software.

Jazelle was for ARM, mostly to make J2EE faster. These days, it's more practical to compile all the Java to ARM when an app is installed or in the background when the phone is charging. For server-side applications, you'll either use Graal, or just live with slow-ish startup times and let the JIT warm up.

Since around 2000 there has been support for AOT in Java commercial JDKs, or JIT caches across execution runs.

So living with slow-ish startup times and let the JIT warm up was mostly a matter for those not wanting to shell out money for them.

Actually, most likely Graal Community edition and its integration in OpenJDK, alongside AppCDS (from J/Rockit), while IBM also brought their AOT/JIT tooling into OpenJ9 that made Excelsior JET decide it wasn't worth any longer.

So besides the BEA, Oracle, IBM commercial compilers now turned into free beer, PTC, Aicas are the surviving ones, but they target embedded scenarios.

And then there is Android Java with JIT/AOT as well.

This reminds me of Sun MAJC [1], which was another processor optimized for running Java.

[1] https://en.wikipedia.org/wiki/MAJC

And Jazelle

https://en.wikipedia.org/wiki/Jazelle

Which ARM technically still supports.

Only in the sense that the instruction to enter Jazelle mode exists, and will always return failure.

Are there any real implementations of Jazelle in 2020 that are in active use in a modern processor supporting the Arm ISA?

And then we can have Clojure on bare metal, and we're almost back to LISP machines!

This doesn't directly execute JVM bytecode in hardware.

It provides a stack machine architecture, and translation from JVM bytecode to its custom stack machine instruction set. I think the theory is that translating from one stack machine to another is a better way than existing JVM implementations which translate the JVM stack machine to register machine instructions (x86/ARM/etc). It may be more elegant in theory, but I doubt the benefits in practice are enough to justify the switching costs.

The biggest problem with any Java-in-hardware design, is the JVM is a moving target (and it is moving faster now than it used too), and without a sustained engineering investment you soon get left behind. Plus, with custom silicon (ASIC), new features will require new hardware. At least this is an FPGA design, so you can field-upgrade the FPGA – but the price-performance of an FPGA is poor, especially as this is a platform for general-purpose computation rather than building some kind of specialised computational accelerator. Although, since it is not directly executing JVM bytecode in hardware, it may be possible to support some newer JVM features just by updating the translation software.

But a cool research project nonetheless.

Some of the bytecodes are executed directly, others are translated to microcode sequences, others are implemented in software. From the linked paper:

"... direct implementation of all bytecodes in hardware is not a useful approach ... Microcode is the native instruction set for JOP. Bytecodes are translated, during their execution, into JOP microcode. This translation merely adds one pipeline stage to the core processor and results in no execution overheads ... 43 of the 201 different bytecodes are implemented by a single microcode instruction, 93 by a microcode sequence, and 40 bytecodes are implemented in Java."

Note that its "custom stack machine instruction set" is an interesting subset of Java bytecodes. The most common bytecodes are interpreted by a single narrow microcode instruction while the more complex ones are executed by little programs. A wider and more parallel microcode would make the complex instructions faster but would make the hardware larger without helping the simple bytecodes very much.

The focus of this project is real time instead of absolute performance, so things like caches are added in only limited ways since they get in the way of predictable execution times.

> This doesn't directly execute JVM bytecode in hardware.

Lisp Machines didn't directly execute their documented instruction set.

JOP works in exactly the same way as commercial Lisp Machines.

Is the JVM spec itself really a moving target ?

I am working on a heavily customized java7/jvm7 internals engine. So I am years behind.

But arent most of the changes in the language / internal implementation ?

I agree however that trying to chase commodity processors is a fools errand.

> Is the JVM spec itself really a moving target ?

Yes, it is. Every new Java release brings a new JVM spec and a new version of the classfile format [1]. How much changes varies from release to release – in some Java releases, little changes other than the classfile version number; in other Java releases, there are significant new features – new opcodes, new class file attributes, new constant pool entry types, etc.

> I am working on a heavily customized java7/jvm7 internals engine. So I am years behind.

The most obvious thing you'd be missing would be support for the CONSTANT_Dynamic_info, CONSTANT_Module_info, and CONSTANT_Package_info constant pool entries.

There are a bunch of new classfile attributes, but you may be able to get away with just ignoring the newer ones, at the cost that certain newer features won't work correctly (such as modules introduced in Java 9).

I don't think they've added any new opcodes since the introduction of invokedynamic in Java 7. (But, there is always the chance that some future Java release will.)

[1] https://docs.oracle.com/javase/specs/index.html

The lines are somewhat blurred even in historical LISP machines. Microcode is hardware.

Yeah, as long as you can “polyfill” or “monkeypatch” in the new JVM instructions, it’s not a huge problem: ideally you take a page from IBM’s TIMI, and you treat compilation as a form of caching: you store the JVM code on permanent storage, and compile it to the underlying hardware on first execution, saving the result until the CPU is upgraded.

Microcode obviously isn't hardware since it is program we can replace. It is firmware.

Why do we optimize our CPUs to fit our programming languages instead of desigining our languages around our CPUs?

The reason we have programming languages at all is to make the process of using the CPU easier. We don't code in assembly because it's easier to do things with higher level languages. If you can make a language both designed to run around CPU hardware well and also be easy for a human to use, then great. If not, then it's trade-offs. If I can make a higher level language like Java run better on hardware, then isn't that great? If it makes the CPU slower because the Javaness of it, then I guess that sucks, but it isn't necessarily the end goal.

Because languages are used for optimizing programmer output (these days) while CPUs are designed for general use compute.

I think it would be a regression in technology to start having Intel Web-Compute Processors and Intel Gaming-Compute Processors but you know... that's just crazy me.

IBM has hardware that accelerates XML processing: https://en.wikipedia.org/wiki/IBM_WebSphere_DataPower_SOA_Ap...

Wow, if that's not proof that xml is a terrible format, I don't know what is.

>Wow, if that's not proof that xml is a terrible format,

A bad format for what? Do you mean that you assume that HTML,markdown would look better in something like JSON? I want to see a "proof" for documents you write in JSON by hand.

It might not be optimal, but it certainly is better than the alternatives regarding machine processing, data validation, graphical tooling and comments.

For decades the CPUs have been optimized to run faster existing applications - regardless in which programming languages the apps are written. On the other hand you can find languages designed around our CPUs (like Rust with its "zero-cost abstractions").

IMO the last language designed around CPU was C designed around CPUs of that time.

Modern CPU have registers, caches, SIMD instructions which are tremendously important for performance. Yet I don't know no language which would explicitly provide any control over those things. You can only hope that your loop will be understood by compiler and optimized to proper SIMD instructions. And any change might break that optimization (including changes in compiler).

> Yet I don't know no language which would explicitly provide any control over those things.

CUDA provides your explicit cache control through "__shared__" memory.

C commonly has low-level attributes. In particular, you have the "register" keyword in C if you really want to go that route, but compilers have a nice greedy algorithm that is damn near optimal, so its best to let the compiler perform its own analysis for register allocation.

SIMD instructions are easily provided through intrinsics: https://software.intel.com/sites/landingpage/IntrinsicsGuide...

-------

There's plenty of low level languages that provide the control you desire. The main reason for __shared__ memory is that compilers can't optimize memory across threads yet, so we still require the programmer to tile memory manually and think about the optimizations across the many, many threads a CUDA multiprocessor supports.

If you want to explicitly control the cache on x86 systems, use the _mm_prefetch() intrinsic. In practice though, it is pretty rare for prefetching to be useful on modern systems. (Auto-prefetchers will optimize most sequential traversals, and those that fail prefetching can still be out-of-order executed at decent speeds)

Serious question: Why not both?

OpenCL / CUDA are good examples of languages designed around GPUs. C itself was based off of the DEC-PDP11.

Optimizing the other way: Many modern processors are based on executing C faster and faster... but other languages have begun to be targeted by CPUs. Not only this JOP, but ARM famously has the "convert to javascript floating point" instruction, to optimize our phone's handling of Javascript and HTML pages.

There's lots of engineers. Why not have all the engineers try to make everything faster? What benefit is there of doing things only one at a time as you propose?

Because we have been doing it for decades, mainframes have always done like that, Xerox PARC workstations used microcoded CPUs, since UNIX and C got widespread, CPU vendors optimize for C including memory tagging as hardware mitigation, and NVidia nowadays designs their cards for C++ and Tensorflow workloads.

We have been doing both for a long time.

C was designed around PDP-11.

And now we design processors to run C fast.

Though RISCs are considered language agnostic, they tend to be optimized for C. One side effect of that is the elimination of rotation instructions which C doesn't have (though GCC can generate them as a special case).

RISCs also don't help with lexically scoped variables which languages like Pascal have but C doesn't.

A processor that was specifically designed for C was called CRISP (and was briefly sold as the AT&T Hobbit):

https://en.wikipedia.org/wiki/AT%26T_Hobbit

I'd much prefer we design processors around languages, because from what I understand, x86 isn't really optimized for anything.

x86 itself has a lot of glut. Most x86 processors don't directly implement x86 in hardware but execute instructions in microcode, much like JVM and Java bytecode.

Ever since the advent of Java 8, developers who had previously moved on to newer generation of programming languages are now slowly piquing their interest in Java again which is equally exciting and reassuring to see. Java has consistently ranked at the top in the TIOBE index. https://www.tiobe.com/tiobe-index/

Not as strange an idea as you would think. In fact the ARM instruction set has a instruction for "Floating-point Javascript Convert to Signed fixed-point, rounding toward Zero" FJCVTZS.[0] Considering the ubiquity of JS I expect more "JS/Wasm instructions" to be added in the future. (A Wasm co-processor appearing in 5 years is a bet I'm willing to make.)

[0] https://stackoverflow.com/a/50966903

ARM has a whole instruction set extension for JVM code (Jazelle).

Azul Systems once had an immensely powerful [at the time] Java Hardware processor that could execute Java operations and lock intrinsics at many times the speed of available x86 hardware.

Might want to put a [2005] in the title, when the thesis was published.

Reply on news.ycombinator.com