SD4J – Stable Diffusion pipeline in Java using ONNX Runtime | Hacker News Reader

SD4J – Stable Diffusion pipeline in Java using ONNX Runtime

github.com

77 pointssimonpure45 comments

ONNX Runtime needed a good demo application in Java and this was fun to do (I maintain the Java API for ONNX Runtime and wrote this SD implementation). I've added SDv2 and SDXL support (and the turbo variants thereof) after the initial release, and I'll upgrade it to the latest ONNX Runtime when that comes out to get FP16 support among other things.

Did you write this code? Any good tutorials for ONNX you recommend?

I did. It depends what you want, for an overview of how ONNX Runtime works then Microsoft have a bunch of things on https://onnxruntime.ai, but the Java content is a bit lacking on there as I've not had time to write much. Eventually I'll probably write something similar to the C# SD tutorial they have on there but for the Java API.

For writing ONNX models from Java we added an ONNX export system to Tribuo in 2022 which can be used by anything on the JVM to export ONNX models in an easier way than writing a protobuf directly. Tribuo doesn't have full coverage of the ONNX spec, but we're happy to accept PRs to expand it, otherwise it'll fill out as we need it.

I have been very impressed at how performant the runtime's cpu inference is. It beat out hand written avx intrinsics by almost an order of magnitude. I had to go find a machine with no discrete gpu to convince myself it wasn't using one despite it saying it wasn't.

If you liked this you might like SDXL Turbo in the browser, along with some custom pixel art LoRAs: https://gfodor.github.io/image-gen/index.html

Old enough to remember the days when Java reached near AI-level hype. Most of the advanced placement and undergraduate CS courses in the U.S. were taught in Java. There was an annual conference in San Fransisco called JavaOne/Oracle OpenWorld that rivaled the size of even NeurIPS, drawing tens of thousands of attendees each year. According to some statistics, there were 10 million Java developers and 3 billion devices running Java worldwide. I wonder what happened? Why does HN dislike Java so much?

edit: Someone should really take the time to highlight all the innovative machine learning research happening in Java. The original implementation of t-SNE [1] was written in Java. Most NLP researchers have heard of CoreNLP [2], also Java. One of the earliest ML libraries, Weka [3], was written in Java and is still actively developed at the University of Waikato. Noteworthy research on ML4Code specifically targets the Java language [4]. There's a bunch of published sketching algorithms for Java (e.g., DataSketches [5], t-digests, ddsketch et al.), featured in an invited talk [6] to NeurIPS this year. Just to name a few off the top of my head.

[1]: https://github.com/lejon/T-SNE-Java

[2]: https://github.com/stanfordnlp/CoreNLP

[3]: https://www.cs.waikato.ac.nz/ml/weka/

[4]: https://openreview.net/pdf?id=bUDmRzeh3PT

[5]: https://github.com/apache/datasketches-java

[6]: https://nips.cc/virtual/2023/events/Invited%20Talk

HN doesn't really dislike it, looking at the past threads. There are usually a few people quickly jumping to comment negatively, but in most cases they do not have enough experience with it or have not touched it since Java 8 or earlier version. Also, it is not a small thing to learn and it seems to be unpopular among the bootcamps - I have seen a lot of Python course ads, but hardly a single one on Java.

There was a _long_ hiatus in Java progress between 1.5 (release in September 2004) and 1.8 (released in March 2014). Before Java 8 the language was, basically, the previous generation's golang. in that timeframe there was no Kotlin, Scala had little toolchain support and some sharp edges. So many smart people left the JVM altogether.

Do you mean between Java SE 6 in Dec. 2006 and Java SE 7 in 2011?

I remember being taught using 6 in high school around 2009.

Java 5 and Java 8 were both very big - generics in 5, lambdas in 8. 6 and 7 were iterative in comparison.

There were very many important changes in the meantime over that timeframe, but generics and lambdas fundamentally changed how you use the language - Java 4 is not the same language as 5, same between 7 and 8. This is not the case for the 6 and 7 releases.

I think one of the important changes was the change of ownership from Sun to Oracle. Sun was loved, Oracle is not, but I don't believe that it affects the sentiment here.

I don't know that it's hated so much as it's a dominant language that has lost hype (because of the transition to Oracle?).

Therefore its popularity stands in the way of buzzier languages. Gotta point out Java's warts to shoehorn in discussion of your pet language.

As somebody who has been professionally developing in Java for 24 years I have spent a lot of time trying to understand continued success of Java.

Personally, I think Java is very poor language. I wish I could program in something like Common Lisp or even Clojure. And yet every single corporate Clojure project I have seen was a total disaster.

I think what happens is for new developers, Java does not come as just language. It comes with a huge ecosystem including books, Stack Exchange/Overflow posts, frameworks, certifications, etc. All of this gives developers a basic template for how an application should look like.

And, paradoxically, all of this is aided by Java which is a poor, repetitive language.

When I open a new Java project, I can easily identify all of the parts. Enough of the structure is a simple copy and paste of another project, that whatever is different must still follow certain rules.

Project that do not follow this blueprint tend to be flops and huge maintenance problem.

So why is that? I think most developer underestimate that structuring an application is a skill in itself. When given a preexisting structure to fill in with functionality, they are following good, established structural patterns that are instantly recognizable and understandable by the community of Java developers. This makes it easy for new people to not get completely lost in even a large application.

When the same Java developers decide to make something in Clojure, they are now responsible for structuring their application without any clear guidelines or established patterns on how to do it. And for some reason they tend to not understand is that their existing experience has not taught them what to do in this situation.

Again, That just makes it sound like a good language that isn't to your taste but is so aggressively adequate that its hard to justify your preferences, no?

What's your opinion on Kotlin?

Kotlin, in my opinion, is a language thought up by bored Java developers to spice their lives a bit.

All Kotlin code I have seen in the past (and I saw a lot of it) is line by line transposable to Java. It is Java written in Kotlin by Java developers. If there are Kotlin features that really make it unique -- I have never seen any of the put to good use other than a tiny bit of syntactic sugar.

I definitely have seen no productivity or quality of results increase of any kind.

You can fuck up an application in any language. And you can write clean code in any language except the most hostile. No programming language has ability to make the programmer write clean code. I can write nice object oriented code in ANSI C. It is all in our heads.

The single biggest problem with Kotlin is that, to be able to program Kotlin, you need to also know Java.

So now you need a candidate to know TWO languages to be productive. That really makes my hiring much easier...

Another problem is there are some Java devs pushing Kotlin aggresively. And I also have lots of developers who do not know Kotlin. So the equation is make a small portion of developers maybe a bit more happy (and I don't believe they will be more productive) and make the rest of developers much less productive.

Why do you think it is a poor, repetitive language?

It feels to me like all programming languages are starting to converge in many ways.

I was just remarking to a friend in leadership that "you certainly wouldn't hand a blank slate to any but your most demonstrably-competent-with-the-blank-canvas lieutenants, and this is why we advocate for the boring things like Java and Python, because the patterns in those ecosystems are well known, and the React boffins are still reinventing the world of state management every thirty-six months."

Reply on news.ycombinator.com