Clojure on the Desktop
vlaaad.github.io
vlaaad.github.io
This app (Finda) hit a performance target of 16ms rerenders for incremental browser history + desktop file search - computers are fast, even if you use Electron.
- https://keminglabs.com/blog/building-a-fast-electron-app-wit...
- https://github.com/neon-bindings/neon
(edited for formatting)
If it's for multi-core computation purpose it can be with Worker Threads in JS or CLJS.
The point of the article is there are some applications where processing speed is important where Clojure's and JVMs concurrency and long running and large dataset performance is very important. And the ecosystem appears to be ready to make it possible to create apps where these strengths vis-a-vis Electron are combined with the other strengths of Clojure compared to low level languages (expressive syntax, interactive development).
Now we're seeing a proliferation of Electron apps - web apps - that are widely disliked/mocked because they are heavyweight, don't use platform-standard widgets, etc. And at least in this piece, an alternative is the JVM.
Trouble is the computers of 1996, combined with the less optimized JVM of 1996, were really not up to the task. And I would venture the GUI libraries were not there yet either. 24 years later we have very speedy JVMs and much faster computers and networks and toolkits like the one in the article (JavaFX) sound much smoother. Meanwhile Clojure offers concurrency primitives that are making threading much more accessible.
The wheel just goes round and round :p
Also worth looking at is Vlaaad's Cljfx (https://github.com/cljfx/cljfx). And I just want to say the number of examples that Vlaaad has created for the project is amazing and really helpful - it's a great example of helping others on-board to use your project - thanks Vlaad.
All the criticism here seems more like criticism of JavaFX itself.
What I mean is, at least for Java apps, this is a possibility. In Electron world, even though almost everyone has Chrome installed, you can't share it, you need to keep bundling the whole thing over and over again.
95MB for the macOS DMG.
177MB for the actual example app.
For a typical hello world application like the HN example, it appears that it would be no better than running an electron-made version given that they both include their own versions of their respective runtimes. In this case, this includes the entire Java runtime which Clojure needs to run.
I think I would be better off with using either C++ with Qt5 or even FreePascal with the Lazarus IDE for this instead.
- still a JRE is shipped (you MUST tailor it to the bare minimum with the flags) -> can be + ~30-40MB
- OpenJFX/JavaFX relies on Qt (especially for the WebView) -> can grow up to + ~70MB
- all the legal documents and docs are shipped together (you have to remove them with a post-process script) -> + ~5-10MB
Using JVM for desktop apps is expensive in terms of space. Time can be reduced consistently thanks to the graal native compiler, but still a C++ project outperforms Java GUI applications by 500%
https://github.com/openjdk/jfx/blob/master/modules/javafx.we...
I wonder where that 500% number came from, and yes C++ does blazing fast against Java in micro-benchmarks, except that on project delivery, developer salaries, time to market, additional costs for bug fixes (including memory corruption ones), available libraries, hardware costs for build performance also play a very big role.
> A web component that uses WebKitHTML technology to make it possible to embed web pages within a JavaFX application.
Qt was indeed there, just like Gtk, EFL and every other plaform that is part of the WebKit source code vendoring inside the Java source code tree.
http://hg.openjdk.java.net/openjfx/8/master/rt/file/f89b7dc9...
Which doesn't imply that the native code shipped with Java has any trace of Qt inside of it.
https://www.oracle.com/java/technologies/javase-jdk8-downloa...
http://hg.openjdk.java.net/openjfx/8/master/rt/file/f89b7dc9...
Of course, Rust and Go here a big game changers in this way
Well, I can also call C++ from Java.
In fact, once upon a time Qt had official Java bindings.
Rust, Go, etc is very far from being recommended for production desktop apps with GUIs.
Spending time with a profiler, we can probably improve the size and speed of a Clojure application (I've never tried using it for desktop software). No amount of time playing with Qt/Lazarus is going to give you Reagent, though.
(ns example
(:require [reagent.core :as r]))
(def click-count (r/atom 0))
(defn counting-component []
[:div
"The atom " [:code "click-count"] " has value: "
@click-count ". "
[:input {:type "button" :value "Click me!"
:on-click #(swap! click-count inc)}]])
Is better than this : import QtQuick.Controls 2.3
Button {
property int count: 0
onClicked: count++
text: count == 0
? "Click me!"
: "The value is : " + count
}
? (testable here : https://tinyurl.com/rjgcfod)I think this is a really cool library. I use clojure daily and love it. But this app isn't a very compelling demo over electron atm.
Lately I have been learning Swift and SwiftUI and application sizes for macOS and iOS tend to be very small.
Except for not being a lazy evaluation language, Swift development feels a little like Haskell, which to me is a good thing. I learned Swift back when TensorFlow for Swift was released and having one language that is adequate for many types of applications is useful. That said, when I want to be happy programming I try to use Common Lisp or Racket. I am approaching my 40 year anniversary for using Common Lisp and there is definitely something really great about deep familiarity with a language and it’s dev tools.
EDIT: also +1 for FreePascal. Someone hired me to use FreePascal on a project several years ago. Good language, small learning curve, and small and efficient applications. On small application sizes: I recently discovered that if you build SBCL Common Lisp from source, there is a compression option for building standalone applications that you can enable that gets application size down to about 15 MB, with application startup around 75 milliseconds.
Or many other things. Not sure why are we so afraid to develop native applications with minimal overhead. V and Zig is going this direction.
Remember those stupid arguments that a competent programmer can write well performing code [0] in any language? Here is the thing... You actually can't. Especially not with the JVM. Let's say you use a super efficient framework that is purely optimized for startup performance and low memory like Micronaut or Quarkus. When you look at your metrics you see that the Java program is consuming 10MB of heap space and pat yourself smugly on the shoulder thinking how well written these frameworks are. Tough luck. Even if you configure the heap size to something absurdly low like 32MB your JVM process will still use 120MB. Why? Because the JVM has a minimum footprint. It has lots of parked GC threads in the background that need 2MB stacks. It has to keep the class files in memory, keep profiling data, the generated code and lots of other things. If I need something that needs to consume as little memory as possible I just stick with C++ or Lua. With those two languages staying under 1 - 2 MB of RAM is trivial.
Also... it is possible your coworkers do not even set a maximum heap and just let it stay on the default values. That's how a tiny little CRUD app as a dashboard ended up allocating 10GB of RAM to itself on a server with 128GB RAM.
[0] well performing = low memory usage in this case
While this used to be true in the past, some of the GCs available to the JVM may now uncommit heap memory. This is for sure the case with ZGC [1], G1 [2] and I think also with Shenandoah [3]
[1] https://openjdk.java.net/jeps/351 [2] https://openjdk.java.net/jeps/346 [3] https://shipilev.net/jvm/anatomy-quarks/21-heap-uncommit/
From looking at multiple projects over many years (as both languages have improved), my rule of thumb is that writing performant Java is about 10x harder than writing memory-safe C++.
One particularly insidious, and related problem is that getting the first working Java program is slightly faster than doing it in C++. There are people that have built a career by producing a series of 90% Java solutions, and rapidly moving on.
They leave a wake of human suffering behind them in the form of large teams that perform heroic efforts in order to clean up the mess. This certainly wastes many billions of dollars a year.
This is a bold claim.
The JVM is incredibly good at generating fast code, to the point that even naively written Java is guaranteed to run in the ballpark of C/C++.
Writing memory safe C++... good luck with that one, especially as soon as you start using external libraries where memory ownership becomes very blurred.
Factually incorrect. Depends on the settings of the GC.
Programmers here tend to be experienced in many different languages and platforms, and when we see a post about packaging an app, we stop and ask what is it that gets packaged first. In other words, what's the point of easy packaging, if the packaged result is slow as molasses and eats RAM like there's no tomorrow. We have literally dozens of tools at our disposal which make packaging as simple, yet the result would runs circles around your demo, even if - and here's the kicker - we considered performance and memory optimization as non-essential. Sane defaults and all that...
To sum it up: we're just as interested in how you package it as in what is in the package. If it's really easy to optimize for startup time and memory footprint as you say, then all the more reason to apply these to the demo. No one here would think the demo is too complex (not simple) just because of that. And if it's not easy... well, that's also something we'd like to know.
The problem is Java sees lots of memory lying unused and figures ... why should I work harder to free it up? It'll waste CPU time and energy and I might need the memory later.
In theory it actually will do background GCs and free memory back to the OS anyway so I'm not sure why it's not happening here; possibly that feature needs to be explicitly activated or maybe I just didn't wait long enough to see it happen.
As for why it needs 350mb of RAM, let's take a quick look:
$ jcmd main GC.heap_info
Tue Mar 31 10:53:42 2020
61074:
garbage-first heap total 102400K, used 28139K [0x0000000700000000, 0x0000000800000000)
region size 1024K, 0 young (0K), 0 survivors (0K)
Metaspace used 58206K, capacity 65566K, committed 65868K, reserved 1101824K
class space used 10342K, capacity 12810K, committed 12928K, reserved 1048576K
So as a first approximation, even after a GC and subsequent heap shrink, it's holding onto about 3x what it really needs. Total heap memory is only 28mb. That's a bit odd. Perhaps G1 is still not aggressive enough in releasing memory back to the OS. For desktop apps it'd really be better to aggressively let go of heap as fast as possible (NB: native apps don't always do this either).The "metaspace" is holding a lot of stuff, mostly VM internal data structures. But I thought metaspace was primarily class metadata, but here there's only 10mb of that, so what's filling it up?
There's "jcmd main VM.metaspace" which tells us about 8mb of it is wasted due to fragmentation (I think). It doesn't tell us much about what's going on though.
These numbers look extremely high for a normal Java app. If I recall correctly Clojure [ab]uses the JVM in odd ways, for instance, by creating a class for every single function in the program instead of mapping them to methods as you would expect. This can easily cause a lot of overhead in class metadata compared to a normal program.
However the bulk of the memory usage you see is really just an "optimisation" - it's there, so why not be lazy and leave garbage lying around until you need the space.
Really? On my machine it eats 2gb for 10 tabs.
Well, it was worse with Chrome, though not by much: the main memory savings come from Multi-Account Containers, which is a really neat feature, but otherwise they seem almost tied.
You'll need to also outline what pages are in those tabs, how much each tab consumes, what extensions you have installed, how your config looks, what your OS is, how much history you have, what is your usage pattern, what experimental flags have you/the system activated and more.
It just depends on so many variables. 10 tabs of CNN is gonna use way more memory than 100 tabs of Hacker News comments, as a basic example. But you have a content script that is injected on every HN page with live feeds of something, now 1 HN tab might use more memory than 10 tabs of CNN.
I explained the situation in a reply to grandparent comment if you are interested to know why it is like that.
Nothing's been dismissed. The post is about a throw away program in a nascent React-on-JVM environment. People want to know why it uses so much memory. It's Clojure. Clojure is a memory hog. Case clojed.
Please don't be disheartened. It's a nice show and tell and resulting discussion.
https://gist.github.com/polymeris/0112389243890709108c4b8b0a... https://github.com/roman01la/proton-native-cljs
For anyone interested there will be a virtual meetup 16:th of April 6PM CEST: https://www.meetup.com/sthlm-clj/events/269204900
(defn counter [num]
(horizontal-layout
(on :mouse-down (fn [[mouse-x mouse-y]]
(swap! counter-state inc)
nil)
(button "more!"))
(spacer 5 0)
(label num (ui/font nil 19))))
Otherwise, haven't heard about Skia before, so thanks for sharing that, looks really nifty.Skia has worked well so far. Since it's used by Chrome, Android, and Firefox under the hood, it means that I can reach a lot of platforms "for free".
I love clojure but personally flutter seems way more viable for the use cases I could imagine using this.
Google seems to be putting huge amounts of effort on Flutter, and it can already run on the desktop (still early stages and in alpha I believe)... JavaFX is mostly orphan right now. Oracle seems to have no interest in it... so it seems to be currently being mostly advanced by Gluon[1], which honestly, does not have the muscle to pull out something competitive with Flutter or Electron.
JavaFX still has fundamental pieces missing... like setting an icon for the app or opening some URL in the system browser[2] (without using horrible hacks, which is what I've been doing - which get you lots of warnings since the module system was introduced, and may stop working soon). Not to mention no support for auto-updates, the fact that jpackage is still in "incubator" stage[3] and intentionally does not have JavaFX-specific features at all (it's just barely usable with JavaFX), the SceneBuilder, which could be such a great tool to develop UIs, similar to Lazarus, but it's nowhere as good, being in about the same unfinished state it was in around 2014, terrible documentation and lack of adoption (most examples you find online are from circa 2015, when JavaFX got a little traction)...
These things all make JavaFX a very hard sell for non-developers apps (if you're a developer who does not mind installing the appropriate JVM version to run the app, JavaFX works quite well - a 1MB jar can give you A LOT of features - one of my apps, my favourite and really useful, is only a 340KB single jar and works on any JVM 8+ - and looks as modern as any Electron app).
It's sad but the best way to get professional apps developed and distributed nowadays seems to be, by far, Electron (because of JS, and its stupid browser-based architecture)... but I am hoping Flutter will break Electron's hegemony soon as I have great experience developing Flutter apps on mobile and the web... Just try the online pad[4] to get a taste of what it looks like! It's amazing.
[1] https://gluonhq.com/products/javafx/ [2] https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8091107 [3] https://openjdk.java.net/jeps/343 [4] https://dartpad.dartlang.org/ecb28c29c646b7f38139b1e7f44129b...
As in stage.getIcons().add()? Or do you mean for the binary that jpackager produces? I wrap my apps with launch4j which allows that. Then I bundle them with jlink-made JRE (about 60 MB). Works pretty well, total size is ~100MB for the whole app with all the dependencies, (not a Hello World app by any means).
>opening some URL in the system browser
As in getHostServices().showDocument("https://google.com")? Then again, nothing's stopping you from using awt's Desktop class, not sure I understand the problem here.
Doesn't work and never did on MacOS.
EDIT: bug (closed as won't fix!!): https://bugs.openjdk.java.net/browse/JDK-8095033
> opening some URL in the system browser
HostServices also does not work in all but Oracle's distribution of JDK 8, which is abandoned now: https://github.com/javafxports/openjdk-jfx/issues/540
Gotcha, I work only with Linux and Windows. There was a problem where the icon was incredibly pixelated on Windows 7, but they fixed that.
> HostServices also does not work in all but Oracle's distribution of JDK 8
I see, I tested it only on Bellsoft's JDK 11, since I don't use 8 anymore.
* You can query elements arbitrarily deep using different patterns (depth, class names, ids, tag names, attribute presence/values).
* You can attach event handlers to arbitrarily deep nodes that you just queried, not just the nodes at the level your "controller" is in charge of creating/updating like traditional frameworks.
* Customizing style and behavior is entirely done via properties and composition rather than subclassing like traditional frameworks.
* And more!
That said, it may well not survive nearly as long as RN will, time will tell.
https://docs.gluonhq.com/client
When Clojure resolves all outstanding Graal issues, one could use Clojure for native apps too, but there are several bug tickets open on Clojure wrt Graal.
> javascript VMs choke on processing large amounts of data, which leaves them a better fit for advanced interactive forms than compilers or data processing pipelines
2005 called and wants it’s quotes back. It is trivial to spawn worker processes today and VM performance is only 2-5x behind compiled ones, often closer. Not to mention WASM.
The elephant in the room is not JavaScript or it’s VM but the browser engine.
EDIT: seems like JavaFX has its own browser-like “scene graph” and embeds WebKit, so this will suffer from the same bloat in memory use and app sizes?
WASM might be able to address some of the shortcomings of this stack, not sure.
The real issue is the size of these monster engines, but a lightweight engine that only supports the subset needed for apps is possible - see Sciter, and I have high hopes for https://ultralig.ht/
Is it? Have you ever tried? I think it is extremely hard to build reponsive UIs that work on every platform and there are no UX problems. We have to deal with a giant amount of boilerplate in many cases because using vanilla things (JS especially) won't cut it anymore. I am not even sure how many frameworks are targeting this space and try to solve the typical issues with these technologies and fail miserably. I think some of these issues are finally started to be handled the right way (Elm or ReasonML) but there are still some painpoints left (CSS). The question still, is it worth it? Couldn't we do better? The performance implications (energy consumption) for an average website is just hilarious. We should have a day where everybody disables JS and measure how much percentage of our computing capacity is going to render websites because we do not want to move away from HTML which was not (by any strectch of imagination) designed for this to start with. If anything, this should be a concern to most of us.
I’ve been doing it for 15 years.
> I think it is extremely hard to build reponsive UIs that work on every platform and there are no UX problems
It is hard on native platforms as well. Developing for mobile is not that far off from web today, everything converging towards flexbox or constraint solving, reducers etc.
I think you’re conflating the problem of a bloated JS stack with the platform itself. We were building perfectly fine interactive UI without any of this crap ten years ago, and can still do it.
Since Java 9 modules were introduced, the webview (and webkit) are only included in the app if the `javafx.web` module is used. Otherwise, it will not be included in your distribution if you pack it with jpackage.
I think using jpackage is easier to start with since there are less cutting-edge moving parts involved, and it imposes fewer restrictions on the runtime.
Browsers have simply set the bar too high.
Is this not exactly how React also works?
I wouldn't want to try and write it in javascript, but I'm entirely willing to believe that clojure/cljs got it right.
I'm glad that we are slipping away from Electron...
It goes even further as it get rid of the webview
https://microsoft.github.io/react-native-windows/
Native widgets, bridge between JS thread and the native world. That gives you good performances and the flexibility of JS/Typescrit and React to glue things together.
(I haven’t tried yet, but that looks quite nice on paper)
I'm curious to hear why you're glad we're going away from Electron into something that seems to be about the same thing, albeit a different language?
1. JS is kept for what it's meant to be and do; nothing more.
2. A WebKit webview is probably lighter than Chromium + NodeJS.
3. The whole thing should be managed to be linked as a shared library, avoiding many pitfalls of Electron's applications (by means of a semantic-versioned library).
4. The whole solution is really language-agnostic, despite my efforts with Go.