GenieFramework – Build web applications with Julia | Hacker News Reader

GenieFramework – Build web applications with Julia

genieframework.com

207 pointsmsravi43 comments

I'm curious if Julia's multiple dispatch could work naturally for server-side web development. In many content management problems, you often run into pages that are parameterized in multiple ways: the kind of data object being viewed, role based permission, and display capabilities, among others. In object oriented systems this often leads to double dispatch in one way or another. With Julia, it may instead result with an "just ahead of time" compile for each permutation; once that's done, page rendering should be snappy. One could use PackageCompiler to build an image with commonly needed permutations so that each computational unit's startup time is reasonable. During development, Revise.jl provides incremental updates to pages affected via code dependency graph.

If it works for Common Lisp and C++ [0], it certainly does for Julia as well.

[0] - In C++ you can emulate it via a mix of function overloading and template metaprogramming.

> With Julia, it may instead result with an "ahead of time" compile for each permutation;

This would be an odd approach, just have the first person who hits a new combination of function and type pay a few ms of compile time. That should work fine.

Oh dear. I meant to write "just ahead of time" (I corrected the post above). This is the Julia community's take on "just in time", but taking into account that there isn't an interpreted fallback. Exactly when a new type specialization is used, it is compiled then and there.

>there isn't an interpreted fallback

julia --compile=no

To Genie users, what is your workflow ? I love Julia for scientific computing but the compilation time is a bit of an issue. I would imagine I would need to compile a lot more when doing web dev. How do you work around this ?

If there is a solution to this, Genie would be awesome as I could directly put into production my Julia models without using a third party language.

Have you tried PackageCompiler.jl?

Honestly, this is an area where the op would be far better served by using go/rust/c++ or another backend language. Can launch a server with microsecond latency, and if it grows get good elastic behavior. No awkward route precompilation or thirty minute build times... Julia sure is neat, but this isn't an area where it's strong.

I do webdev in Go, and freely recompile as it takes a second or less on my hardware which consists of a rock, a rubber band and two sesame seeds on 20x2 liquid crystal. Its no struggle to recompile 100x in a day just iterating.

microsecond latency? on what exact hardware are you seeing that?

Must be new M1 Ultra or something.

P50 for basic routes using other web frameworks can be sub 0.5 ms. See the techempower benchmarks. These benchmarks are somewhat flawed, but I'd be surprised if genie was faster then flask all around, and flask is widely considered expensively slow.

0.5 ms is 500 times longer than the comment above, though

That was a somewhat hyperbolic statement. Make an application in node, genie, and rusts actix. Run them and see how fast it takes for the cli to load each of them. Feel free to package compiler the Julia one if you want. It's a good experiment with visual results.

I haven’t used Genie (though after this post I am tempted!) but I imagine compilation overhead would only occur once (e.g. the first time each route is used) and that you could pretty easily use precompile statements and/or PackageCompiler.jl to do the compilation before the service goes live. After that point, Julia should have produced fast code and provides a good runtime for these kinds of async/multithreaded applications with built-in M:N scheduling (perhaps a concurrent/multithreaded GC might help reduce tail latency but I imagine it should already be competitive with everything but well-written Rust/C++/C).

Revise.jl[1] is very helpful for development, as it allows for seamless recompilation without restarts. Cold start can also take a long time, so sometimes it helps to add options[2] such as -O0 when developing. Two paths forward for AOT compilation are PackageCompiler.jl[3] mentioned already in this thread (things like RelocatableFolders.jl[4] come in handy) and the possibility of GPUCompiler.jl[5] some day.

[1]: https://github.com/timholy/Revise.jl

[2]: https://docs.julialang.org/en/v1/manual/command-line-options...

[3]: https://github.com/JuliaLang/PackageCompiler.jl

[4]: https://github.com/JuliaPackaging/RelocatableFolders.jl

[5]: https://github.com/JuliaGPU/GPUCompiler.jl/issues/3

I only use Genie slightly to add a simple UI to my projects - I'm no web dev. Time to first request is as terrible as you would expect, but one can then dynamically iterate on a running interpreter by changing and adding pieces, and then it's not that bad as only the new bits need recompilation.

I never tried the fancy to speed up compilation.

I would caution someone interested in this to consider some of the ramifications of using Julia / Genie. When I last used it, it was a extremely painful development experience because of the time it took to for Julia to reload when "live reload" was enabled. Further more, many features like authetication were nowhere to be found. Also, startup time for the server was SO large that it just does not work on heroku, even with PackageCompiler. And forget about using AWS Lambdas or Cloudflare pages or something like that. IMO, Rust, Go, Javascript or even Python is a waaayyy better option for this.

Kudos to the devs for continuing this work though. I'm sure it'll get there some day.

If you absolutely want to use Julia because you have an existing Julia code, you can use HTTP.jl instead. It's more lightweight and you can build routes using that into your application.

Adding a couple of responses from Genie's creator [1], when parent commenter mentioned these issues a few months ago, for visibility in case it's helpful to someone here:

> startup time for the server was SO large that it just does not work on heroku

Response: The solution (which granted should be maybe better documented) is to set the `EARLYBIND` env variable to `true` - this will allow Genie to bind to the port very early so Heroku will love it.

> many features like authetication were nowhere to be found.

Response: Genie has built in support for handling (encrypted) cookies and sessions and plugins like GenieAuthentication (or GenieAuthorisation) and SearchLight provide straightforward features. Have you used these?

----

With that said, I myself do think of Genie as (currently) something to be used when (a) you already have Julia code and need a web front-end for it, or (b) you want to experiment and possibly break things for the potential of expanding horizons.

[1] https://news.ycombinator.com/item?id=29830003

Thanks for sharing the information about EARLYBIND and GenieAuthentication.

I will say that when I used EARLYBIND it still took too long for Heroku. That's not to mention that didn't solve any of the live development workflow issues.

I looked into GenieAuthetication but it doesn't support SSO and a bunch of other features that other modern authetications libraries support.

Folks might be interested in: https://www.freecodecamp.org/news/how-to-build-web-apps-in-j...

Good stuff! I think it's important that analyses done in Julia can be presented in dashboard form with Julia tools as well. Having a one language stack is just that much less work.

I'm curious about this project.

Web development benefits a lot from near-instant recompilations, which I heard aren't possible with the Julia programming language as codebases grow. Is that still the case?

Revise.jl is used to provide incremental runtime recompilation; and Genie now has Revise built-in.

Good to know, thanks for pointing it out.

Probably worth staying revises limitations. Ie can't change structs, etc.

I started a new web project recently and I decided to either use Elixir/Phoenix or Julia/Genie. I wanted to use the later but two main issues led me ultimately to the former. 1) JIT compile times to first view were very slow — like 10 seconds, 2) Macro usage in Julia seems too common making the code ugly and difficult to read. e.g. @with_kw. What’s the point of an elegant language if you have to annotate it with obtuse macros at every turn?

In regards to startup time, isn't Julia an order of magnitude slower than everything, even Ruby? Why would anyone want this?

For this use-case, PackageCompiler (for dependencies) + Revise (for your code) fixes that.

This is just not true. Even with PackageCompiler I was getting 30 seconds or more startup time for a medium size codebase with Genie.

Also package compiler takes a long time to run...

A webserver spends a minuscule amount of its time in startup.

Startup time in a big Spring Boot or Rails project can easily be two minutes or more. Sadly I speak from experience.

why? i like Julia but it's not a great general purpose language IMO.

This looks utterly bananas. I mean… why. Because variety is the spice of life? How are also those companies using this framework?

Imagine spending a lot of time on learning Julia, then imagine you want to do web dev, then imagine your favorite language is Julia, and you don't want to use time on learning something else? So why not I guess. From reading the landing page, I don't see a compelling reason to use it unless you are already in the ecosystem, which is fair.

Frameworks are helpful when a community would benefit by having a bit less variety in how things are done. Even if it's sub-optimal for most individual use cases, having a standard way of building applications in a community makes it easier to construct and maintain projects.

why not ?

People are free to use their time & energy to develop interesting things in whatever language/toolchain/techstack they like and share with the community.

That is how ecosystems evolve, expand, and new ecosystems emerge.

We don't want to be locked into incumbent solutions forever.

Not everything needs to have corporate support or large scale use from day one. It doesn't even have to be useful at all on day one, or ever for that matter.

Single language stacks are simpler. For heavy duty webdev tasks you should use a real web language. For simple sites this looks fine.

Julia is a fairly elegant nice language and multiple dispatch makes a lot of sense in all sorts of places.

I don't really get why Julia is pigeon-holed for numerics. I prefer its syntax to Python and would consider it for scripting applications where I use Julia now.

I also think Julia is a very nice language and I prefer its syntax over Python.

However, I can understand why Julia is "pigeon-holed for numerics". It started as a language for scientific computing. Today, it promotes itself as a general-purpose language suitable for any task - not just for scientific tasks. Only time will tell if perception of the language will shift. As this discussion shows though, the association with scientific computing remains strong.

> It started as a language for scientific computing.

Did it? The way I read the famous old “Why We Created Julia” blog post (https://julialang.org/blog/2012/02/why-we-created-julia/), it was intended as a general purpose language from the beginning. Quote:

“We want a language that's open source, with a liberal license. We want the speed of C with the dynamism of Ruby. We want a language that's homoiconic, with true macros like Lisp, but with obvious, familiar mathematical notation like Matlab. We want something as usable for general programming as Python, as easy for statistics as R, as natural for string processing as Perl, as powerful for linear algebra as Matlab, as good at gluing programs together as the shell. Something that is dirt simple to learn, yet keeps the most serious hackers happy. We want it interactive and we want it compiled.”

The language may have been intended to be a general-purpose language from the beginning, but my recollection is that the focus was firmly on scientific computing (and remains the principal sphere of activity). The quote you shared also seems to hint at that focus.

Reply on news.ycombinator.com