The Sovereign Tech Fund invests in Scala | Hacker News Reader

The Sovereign Tech Fund invests in Scala

scala-lang.org

117 pointsbishabosha91 comments

Scala isn't as hot as it used to be. I think the rough Scala 2->3 transition, coupled with improvements in the base Java language, emergence of Kotlin + Android support, and popularity of Python in data science and data pipelines (lets just do everything in one language became popular) kind of made Scala not quite as popular as it could have been. Plus the long compile times are a pain. However it seems to have a really high coolness ratio for a language. The few jobs I do see in Scala are very cool looking. Very few boring looking jobs.

What would you categorise as a "cool looking job"?

Jobs that you work on hard, complicated things. Scala is relatively popular in Fintech and Finance in general. There's things like Chisel or Spark. But there are relatively few simple CRUD app companies using Scala.

Scala is too complicated. Most scala code bases I have worked on have no enforced structure, the language allows for all sorts of unconventional programming paradigms

That's sorta the curse of Lisps too.

Yeah sorta

I would say Clojure is a big exception to that - Clojure applications tend to be more uniform than even non lisps

Clojure is an exception to pretty much everything in the category. I really wish I had the opportunity to use it in my professional career.

This is exactly what turned me off. It supports so many paradigms that every line of code I wrote I had to sit and think if I was doing it the "right" way and it was miserable.

Part of that I think is the culture and not the language. Personally I try to use the least powerful method that gets the job done and that usually keeps me unblocked. In practice that usually means using it as a better Java and not going down the functional monad path. I know scala has gone through a rough patch and maybe migrating from 2 to 3 is painful. But if you try starting a new project now with the latest Scala 3, I think you'll find that its pretty nice. Even IDE support is pretty good.

Yeah I love it when people start defining their own operators all over the place and give them all inscrutable names. "Dude just use the eggplant parm operator: <<=-=>>"

This is the type of thing that a good PR review culture will handle. I love that this is an option in some languages. But in a company, you need to decide what cool features should be used and when and how much.

Good PR review isn't really enough unless the organization is only large enough to handle around one PR at a time.

With languages like Scala I think its a clearer necessity that someone or some small group in an organization maintains a dominance of expertise or you have different groups that are only using the same language in name or facing overhead to keep in agreement where a lot of the best developers might be basically doing ambassador work.

yeah a small group of experts can leverage scala, its not a language for a corporate environment

^ This meme is from 10+ years ago when Scala was at the peak of its hype driven by the FP craze. Nobody seriously writes cryptic-symbolic-operator code like that nowadays. Scalaz, the FP library most notorious for cryptic operator/method names, hasn't been relevant for many years. Today everyone uses Cats, ZIO, or plain Tapir or Play, all of which are quite ergonomic.

One reason why I keep bouncing off of Haskell.

I think its because we dont have too many established paradigms for functional programming. Having said that I think Scala is just marvellous. I had to work with a Scala codebase written by a set of very mature devs and it was an absolute joy. It influenced how I write Java.

You got it. Scala had a shot being an early mover in the JVM functional programming space, but they really shot themselves in the foot with their version transition problems and tooling issues you allude to. Java is probably "good enough" for most shops now, and if you are not bound to the JVM I really don't understand why you would go with Scala today.

for spark

Scala's decline started before Scala 3, which brought its share of breakage (sometimes for dubious reasons, like the new syntax) but also fixed many warts. Tooling has improved a lot lately, but it's too late.

> if you are not bound to the JVM I really don't understand why you would go with Scala today.

Scala's metaprogramming abilities coupled to a powerful type system are still unmatched. Among mainstream languages, only TypeScript gets somewhat close. For your typical service oriented architecture, libraries such as Tapir or ZIO HTTP are pretty nice. I haven't found anything as pleasant in other languages.

That said if an LLM can write 95% of your code today, this point is a bit moot, sadly.

It's not really moot though. The primary benefit of concise but robust frameworks like ZIO is that they are easy to read (like the program is mostly business logic with minimal syntax noise/programming language bookkeeping), not that they're easy to write (though that's true too). Metaprogramming also works nicely with LLMs because you get the expressiveness of something like Python (or better) while retaining a strong type checking layer to give a feedback loop to the LLM. In fact, it kind of shines with an LLM because you largely don't care if things like macros are ugly as long as they produce correct code to present to the typechecker/compiler, so it makes more sense to vibe code the metaprogramming layer to give yourself whatever you need to have straightforward business logic. Conciseness and direct encoding of business logic also helps to keep context focused.

Really this is all true with humans too, but IMO it's multiplied with LLMs because they are insanely capable at dealing with the guts of metaprogramming wizardry if they need to, so you don't end up in a world where that one guy that understands it leaves and then no one can possibly maintain it.

I haven't used Scala for quite a while now - but a while back they had a serious asshole problem with a lot of the community.

Scala’s downfall is the culture of writing libraries with the most esoteric syntax imaginable. It’s the opposite of Ruby’s “program with pleasure”—-it is “program with maximum cleverness” and “if it doesn’t look like an obscure math equation or symbol soup, it’s probably not idiomatic Scala.” Java’s designers were opposed to operator overloading due to potential for abuse, and did Scala deliver living proof/nightmare scenario of that.

Now I still want operator overloading, because I favor enabling instead of disabling approaches, and I want, for example, new numeric types to have natural syntax that blends in with built-ins, or an equality operator that works consistently. I’ll even settle for += append to strings and collections.

But even Scala standard libraries take operator use to wild extremes. Apparently it’s not enough to use + to append collections, you must use ++. Except Vec you can add with +. There’s ++= and ++=:, and /: and :: and ::: and ?^ and ?^.

Spark Scala interface introduces =!= for inequality comparison. There’s |@| from Cons library. Don’t get me started on Akka. SBT introduces <++=

Surprisingly you can’t override ==, so Spark implements ===. And specs2 testing library implements ====.

I've onboarded experienced and inexperienced developers to fairly big Scala codebases for the best part of a decade and I can't think of a single time symbolic operators were brought up as a source of complexity of confusion. Sbt deprecated <++= 10 years ago.

Agreed. There was a period of time roughly 10-15 years ago where symbolic operators in Scala were very en vogue. That fell out of style in a big way and I haven't encountered symbol soup in a very long time.

Most of the conversations I have with folks about Scala issues these days center around implicits, tooling, and its decline/lack of popularity.

I agree. I took Martin's fantastic online Scala class way back when and I was a fan. Now Clojure is a great JVM alternative, and as you say Java has kicked it up a notch.

I learned recently that one of the killer apps for Scala seems to be in hardware design. Chisel [0] is the core technology of the best open source RISC-V chips. Chipyard [1] is designing leading edge type OOE and AI chips and all of the code is written in Scala. Personally, I can't wait for some of these designs to start being mass produced and put in laptops and phones.

[0] https://en.wikipedia.org/wiki/Chisel_(programming_language)

[1] https://github.com/ucb-bar/chipyard

So, as a justification for support of scala, the thing that seems lacking to me is that Chisel seems to still be centered around scala 2? E.g. their recommended template for getting started still uses scala 2 ... so without support to motivate them to use scala 3, it's not obvious that Chisel benefits much from current work on the scala language? I have not fully understood the Chisel project but I see they have a "compiler plugin" which suggests to me that moving fully off scala 2 requires a meaningful redesign.

https://github.com/chipsalliance/chisel-template/blob/main/b...

I’m not super plugged into scala but I work with Spark quite a bit and my observation has been the whole scala 2.13 -> 3 transition is a huge mess for just about everyone who touches it. I don’t have enough hands-on context to understand why it’s so painful but it seems to be similar or worse to the python 2.7 -> 3 transition in terms of sticking friction.

It is a mess. I've spent some time trying to convert some academic oss projects and some removed features really force large redesigns. I think rather than funding the stuff on this announcement, I wish they would fund a team of experts to work on migration of a prioritized list of projects. This would both provide example patterns of migrating substantial projects and unblock projects who have been saying "we would like to try migrating but library X we use still hasn't"

Well, a more optimistic take here is that if future development on the Scala language was funded explicitly by/for people who are current using Scala 2, that means that the developers would more clearly understand their requirements in terms of making an easier transition for users moving from Scala 2 -> 3

Chisel absolutely isn’t the type of software that benefits from upgrading because it’s largely standalone. They could be the last project still stuck on 2.x a decade from now and it wouldn’t make much of a difference to its users.

I’ve only used Chisel for a few projects but I’ve never used anything but Chisel in those codebases. Simulation, verification, and all the painful stuff in FPGA/ASIC development depends on non-Scala tooling and all of the inputs (parametrization) are just read in from JSON files produced by scripts in other languages.

It would be nice to be up to date but the hardware NRE is so damn high that working around any limitations in Scala support is a rounding error. Chisel’s outputs are sent out for $X00,000 fab production runs so no one gives a damn whether it’s Scala 2 or 3 as long as it ships a working IC. The last time I used Chisel I was working on a mixed signal design where the Synopsys Fusion Compiler (maybe Custom compiler?) licenses alone ran into the hundreds of thousands per year (iirc it was per seat, so we must have spent over a million per year on Synopsys alone).

Chisel picked up Scala 3 support in 7.0.0 released last September (https://github.com/chipsalliance/chisel/releases/tag/v7.0.0), right now they cross-compile between Scala 2 and 3 but from what I heard they should eventually move to Scala 3 only to start taking advantage of new features like named tuples (https://docs.scala-lang.org/scala3/reference/other-new-featu...).

This is great news, nice win for Scala.

It's a great language, I've been working with it for 10 years now. Full stack Scala with Scala.js on the frontend is so very nice. My experience is mostly in fintech & healthcare startups where the language helped us get correctness, refactorability, clarity, and high velocity at the same time without blowing up the team size.

Initially I learned Scala on the job, but I've been writing open source Scala for years since then. It's a cool language to learn and explore ideas in, since it has lots of elegantly integrated features (especially FP + OOP).

Scala may not be the #1 most popular language, and that's fine. Popular stuff surely gets the benefits of funding and attention, and sometimes lacking such support is really annoying, like a few years ago when Scala 3 was first released, the IDEs took a looong time to catch up. But I still choose Scala despite those occasional annoyances, even though I also have years of experience in JS / TS and other languages. It's just a much better tool for my needs.

Why would the governments invest money on such a niche language? "Scala is widely used to build and operate essential systems across multiple industries." - very bold statement.

I made another comment about it. But the answer is Scala is the number one language for building hardware these days via Chisel.

https://www.chisel-lang.org/community

second link - 404

third link - achieved project on github

fourth link - educational project

Perhaps it's a very know and useful project, yet indeed seems very niche to me.

Chisel is very neat but "the number one language for building hardware"? VHDL and SV are the only things in this space that actually matter. Chisel is still a blip for now.

"...these days." Don't quote me out of context.

I genuinely don’t understand what you think “these days” means or how what I said was incompatible with it.

Ok, let's say you were going to start a brand new project today. What tech would you use?

Hmm, I think we're talking about different kinds of "number one".

The ancestor comment called Scala "niche" and criticized the characterization of it as "widely used". So given that context, I was coming at this from a perspective of popularity; Chisel is orders of magnitude nicher than even Scala itself and orders of magnitude less widely used. Most of industry is still choosing VHDL or Verilog for most greenfield hardware projects.

I think you mean in terms of "best way to do it". Chisel can at least lay claim to that crown, sure, though I think you could say the same about Scala too.

(I might say Clash for hardware and Haskell for software, not that that does me any good.)

As a European project the foundation is likely pursuing funding from EU sources using the fact that it isn’t US tech as a selling point.

Probably because it's an amazing language for people who know what they're doing

Is it niche? Scala is arguably the single most successful functional language. It interoperates with the whole JVM ecosystem. It's probably the #3 JVM language after Java and Kotlin.

Spark is Scala, Twitter was (is?) Scala https://sysgears.com/articles/how-and-why-twitter-uses-scala...

Anything that runs on JVM should not be invested in.

Why? I bet you never even wrote custom VM for your lang/architecture.

There are like dozen of implementations for JVM alone: sun, Oracle, gnu, IBM etc etc

Because the idea of classes is baked into JVM. So when writing inline code, with groovy, scala, kotlin, whatever else, the compiler has to basically insert fake classes into the jvm.

This may seem trivial, but because its fundamentally "hacky" its not something that is ever going to be as easy to work with and expand.

This is nonsense. And, not that there's anything wrong with classes, but compilers no longer need to 'insert fake classes' for anonymous functions since Java 8.

Pretty sure everything in the jvm has to be tied to a class structure by definition.

Yes, and? Classes are not inherently bad, hence saying that JVM should not be invested in because its bytecode requires classes is nonsense.

re: inline lambdas, they have to be _tied_ to a class, and defined as methods, indeed. But they no longer have to be standalone classes themselves since JDK8

There's also scala.js and Scala Native, even if the JVM is the primary platform.

Scala was niche (but of course, you need to define niche first) and its market share has shrunk further in the last decade.

> Scala is arguably the single most successful functional language.

That would be Javascript.

You're being downvoted but you aren't necessarily wrong. Javascript is probably one of the most approachable functional-like languages without being dogmatically functional.

Scala may have fallen out of favor but was quite popular few years ago. And perhaps still is the most popular EU-designed language (developed by EPFL).

Python. Python is the most popular language designed in EU. EPFL is not even in EU

Others worth mentioning are Kotlin, Ada, Pascal, Haskell, Zig, Erlang, Elixir, Prolog, Ocaml.

Former EU, there's Gleam.

Kotlin is from St Petersburg, it's even named after an island near it.

St. Petersburg is in Europe.

I thought what was worth mentioning was being European or not, rather than being part of an European legal entity or not.

PHP is also from Europe. Not sexy perhaps, but popular enough to mention.

How is Zig an EU language?

Andrew started the language while he was in Germany.

Sadly, I’m old enough to remember that Ada was the result of a US initiative to standardize a capable language for embedded development.

A good friend worked on the well regarded Telesoft compiler…

True to both. My brain not braining. Was thinking Europe-based/driven. Python started in CWI but PSF is USA-based.

> EPFL is not even in EU

For people who don't get this, EPFL is the Swiss Federal Technical Institue in Lausanne. Switzerland isn't part of the EU or EEA but has instead integrated itself with the EU very closely via a mindboggling number of bilateral agreements with the EU members and the Schengen agreement which allows for free, borderless movement. This has the effect of making it seem very much like they are part of the EU without actually being as such.

https://en.wikipedia.org/wiki/Switzerland%E2%80%93European_U...

Maybe they've applied for the grant 10 years ago when Scala was all rage

It wouldn't have to be used in very many places to justify a 377k investment. A few big European banks alone would be worth it. Their website says "we invest globally in the open software components that underpin Germany's and Europe's competitiveness and ability to innovate". The fact that Scala is used at a university could also be classified as innovation. This is a minor amount of money if you're going to compare it with a STEM or medical research grant.

I don't know what it looks like on the ground now, but Scala was the defacto language of data infrastructure across the post-Twitter world of SV late stage/growth startups. In large part, this was because these companies were populated by former members of the Twitter data team so it was familiar, but also because there was so much open source tooling at that point. ML teams largely wrote/write Python, product teams in JS/whatever backend language, but data teams -- outside of Google and the pre-Twitter firms -- usually wrote Scala for Spark, Scalding etc in the 2012-2022ish era.

I worked in Scala for most of my career and it was never hard to get a job on a growth stage data team or, indeed, in finance/data-intensive industries. I'm still shocked at how the language/community managed to throw away the position they had achieved. At first I was equally shocked to see the Saudi Sovereign Wealth fund investing in the language, but then realized it was just 300k from the EU and everything made sense.

It's still my favorite language for getting things done so I wouldn't be upset with a comeback for the language, but I certainly don't expect it at this point.

Did the Scala 3 changeover blunt its momentum, you think? Or just Python won out in a sort of worse-is-better way?

Mostly the latter. Scala 3 is almost completely irrelevant to the big data space so far. Databricks took six years before upgrading their proprietary Spark runtime to Scala 2.13. Flink dropped the Scala API before even moving to 2.13. I don't know if Scio will seriously attempt the move to Scala 3. All of them suffer from Twitter libraries being abandoned, which isn't insurmountable, but an annoyance still.

And I don't think it matters anymore. I predict that the JVM will eventually be out of the equation. We're already seeing query engines being replaced by proprietary or open source equivalents in C++ or Rust. Large scale distribution is less of a selling point with modern cloud computing. Do you really need 100 executors when you can get a bare metal instance with 192, 256 or 384 cores?

People want a dataframe API in Python because that's what the the ML/DS/AI crowd knows. Queries and processing will be done in C++ or Rust, with little or even zero need for a distributed runtime. The JVM and Scala solve a problem that simply won't exist anymore.

Yeah, this is certainly the correct take. There's an alternate timeline where the Scala community focused during the peak on making it a better language for numeric computing/ML rather than the Nth category theoretic framework, but here we are. At a job almost a decade ago, we made some progress on an open source dataframes (and unfortunately proprietary data visualization) library for Scala, but we didn't get far enough before the company closed and the project died [1].

Still my favorite language I had the privilege to work with professionally for over a decade. However, in this post-JVM world, I'm actually excited to see a lot more OCaml discussion on here lately. The Jane Street work on OxCaml is terrific after a long period of stagnation with the language. I'm using it for most of my projects these days.

[1] https://github.com/tixxit/framian

For one, it is a programming language born in Europe, and mostly developed in Europe via EPFL.

For our technology freedom we need to focus on programming languages where PR aren't coming from contributors living in adversary nations.

As a longtime Scala lover, I’m so happy to see this. Everyone in here hemming and hawing about version incompatibilities, build tooling and such conveniently forget the warts of other languages and their ecosystems. Scala is an incredible language, especially for the language being so flexible, which is a strength, not a weakness.

Very on brand for Germany to invest tech 15 years past its prime.

This was a good joke. Surprised it got downvotes. Must be all the Germans who can't take a joke.

(In case you Germans still don't get it -- that was a joke, but feel free to downvote :) )

More like 10. The Scala ecosystem was still really rough in 2011, remember how many companies got burnt trying to follow Twitter's footsteps (cf. the infamous Yammer testimony).

As a fellow german, this was the very first thought that popped into my head.

Happy to see investment in sbt and the stdlib

Sad to see code coverage tooling called out as something they’re spending money on

Happy to see scala get sponsorship

The lead maintainer of sbt does it as a labor of love. I am very curious if he actually will be receiving any money.

He won't: https://bsky.app/profile/eed3si9n.com/post/3mdgbjkg4ck23

Hmm, I wonder when Scala will be possible to bootstrap from source.

https://bootstrappable.org/projects/java.html https://bootstrappable.org/projects/java-tools.html https://bootstrappable.org/projects/jvm-languages.html

I am very happy for Scala. So many people taking the time to rant on it. Yes, you can do anything with Scala in a million different ways. So what? So can you do it in C++, Python, Rust, etc. I agree that the whole "Category Theory" libraries are way over the top, but so are libraries in Java using "factories" everywhere. Every language has its pros and cons.

I'd say category theory concepts are not over the top - but category theory libraries could be! I just about never define or use explicit categoric typeclass instances - even though I almost always define categoric methods, e.g. `++` and `empty` for everything that looks like a monoid, `map` for everything that produces values, `contramap` for everything that consumes values. That's usually more than enough - and making structure explicit that way prevents your data structures from semantic drift while they're being handled by other engineers.

That's the problem with state investments in software. One can rightfully complain about misallocation of capital by private investors, but state investments are a whole new level.

The Albanian government invested 10M in Mira Murati's startup.

All on the basis of her...being Albanian.

Talk about public money mismanagement.

Is it really accurate to call this an “investment”? The details are not known but it looks like a grant or donation by a charity rather than an investment?

I think that any time to expect to get some benefit from it, other than a nice feeling for helping out, it should not be called a donation, at least not a charity donation.

I've always liked Scala as a language, but it's challenging to write high-performing and memory-efficient code on the JVM in general. Whenever you raise this issue, you'll encounter a horde of JVM fanboys who insist that it’s not true, giving you all kinds of nonsense excuses and accusing you of not measuring performance or memory consumption properly. If you genuinely want to produce well-performing JVM code, you're essentially writing C-style Java. As soon as you introduce abstraction, performance issues inevitably arise – largely due to the fact that features and modernizations from Project Valhalla haven’t yet been implemented/shipped. Scala proponents will suggest using macros and opaque types, but at scale this approach becomes incredibly cumbersome and even then you won't be able to completely prevent boxing that would actually be unnecessary; you could just as well be writing Rust.

What a waste of my taxes.

Happy to throw some of mine for that. Scala may not be as popular as some other languages, but it's relevant for use cases as diverse as those for which you would otherwise use Java/C#/Go, JS/TS, C/C++/Rust (via scala-jvm, scala-js and scala-native), and it's a very nice language to work with at that.

Reply on news.ycombinator.com