Glad to see things are progressing!!
This is similar, except more boilerplate and much, much slower.
[0] Objects that need pinning are pinned(by toggling a bit in object header), byrefs are pinned by simply storing them on the stack, arguments that need marshalling involve calling corresponding marshalling code. That code can allocate intermediate data on heap, on stack or call NativeMemory.Alloc/.Free C-style.
[1] Overhead can be further reduced by 1. annotating FFI calls with [SuppressGCTransition] which saves on possible arguments stack spills and GC helper call, replacing the call with a single flag check and optional call into GC in epilog, 2. in NativeAOT, p/invokes can be "direct" which saves on initialization checks and indirections (though they are reduced in JIT as it can bake data directly into codegen after static init has finished on recompilation). This has a tradeoff as system's dynamic loader will be used at application startup instead of regular lazy initialization and 3. direct p/invokes can be upgraded to static linking, which transforms them into direct calls identical to regular C calls save for the same GC flag check in post-condition. This comes with compiling .NET executables and libraries into a single statically linked binary (well, statically linked for the native dependencies the user has opted into linking this way).
While a step closer to Valhala, the whole dev experience is still quite lacking versus what .NET offers.
Currently is too much like making direct use of InteropServices.
Doubtful, given that this is something we worked hard to avoid. To be efficient, a P/Invoke-like model places restrictions on the runtime, which inhibits optimisation and flexibility and this cost is worth it only when native calls are relatively common. In Java they are rare and easily abstracted away, so we opted for a model that offers full control without giving up on abstraction, given that only a very small number of experts (<1%) would directly write native calls and then hide them as implementation details. I'm not saying this approach is the right one for all languages, but it's clearly the right one for Java given the frequency of native calls and who makes them.
Of course, you can wrap FFM with a higher-level P/Invoke-like mechanism, but it won't give you as much control.
I will rather keep writing C++ with JNI, instead of enduring the current boilerplate, specially if I already need to manually create header files to feed into jpackage, for basic stuff like struct definitions, which I don't feel like writing by hand.
As for performance, this is something I agree with neonsunset, unless we see Techpowerbenchmarks level of Panama beating P/Invoke, it is pretty much theoretical stuff at the expense of developer convience.
s/jpackage/jextract/gI would encourage those who think that we're consistently making suboptimal choices for Java compared to choices made by significantly less successful languages to consider whether it is possible that their preferences are not aligned with those of the software market at large. Java is and aims to continue being the world's most popular language for serious server software, and that requires tailoring designs to a very large audience.
I always notice a certain lack of respect on forums such as HN for the world's most consistently successful and popular languages -- JS, Java, and Python. Different programmers have different preferences and I'm all for rooting for the underdog now and again, but you simply cannot consistently make wrong decisions over a very long period of time and yet consistently win. What we do may not be everyone's cup of tea (no language is), but it is clearly that of a whole lot of people. We work to offer value to them.
[1]: E.g. the design of native interop has significantly impacted that of user-mode threads (or lack thereof: https://github.com/dotnet/runtimelab/issues/2398) in both .NET and Go, and we weren't willing to make such tradeoffs in either performance or programming model.
That is it, other use cases, have other programming stacks.
As such our native libraries are written in consideration to be consumed at very least, across .NET (P/Invoke, C++/CLI, COM), Java (JNI), nodejs (C++ addons), Swift.
So to move the existing development workflow from JNI to Panama, it must be an easy sell why we should budget rewrites to start with.
Also in regards to "hate", if all decisions were that great there wouldn't be needed to create a new library support group to help Java ecosystem actually move forward and adopt new Java versions, as I learned from JFokus related content.
To elaborate just a bit more on what I wrote in my previous comment, to get a straightforward interop with C you need to place certain restrictions on the runtime which limit your ability to implement certain abstractions such as moving GCs and user-mode threads. Because native interop requires special care anyway due to native memory management, which makes it significantly more complex than ordinary code and so less suitable for direct exposure to application developers -- so it's best done by experts in the area -- and on top of that native calls in Java aren't common, we decided not to sacrifice the runtime in favour of more direct interop. As a result, native interop is somewhat more elaborate to code, but as it requires some special expertise and so should be hidden away from application developers anyway, we decided it's better to place the extra burden on the experts doing the interop rather than trade off runtime capabilities and performance. We think this is the better tradeoff for Java. Consequently, we have both compacting collectors and no performance penalty for native calls on virtual threads. Other languages made whatever tradeoffs they thought were right for them, but they did very clearly sacrifice something.
In general, being successful and popular had little to do with how well a PL is designed. Visual Basic, PHP, and even C are some historical examples that I have plenty of personal experience with.
Perhaps, but it is fairly easy to design a product for a small, self-selecting group of fans who find the aesthetics appealing and so declare the design good for their taste. Unless a language becomes heavily used in codebases that are maintained for years by a large variety of programmers, it's hard to tell how well it is actually designed as a mass-appeal product.
Two of the three languages you mentioned weren't able to attain nearly the same success as Java for as long a duration. I'd give C a similar success score because what it lacks in popularity it still makes up for in longevity, being almost twice as old. There are good reasons for why C is still as popular as it is. For example, in its domain -- which requires compilation to exotic architectures -- "good design" entails being able to easily implement efficient compilers.
And to be clear, I'm not advocating for aesthetics here. It's not like C# is a model of purity, either; but I would say that their choices over the years have been more pragmatic overall from the perspective of someone who needs to write readable, good quality code without jumping through too many hoops or getting lost in the verbiage.
I personally don’t agree with that, C# is very “impulsive” at adding new features, which sounds cool in isolation, but makes the language significantly more complex to understand, and has non-intuitive interactions with other features.
I think C# is quick at going the C++ way, and there is no return from there if we guarantee compatibility.
I much prefer Java’s approach, where yeah, at times one might lack some syntactic sugar/nicety (often greatly overcome by IDE/tooling’s advancements), but over time they do add important ones, but only commit to features that have been earnestly tried and sustainable.
To be fair to Microsoft, they have favoured rich, complex languages for a long time now. They were probably the biggest champions of C++ and TypeScript also doesn't seem to be going down a particularly minimalistic route (an understatement). They're fans of rich languages, and while such languages are not my cup of tea, they do have a large audience (although I think it's a large minority audience).
My point is that when people say that one language is technically superior to another, what they really mean is that it's superior in the technical aspects that they themselves value more than the aspects where the other language is technically superior. This is all fine, except that these personal preferences aren't distributed equally. This is a little like the Betamax vs. VHS debate. Sure, Betamax had a superior picture quality that some valued, but VHS had a superior recording time, which others valued but that latter group was bigger.
As for C# -- strong disagree there. I think they're making the classic mistake of trying to solve every problem in the language and soon, resulting in a pretty haphazard collection of features, quite a few of them are anti-features, making up a pretty complicated language. For example, they have both properties and records, while in Java we figured that by adding records we'll both direct people toward a more data-oriented form of programming and at the same time make the problem of writing setters so much less annoying to the point it shouldn't be addressed by the language (while properties have the opposite effect of encouraging mutation). They've painted themselves into a very tight corner with async/await (the same with P/Invoke, which constrained their runtime's design making it harder to add user-mode threads), and I think they've made a big security mistake with interpolation -- something we're trying to avoid with a substantially different design. Also, while richer languages do have a lot of fans, all else being equal more people seem to prefer languages with fewer features. Our hypothesis is that it's better to spend a few years thinking how to avoid adding a language feature (like async/await or properties) than to spend a few months adding it.
Also, every feature you add constrains you a little in the future (and every language makes this tradeoff early when it's trying to acquire users, but once it's established you need to be more careful). That's why we try to keep the abstraction level high at the expense of a quicker and tighter fit to a particular environment. This delays some things, but we believe it keeps us more flexible to adapt to future changes. It's like having an adaptation budget that you don't want to fully spend on your current environment (I think P/Invoke and properties are such examples of overfitting that pays well in the short term and make you less adaptable in the long term). The complexity budget is another you want to conserve. Add a language feature to make every problem easier, and over time you find yourself not only constrained, but with a pretty complex language that few want to learn.
Green threads experiment proved net negative in terms of benefit but the follow-up work on modernizing the implementation details of async/await itself was very successful:
Issue https://github.com/dotnet/runtime/issues/94620
Technical details https://github.com/dotnet/runtimelab/blob/feature/async2-exp...
The result is such that regardless of p/invoke existence green threads would have been a worse tradeoff.
It also seems that common practices in Java indicate that properties are not a mistake as showcased by popularity of Lombok and dozens of other libraries to generate builders and property-like methods (or, worse, Java developers having to write them by hand). In addition, properties existed in C# since its inception, that's...not a few years.
Not entirely sure about string interpolation but if you are alluding to `var text = $"Time: {DateTime.Now}";`, then it's a non-issue - APIs that care about it in complex contexts like querying a DB or logging can handle it with interpolated string handler API which allows to pass string interpolation expression to methods accepting interpolated string handler types, which can then, for example, generate parametrized query with sanitized inputs, without any friction for the user. Something that Java does not seem to sufficiently appreciate.
Example: https://learn.microsoft.com/en-us/ef/core/querying/sql-queri...
We put a lot of thought into which features we want to add to the Java platform and in what form, and also consider what other languages have done. Sometimes we choose to make different tradeoffs based on what we think are the right tradeoffs for most Java users (a tradeoff that's right for language X may be wrong for language Y [1]), and sometimes we disagree on aesthetics or technical merit. But the choices we've made have worked well for Java. We're well aware of differing opinions, but it seems that we're managing to align with the majority opinions (don't confuse "popular" with "majority"; something like Lombok is quite popular in absolute terms, but is still liked by a minority, i.e. it is less popular than not using it; Kotlin is also quite popular, but it is still more than ten times less popular than Java so does that mean we should follow its decisions?). At the adoption levels enjoyed by JS, Python, and Java, something could be hugely popular in absolute terms yet liked by a minority.
In our primary domain of serious server-side software, no other language has done better (or as well), and we and our users are happy, for the most part, with the choices we've made (except maybe for choices made very early on, but that's true for all languages). The mere fact that sometimes not everyone agrees with our choices (let's be honest, programmers rarely agree on anything) doesn't mean we should change them, especially as languages that go a different way don't seem to be doing as well. Still, different programmers will continue liking different things, and most will continue insisting that their preferences -- however popular -- are somehow "objectively" better with or without bottom-line metrics to support their beliefs.
In general, thinking about a programming language from the perspective of a programmer situated in specific circumstances can be quite different from thinking about a programming language from the perspective of the language maintainer, who needs to take into account different and often conflicting needs of many programmers situated in a variety of different circumstances. The wider the market you're targeting, the more aspects there are to consider and the closer attention needs to be paid to the distribution of programmer preferences.
[1]: E.g. the technical constraints that impact the design and performance of user-mode threads in Rust or C++ are fundamentally different from those that affect Java (re. e.g. the cost of allocating memory, and where pointers are allowed to point). The constraints around async/await in JS -- where a lot of code is already written under the assumption of no intervention -- are also very different from those in Java, where threads have existed from day one.
Ah, that must be why I see FooMethod and FooMethodAsync side-by-side in C# all the time.
Native calls are rare in Java because they're such a pain. If it wasn't so hard to do native calls in Java, it would be common even for non-experts to make use of non-Java libraries.
Note that interaction with native libraries often requires a more careful management of native memory that, though much easier now with FFM, is still significantly trickier (and more dangerous in terms of introducing undefined behaviour) than interacting with Java code regardless of how that interaction is declared in code. In Java, as in Python, interaction with native code -- in the vast majority of cases -- is best encapsulated inside a Java library and not often directly exposed to application programmers.
That's JNI, which really was truly terrible. Java 22 introducing FFM is finally an admission that JNI was crap and a dead end.
There's a reason they're calling it the 'FFM' API and not JNI v2. The API devs were correct in rethinking the approach to native interop.
This just proves my point; being crappy ON PURPOSE is why it's a dead end; it's very difficult to improve something that's been deliberately designed badly.
Besides that, no Java dev in their right mind is going to continue to use JNI once they upgrade to Java 22 and realise FFM exists.
Secondly, our libraries also land on Android applications, and lets see if FFM ever lands on ART.