Hacker News Reader

Comment by javagram | Hacker News Reader

Title could be improved, as I was wondering if the command line tool npm had itself been rewritten.

It’s actually one of npm’s web services that was rewritten.

FTA: “Java was excluded from consideration because of the requirement of deploying the JVM and associated libraries along with any program to their production servers. This was an amount of operational complexity and resource overhead that was as undesirable as the unsafety of C or C++.”

I find this comparison a bit odd. Even if not using containers, the JVM isn’t hard to deploy as distro package managers include it. Unless a team is managing servers manually rather than an automated tool this doesn’t seem that complex. Am I missing something here?

View on HN

Well, speaking from experience with a JBoss application layer in a recent software project I worked on:

New java versions do break existing libraries or apps, and need to be tested thoroughly. When the company hasn't budgeted for that expense, it becomes difficult to update.

Often an architect or software team will insist on using the Oracle JVM rather than the included openjvm. That adds extra steps to download, store as an artifact, distribute, verify, etc etc.

The people who wrote the build pipeline have since been laid off, and an updated set of libraries requires a lot of work to trace back through poorly documented and understood code to make changes.

(Not to disagree with you here, it's more that I'm trying to illustrate how, with poor foresight, Java dependencies can get difficult to manage)

> New java versions do break existing libraries or apps

Have you used the Rust compiler? My experience tells me that any Github Rust project not updated in the last two years doesn't work with my Rust compiler. Whereas Java apps written a decade ago still compile and run on OpenJDK/Oracle often with zero or near zero changes.

> Often an architect or software team will insist on using the Oracle JVM rather than the included openjvm.

Yes, you could choose this. But if you have the ability to choose an entirely different language, I guarantee you have to ability to choose OpenJDK if that's what you really want.

> build pipeline , an updated set of libraries

Doesn't Rust have libraries?

In fact, if anything, Java libraries have a longer shelf life and greater compatibility because they don't require integration of build systems. Java libraries can (and do) use 5-year-old compilers, postprocess bytecode for perf (e.g. Hikari) or size (e.g. Proguard), and even use entire other languages like Scala and Kotlin. But the only thing you as the library user need is the JVM bytecode, which is still high-level enough to maintain runtime interoperability but sufficiently low-level enough to achieve strong build-time interoperability.

---

Much of the "organizational overhead" seems to come down to "I don't like managing the runtime". And that's not wrong; static binaries are nice.

But how much harder is it really to manage/isolate/version your JVM runtime for your server deployment, than manage/isolate/version your Rust compiler for your CI pipeline?

> My experience tells me that any Github Rust project not updated in the last two years doesn't work with my Rust compiler.

Please file bugs against the Rust compiler then, because that would be a serious violation of the compatibility rules!

It's definitely a thing. It's easier to see with applications that use a lock file rather than libraries that always try to fetch the latest dependencies. That's because there have been incompatible changes made, but within the "allowed breakage" policy. For example, try checking out 0.2.0 or 0.3.0 of ripgrep and building it with Rust 1.33. Both fail for different reasons. The former fails because rustc-serialize 0.3.19 was using an irrefutable pattern in a method without a body, and this was prohibited: https://github.com/rust-lang/rust/pull/37378 However, if you run `cargo update` first, then the build succeeds.

The 0.3.0 checkout fails because of a change in Cargo. Specifically, I think it used to define OUT_DIR while compiling the build script, but this was not intentional. It was only supposed to be set when running the build script. ripgrep 0.3.0 got caught here in the time after when the bug was introduced but before it was fixed. See: https://github.com/rust-lang/cargo/issues/3368 In this case, cargo update doesn't solve this. You cannot build ripgrep 0.3.0 with any recent version of the compiler. You need to go back to a version of the Rust distribution which ships a Cargo version that sets OUT_DIR when compiling build scripts. (Or, more likely, patch build.rs since it's a trivial fix.)

Of course, both of these things are fine on their own. But strictly speaking, from a user experience point of view, code does break from time to time. With that said, ripgrep 0.4.0 and newer all compile on Rust 1.33, and ripgrep 0.4.0 was released over two years ago. So, strictly speaking, it does show the GP is wrong in at least some circumstances. :-)

This has less something to do with Rust itself but the ecosystem, but once I updated my system and obtained a newer version of openssl on my OS.

Sadly, some of my projects still used an older version of the openssl crate somewhere inside their crate tree.

The openssl-sys crate author chose to check that the native library version it was compiled against was below a certain version, so it broke. All requests by users to fix the legacy bug were deflected with the mantra "go update the openssl crate the fixed version is out since a year already"...

https://github.com/sfackler/rust-openssl/issues/994

https://github.com/sfackler/rust-openssl/pull/1001

Unfortunately, I don't know enough to rewind history and do enough archaeology to find out whose "fault" it is.

But as a possible example, typemap [1] which is a library featured on Awesome Rust [2] ("curated list of Rust code and resources").

Project uses cargo. Has a lock file with two deps. Last code commit was May 2017. And I can't get it to compile.

[1] https://github.com/reem/rust-typemap

[2] https://github.com/rust-unofficial/awesome-rust

There's no lock file in that project.

But moreover, you stumbled upon one of the worst cases, a soundness problem that has yielded a lot of discussion because of that crate specifically: https://github.com/rust-lang/rust/issues/50781

The issue here is that the typemap crate was found to be relying on a compiler bug. The compiler bug could be exploited to write transmute in safe code: in other words, all the safety guarantees of Rust go out the window unless we break that crate. It's a bad situation, but I don't know what we could have done differently. I think everyone agrees it's not worth sacrificing all the safety properties of Rust to keep typemap building.

What error do you get?

My project stopped building on compilers w/ the new module system. I did file an issue (56247, see also 56317) and was told it won't be fixed. The fix to my dependency was trivial, but it did stop my code from compiling with no change to my Cargo.toml/Cargo.lock, and ultimately it did require me to unpin that crate's version and update it.

My understanding was that by virtue of my project not having the edition 2018 key I would have been isolated from such changes to the language.

> any Github Rust project not updated in the last two years doesn't work with the latest Rust compiler.

I don't think this is correct? Rust introduced an "editions"-based system in 2015, with the very aim of ensuring forward compatibility on a crate-by-crate basis. The aim is definitely that any crate written to be compatible with some "edition of Rust", whether 2015, 2018 or whatever, can be made to compile on a future version of the compiler.

One possibility is that he used something written against the nightly Rust compiler. That code would be very unlikely to compile today.

Maybe, but you have to specifically opt-in to the nightly toolchain in order to compile something that uses it. It's not something that's included by default, or appropriate for use in production scenarios (unless you're willing to deal with the resulting breakage, of course).

Of course it's not suitable for production, but it was still quite common to look at a rust project and find that it used the nightly compiler.

It's been awhile, but if memory serves tons of stuff was using the nightly compiler.

So, yes buyer beware. But when the whole ecosystem does it...

Exactly. I think it's getting better now that some of the cool things people were waiting on have made it into the 2018 edition. Off-hand I know that Remacs switched from the nightlies to 2018.

It's probably just the lack of emote via text, but your reply comes off as confrontational. It's also seemingly replying to me about general Rust issues when I'm just sharing my experience of a recent Java project. I don't have much experience of Rust beyond reading some code every now and then when an interesting blog post pops up here.

I'm also not condoning the decisions made in the project I describe. In fact, those ancient decisions are responsible for a significant amount of stress and work for me right now.

I've absolutely had experience of good JVM based projects that had loose dependencies on libraries and jdk versions such that it ran just fine under the openjvm, and ran just fine when that package was updated.

> I don't have much experience of Rust

Oh, then that makes more sense.

It's always possible improve. But Java is pretty much king when it comes to compatibility and maintainability. (Not so much in other areas, like verbosity, language features, type system, memory overhead, etc.)

I just like that someone at npm would avoid something because it has lots of dependencies and overhead. The irony is strong with this one.

Not so ironic—just because a practice is common among the median engineer in an ecosystem, doesn’t mean it will be common among the most experienced engineers in that ecosystem.

Node gets a bad rap mostly for the fact that it has tons and tons of inexperienced engineers using it (probably as one of their first programming languages.) Same reason PHP got a bad rap back when. You can build solid software in both, by finding and absorbing engineering best-practices from people who have already had hard-fought battles to learn them; but an engineer will have to be burned at least once on building/scaling+SLAing something before they start looking for those. You might say that the Node ecosystem has a lot of programmers that are “engineering virgins”—they’ve never been forced to contend with the real problems of engineering software. But they’d be “engineering virgins” no matter what language they’re using; that’s not an indictment of the language, just a consequence of its popularity and approachability.

Do you have any advice for PHP best practices? I'm starting a job that will probably have me using PHP...

I think majority of the good practices are the same in PHp as in other languages like Java, many PHP issues were caused by creating SQL statements using user input(you can have same issues with beginner devs in most languages), so the best practice is to find a framework that does all the ugly part for you(the logins, session management, routing), there are large and small frameworks , I suggest use a popular one even is not perfect(to avoid the risk of hitting bugs if is not used that much or the risk of it getting abandoned or deprecated),

Don't get me wrong, if you need to do one small thing, like for example I had a desktop app and the user could submit feedback directly from the app, then a single PHP file was enough for this case(no dependencies, no frameworks), you get the submitted data, clean it and put it in the database or submit it to a third-party API that can handle it.

A fair amount of the pitfalls are already gone. Register_globals is off by default now, placeholders for sql queries are the default examples, etc.

You will probably hear that you should use Laravel or similar. I'd argue it's a pretty big hammer, so don't reach for it if you don't need it.

The biggest issue is probably still the breadth and inconsistency of the standard library. Too many ways to do the same thing. Also, the general issues of a dynamically typed language, sprinkled in with things like == vs ===.

I like the books this guy writes: https://github.com/codeguy

https://phptherightway.com/

http://www.phpthewrongway.com/

https://paragonie.com/blog/2017/12/2018-guide-building-secur...

Edit: formatting

I find it funny that the "right way" link has HTTPS whereas the "wrong way" link doesn't.

We're not talking about average JS developers, we're talking about the NPM maintainers.

EDIT: I did miss that. Sorry for the noise.

You missed the joke. Let me quote the GP again:

> I just like that someone at npm would avoid something because it has lots of dependencies and overhead. The irony is strong with this one.

For this to be "ironic", having lots and lots of dependencies would actually have to be a Node.js "best practice." But it's not.

It seems like a best practice to outsiders, for the same reason that setting `register_globals` seemed like a "best practice" in PHP back in PHP3/4. Because it was extremely common, one might assume that it was endorsed as a canonical approach. And so you do it yourself, and write sophomoric tutorials suggesting others do the same, perpetuating the problem.

In reality, the "best practice" followed by experienced software engineers (for Node.js or any other language) is to carefully consider your dependencies, and to try to avoid dependencies that cause an explosion of sub-dependencies. The NPM maintainers are experienced engineers, and so they follow this best practice.

There is no irony here. It is not "the Node.js way" to use tons and tons of dependencies, such that the NPM maintainers are going against the grain somehow. It's just the way of programmers inexperienced in engineering to not care about dependency proliferation; and then, further, to make a large set of their own tiny libraries (with already-exploded deps trees) because they aren't yet at a stage of programming expertise where they see that code as trivial to bang out whenever they need it (see: the left-pad package) that then further encourages others to depend on them. It's the "copying and pasting a solution together from bad code in five StackOverflow posts" phase of one's programming career, except instead of having to copy-and-paste, all the snippets are symbolically linked together into a big tree and you refer to them by name. (Again, that's not an indictment of Node.js—there's nothing you can do to stop a bunch of inexperienced engineers from doing this to your package ecosystem as well, if they happen to be drawn into your community.)

I'm not convinced. I see stuff like this fairly often: https://news.ycombinator.com/item?id=19290801 (Basically how it might be common to have 1000+ dependent files in a JS app)

From people that seem plenty intelligent and experienced.

That thread is about a module system. 1000 files != 1000 third-party dependencies.

On the other hand, NPM itself kind of paved the way for ecosystems with tons of dependencies, managed and versioned with a single tool. Installing system software like the JVM is more old school.

Npm doesn't seem to break any substantial new ground to me. Things like apt, CPAN, etc, have been around longer.

You don't have to install a JVM, you can bundle it with a distribution as well.

If I have a choice between dealing with it and not I’ll choose not every time. It’s an annoying dependency that got even more annoying when Oracle decided to be a pain in the ass. Just because something is “easy” doesn’t mean it’s easier than the alternative.

I just unpack JDK, I have no idea what so complex about it. It was complex on Windows because I used VM to install Java and then copy folder, but they made it easier with 11.

One example might be that if you unpacked the Oracle JDK, you owe Oracle a non-trivial amount of money now.

Only if deployed in production, and there's nothing hard to understand it.

Commercial development or test use isn't free. It's only free for personal, noncommercial use. So I'd argue it isn't so easy to understand.

How it isn't? You just explained it quite well.

Not to mention that the Web is now full of discussions around this, with official posts from Oracle, Red-Hat, Amazon, IBM, Azul, Microsoft explaining how to go forward.

Figure out what you would pay if you used it in an elastic ECS setup. Using only Oracle documentation as a guide :)

Why should I? There is OpenJDK for that.

But assuming that Oracle support is actually desired, here is what you are asking for.

https://www.oracle.com/technetwork/java/java-se-support-road...

https://www.oracle.com/java/java-se-subscription.html

https://shop.oracle.com/apex/f?p=DSTORE:2:::NO:RIR,RP,2:PROD...

Old Java dog here.

But lets just forget about the efforts from AdoptaJDK, Red-Hat, Amazon, Azul and bash Oracle, it is more fun.

"here is what you are asking for"

None of that helps you with what the charge is when you're elastic, or using multiples of hyperthreads that don't add up to a integer number of cores. I agree that OpenJDK is a better idea.

If you want to be fully sure and insist in using Oracle JDK, then do like any enterprise shop and call the sales team to get their written word, what is so hard about it?

Why would use use only Oracle documentation? Call sales guys and ask them for a license. They would be happy to sell you something and there's even some chance for some kind of discount.

To me it reads as "We didn't want to use Java".

Yeah. I can see the Java issue as a minor annoyance, but to put it on the same level as the lack of safety with C/C++ seems hyperbolic to me.

It reads like an innocent misperception due to inexperience with Java.

Very few people would choose to install libraries (JARs) used by their code via their OS package manager for instance.

JVM tuning (especially the GC and memory allocation scaling) can be a huge PITA.

But having options is always good, isn't the same with compiled languages? you keep the defaults options but if you want extra performance you try different compiler flags or different compiler if the language has more then one.

Honestly I’d rather have good defaults than lots of options. If one tool meets requirements out of the box, then it’s preferable to a tool that requires a lot of tooling.

But if you do not have options then what is a "good default" , you have a default option if you have more then an option, so you prefer software with no options?

Can you give examples of languages(compilers or VMs with bad defaults in your opinion?

Java by default has a hard memory limit which when hit causes services to hang in a semi-responsive state rather than exiting cleanly, requiring every user to hand-tune limits and carefully setup health checks and monitoring. Two decades in, they added a non-standard option to instead exit.

If they had instead made the default to act like almost everything else it would have worked with standard process monitors and limits with no effort required and a substantial fraction of the downtime I’ve seen for Java applications would never have happened.

Isn't this default better for average Joe that runs a Java desktop app, and developers that deploy apps should know how to change this defaults?

How is it better for something to stop working but not exit? It’s exceedingly uncommon for anyone to have correct handling for OOM exceptions so the most likely effect is that stuff partially works - servers accept requests but never respond, apps have menus/buttons which don’t work, etc.

Similarly, if developers who deployed apps knew enough to avoid this, we’d know by now because it wouldn’t happen so frequently. It does highlight who failed to do real health-checks (e.g. years ago most Hadoop services needed huge amounts of RAM to start without crashing but they’d be listed as okay) but it’s the kind of thing sysadmins have been cleaning up for decades.

You are probably right for this case, my initial comment was related to performance tuning configuration, the comment I replied sounded like JVM should have had X set to my preferred value or "I prefer software that has no options to confuse me with aka GNOME mentality"

You can have options and have good default values for those options. Good default values are those that are reasonable for most people most of the time such that they don't need to be (or hire) domain experts to properly configure their tool.

My opinion of a bad default would include a language toolchain which defaults to dynamic linking and requires one to opt into static linking (Java, Python, JS, etc, etc, etc). Worse than that is an anorexic toolchain that has no defaults whatsoever and requires you to pass every little detail directly to the compiler--bonus points if your language has a massive ecosystem of competing tools which are meant to manage these sorts of details for you but utterly and uniformly fail to do so (looking at you, C/C++).

So the JVM run-time is installed and used by average people, so the defaults should be set for this people that install Java to run a desktop app IMO, the developers that want extra performance should read the manual and configure things.

I’ve never heard of a runtime that forces a dichotomy between end-user- and developer-friendliness (putting aside for the moment that end users are famously annoyed by the Java runtime). Rust, Go, etc don’t have runtimes which force a choice between end-users and developers...

JVM can be used for Desktop, server and in the past applets , it would be impossible to find a configuration that is optimal for all applications, so competent developers would tweak the defaults or use different deployment methods like AOT compilation or bundle your own JVM , is your problem that JVM is used in so many different places that different algorithms, runtimes and optimization were created for it ? Your examples of Go and Rust are of languages that are so far not been used in such diverse places, there are no good alternative compilers but something like Python has such a diversity of runtimes.

How much is that actually necessary these days? I remember spending ages tweaking flags in the 1.4 - 1.6 years. But there have increasingly been sensible defaults with broad applicability. Now that G1 is both default and usable, even more so.

As i've moved onto 11, i've slashed our apps' JAVA_OPTS down to almost nothing - max heap size, some GC logging flags, that's it.

At scale? Very necessary. The default GC as of Java 8 (the last version I used in production) suffered significant performance issues above 40 GB of working set, and required careful tuning thereafter.

I completely agree. The JVM is fine until it isn't, and at that point it becomes very frustrating to deal with - even when using less than 16GB RAM.

The default GC as of OpenJDK 8.

The default GC from IBM, Azul, and others wasn't the same one.

In fact Azul has made a business of selling JVMs with GC that can handle hundreds of data GBs.

Modern OpenJDK versions also have such GCs now.

Hardly any different that having to use VTune or perf to optimise C and C++, which require the added steps of recompiling with different flags and redemploying.

I wouldn’t recommend C or C++, but in their defense, you don’t need to worry about tuning as soon as you do on the JVM. Many apps won’t need to bother with it at all (which is good because C/C++ programmers have all manner of other things to worry about that aren’t a concern for modern languages).

There's also a notion of failure mode for improperly tuned apps. This is certainly a personal preference, but I prefer true failures over constraint failures. I'd rather OOM in golang than run out of threads in my Java execution context while having plenty of free memory. At least when starting out.

Me neither, I do like Rust and see its adoption at NPM as good PR for the language, however I also think that the way the decision process went wasn't free of bias.

I'm not a fan of relying on distro package managers for installation of runtime dependencies on servers. Too many opportunites for variables to creep in if the version isn't locked, and then having to make sure all the package manager dependencies and config themselves. Even if you automate you're still at the mercy of the repo to have the version you need etc. and often times you need to customize the install for a highly-available and/or virtualized environment. Bad times all around.

What's the alternative? Not installing the dependencies so the app doesn't work?

The alternative is vendoring the dependencies. That includes the JDK or Node runtime. With “modern” deployments you’ll see this with container based packaging that includes the runtimes either explicitly or via system packaged (in the container). The classic approach is copy the runtimes into your final application tarball.

Either way the runtime is baked into the app and gets deployed and tested with it as a core component. Runtime upgrades then become vanilla deployments.

Lots of options: containerization, downloading prebuilt binaries, downloading and building from source, or downloading, building, then packaging the artifacts and leaving somewhere centralized for deploy.

Using the language package manager - npm, cargo, etc.

Also with GraalVM you may get away with building a native image which doesn't require a JVM at all.

That is not the only option, though.

Enterprises that care about AOT Java have been using them almost since Java exists.

A really cool demo that shows you the power of GraalVM

https://youtu.be/9BQiDmvOnZw

Deploying jvm app/service is now even more complex than figuring out which version of node to use to run a service. Is it supposed to be oracle java? Openjdk? Adoptopenjdk or one of a half dozen more? Which version of it? Anything to tweak in gc or startup settings for it/the version? Do we need to regression test the service on minor jdk upgrade? Is that jvm compatible with some os version we are running that has some security patches and other settings?

> Deploying jvm app/service is now even more complex than figuring out which version of node to use to run a service. > Is it supposed to be oracle java? Openjdk? Adoptopenjdk or one of a half dozen more? Which version of it?

All of these must have been tested against the Java TCK. So, the answer is "whichever you/your company decided is okay" for the first part and for the second part "newest update of the Java version your software runs on". Doesn't sound very complex to me.

It is pretty clear to any regular Java developer that bothers to keep up with the news.

Regression tests are required for any toolchain.

You are making the parents point for them.

Yeah, "npmjs.com" would have been more enlightening.

The issue with the JVM is keeping it updated, in the context of a constant stream of security issues that need patching.

Reply on news.ycombinator.com