Hacker News Reader

Comment by mnw21cam | Hacker News Reader

I write software in Java. I can't stand python - too wishy-washy.

But my main complaint about Python is trying to use someone else's software. Then it degenerates into a maze of twisty little passages of trying to install the correct set of dependencies, not knowing where those files are installed (which makes it difficult on a cluster, as you may need to do the install on every node separately). Then you try using pip and it fails because some element of what it is trying to install isn't compatible with your operating system for reason 1. So you try anaconda instead, and that fails in a different way, like it runs a C compiler on some file (!?) which fails because $REASON. I have numerous examples of Python software where I have followed the (detailed, precise) installation instructions, and it doesn't work.

Whereas with Java, you just dump the necessary jars into a directory, include them in the classpath, and it works.

Don't get me started on R.

View on HN

I came to say say that's not a language problem, that's a packaging problem. But that would be misleading.

The reason packaging is still a problem in Python are linked to the language:

- to get perfs, you need compiled extensions, which java doesn't need. So a Jar can be fast and portable, while in Python it's an "or" proposition. Wheels help, but they still are OS dependent, so you need a build per OS.

- PYZ arrived very late in Python, while WAR were a very early thing in Java. Few people even know you can distribute a whole Python program in a single zip file with dependencies using something like shiv (shiv.readthedocs.io). So few people do it, and it's a shame. We do it for 0bin though: https://github.com/Tygs/0bin/releases. And it doesn't solve the wheel problem, although nuitka does but it a higher prices to pay.

- packaging tooling have been a mess for 2 decades. Setuptools/distutils/setuptools2, easy_install/pip, and setup.cfg/setup.py/pyproject.toml. We failed as a community on this one. It's getting better, but it's a slow process.

- the Python community is at least half composed of amateurs. It's a good thing for the dynamism of the community, but it also means the software are not very polished. Hard packaging + amateurs = packages that are releases when they reach "just good enough" stage, not one inch more. It also means you'll get things like leaking stack traces, config files in src and the likes.

So yeah, compared to Java, Python packaging and distribution sucks.

But compared to rust all packaging and distribution for all languages do, so there is that.

It certainly doesn't help that the software I'm interested in is written for science, so there is a tendency to make it work just well enough on the author's machine to allow a paper to be published, and then abandon it.

There is a clear training problem in science. But I believe part of it is that the knowledge on how to write good software is stuck in companies. You can't learn it in school either. I keep hearong about companies that have to make their new hires unlearn what they learned in class... So where would scientists learn good practices? (in my experience books arent the answer either)

I don't think it is primarily a problem in training. It is a problem in incentive. For the most part, scientific software is written to fulfil a goal, and usually that goal is in order to publish a paper. Paper publication is what scientists are judged on, not the quality of the software they write. The software only needs to work well enough to get the paper published. Once the paper is published, all incentive to maintain or improve the software disappears, especially if the message of the paper is primarily the results obtained using the software, because then all of the citations will be referring to those results, not the software.

There are two main exceptions to this rule. One is when an actual software engineer ends up working as a scientist and having to write papers (like me). Then, the paper is more likely to be about the software itself, and the citations of the paper are from people who have also used the same software. This creates an incentive to make the software high quality, in order to get those citations. The other exception is when an institute decides that the world needs a decent piece of software to fulfil a vital role, and actually funds its development.

Oh, I should probably mention - I just helped out with a programming course for undergraduate medical science students. Most of the time I was tearing my hair out about the terrible programming practices being taught by the module leader, who admittedly had absolutely no computer science qualifications whatsoever (and in fact didn't know the difference between a forward slash and a backslash).

I think a solution to this is to recognise software quality as a common benefit, and divert funding towards it, allowing real software engineers to have an actual meaningful career in university. Universities are usually all over common benefit stuff, so you would have thought this would be easy enough. The Research Software Engineer programme in the UK is a start.

Is shiv better than pyinstaller? When turned my script into a binary with this, it took 10 seconds extra on every execution just to start running the script (Eg to just print the help message). I found that to be unbearable since this was a command line script that was invoked anew for every execution

If you're not already using the --noupx flag for pyinstaller, you can probably cut that startup latency at least in half. With UPX enabled, the executable has to be decompressed every time it is called. The space saving tends to be minimal and not worth it, definitely not for a CLI where startup latency much more directly impacts the user experience.

They don't serve the same purpose.

Pyinstaller builds bundle the python binary with the project. It works without a Python installed on the system, is not portable and the resulting binary is fat. It also can fail in numerous way and be slow, but end users will find it the easier to use when they work.

Pyz files just contains python code. Shiv produce them by bundling your code and the dependencies, but you still need a Python installed on the system to run that. You pay the cost of zipping only on the first run though, after that, it's unzipped. And their are portable if you use no binary wheel. It's light and easy to get right.

In that sense, they look more like War files in Java.

Pyinstaller, like cx_freeze and nuikta are for end users.

Pyz are for devs.

>the Python community is at least half composed of amateurs.

or, even worse, scientists.

And most recently, tons of self styled "Data scientists". This has, IMO, diluted the rabid rhetoric of the 90's/oughties python community which is good, but replaced it with people who don't know and don't care what good code even looks like, so it can be difficult to work among them.

I feel sometimes that they pride in writing shitty, unmaintainable code; that they do it on purpose. One of the ways they enjoy life is to disregard all "best practices". And I'm not talking about some high-level OOP principles, but the basics - like keeping consistent code style, or even writing some tests.

The previous crowd with love for monkeypatching everything and hate for abstraction other than dictionaries wasn't too pleasant, too.

lol I was just sitting here thinking I miss my Python job and not having trouble with IntelliJ detecting changes in dependencies and how frustrating it is to invalidate caches and reindex the whole project multiple times before I can run tests for the dependency change. I regularly miss the testing libraries we used, too -- it was so absurdly clean and easy to build robust mocks in our DJango stack (and I set up that tooling, largely on my own iniatiative, based on my team's feedback, so it's not like I'm just like wishing other people had already done the work). Now I work with a bunch of Java devs who use to work for Amazon and somehow use that as an excuse for not caring about tests, our ci-cd tools, and all these other things I'd gotten use to thinking of as standard in my past life as a Python dev.

Code style! Hah. First day on the job I asked about that and was told nobody cared and everybody had their own preferences.

... I'm very bitter today. I'm working on it.

This sounds more like a people problem than a language problem to me.

I’m in this forks and I have a hard time disagreeing.

Everything is jupyter notebooks, to the point where example documentation is delivered as notebooks and services run notebooks as it’s apparently easier to do that than to get people to write their code properly. “Oh but data exploration” they say, which is fine, but you shouldn’t be doing data exploration in your prod ML pipeline anyway, and you’re writing code, it’s really not that hard to follow some even rudimentary practices.

So... Java is bad, but let's ignore Python's problems.

I'm forced to use Python at work, because data scientists demand it, and I can rant about your "good for data" language for at least 24 hours straight.

Python isn't great at anything specifically, but it's crap in a few specific places. No multithreading? Fine! But then Python core libraries are crap at IPC, when multiprocess is the preferred way of parallel computing!

> So... Java is bad, but let's ignore Python's problems.

I don't know where you read either of your propositions in any of my comment.

And I quote "But it's way better for high level stuff, data mangling, scripting, gluing, and for most web dev stuff."

You've listed things that rely on full stack of libraries and deployment options.(except scripting) As someone who has the pleasure to work with Python - Python is horrible at a lot of these things, in spite of the beauty of the language itself.

> Python isn't great at anything specifically

Python is the best language in the world for small tools under 500 lines to solve glue and data munging problems.

Python is too explicit to be the greatest, but it's popular nontheless.

Java is so much worse on this front.

> with Java, you just dump the necessary jars into a directory, include them in the classpath, and it works.

Ant, maven, or gradle, and then some jars, somewhere? What's the classpath and whats $JAVA_HOME for this project?

Oh and now I have to make it work in the IDE too, click through a billion menus and dialogs, sync the build system, sync the dependencies, configure runtimes and classpaths, try to rebuild for the 10th time, pray.

Nobody has started a project with Ant in the last decade. Of all the Java projects on Github Ant is probably not even 1%.

> Oh and now I have to make it work in the IDE too, click through a billion menus and dialogs,

You click "Project from existing sources" in IntellIJ and it will recognize both Maven and Gradle projects and just work?

Sync the dependencies? configure runtimes and classpaths? What? With Maven and Gradle? Your talking Ant here.

I only had to go a few comments to see someone (the person you're responding to) think Java is the same as it was 15 years ago.

I'm talking about a maven project I have to use sometimes at work. Let me walk you through all the shit I had to do:

  - Install and set the right java version system-wide
  - Copy some config files for the project system-wide (granted, this is not java specific, but still)
  - Click in some IDE menu to enable some build settings
  - Click in some IDE menu to install some "Lombok" plugin
  - Click in some IDE menu to select the right jdk
  - Click in some IDE menu to edit the run configuration:
    - select main class
    - paste program args
    - paste env vars
    - select right “Use classpath of module:” option
  - Click in some IDE menu to fiddle with "Spring Loader path": modules, dependencies, reorder the dependencies (!)
  - Click in some IDE menu to configure getting the actual logs
  - Click maven-sync, rebuild-project multiple times, restart IDE until it suddenly works

The first guy who got it to work had to write a guide with screenshots!

I know you will say, this is all fault of whoever made this crap project, or my fault for not fixing it. However this was my experience, and it has been similar every time in my life that I had to touch some java project (although this last one is the worst).

IDE's can be fiddly, and anyway I don't think it's good to depend on an IDE for building.

The project could have been set up instead with the "Maven wrapper"(1) which provides "mvn.cmd/sh" scripts at the top level of the project to fetch and run the proper Maven. So after cloning the project, all you have to do is set a proper JAVA_HOME if it's not already set, and then do "./mvn clean package" at the top of the project to do a complete build. The IDE's will then set themselves up properly if you import the project as a Maven project, and will sync with changes to the Maven pom.xml whenever it changes (Intellij does at least). That's really hard to beat.

(1) https://github.com/takari/maven-wrapper (Being integrated into Maven itself soon).

I can (and often do) build from CLI, that is much smoother. I tried using vim on java, but honestly you need an IDE for all the constant code generation and incessant fixups by mouseclicks that are needed. Vim+coc is more than enough in the other languages that I use.

You're living in a parallel universe from the rest of us. IDE isn't necessary (and positively odd) for building Java.

You do need to be able to build from IDE when you set breakpoints to debug.

Your IDE should be able to connect to any JVM that has been launched with the proper VM arguments.

We occasionally do remote debugging at work. It’s just a matter of copy/pasting the correct JVM incantation into the IDE.

I feel the parent comment was about editing, not building.

He is. VsCode (or any editor) + the Eclipse Language Server is more than enough. His complaints about debugging is because he doesn't know about attach debugging.

This is one of the main reasons why I won't work with Java. Everything is much more complex and unnecessarily arcane compared to other languages.

> I know you will say, this is all fault of whoever made this crap project

It is. There's really no excuse for that. If it's been similar every time then you've worked on crap every time.

Building a java project should be (and is on projects I work on) as easy as installing a jvm and running either maven or gradle to build it. End of story.

"Install some lombok plugin" is only necessary if you want the IDE to understand lombok, IntelliJ will just pop up a box saying "Want this?" and you click "Yes". I don't see what's so onerous about that.

The rest you describe is really down to poor developers.

No offense, but regardless of how right you might be, you come off as a bit of a condescending asshole. Like "Oh, you don't know that there's a more efficient way to do things? There can't possibly be any explanation other than you're literal human garbage and presumably have brain damage!". Come on, get off your high horse.

I'm really not on my high horse here, if you have a team working on a java project that's that hard to build, they're doing it wrong. Doesn't even matter what the language is really.

So "no offense", but shove it.

There's a difference between - "anything can be turned into an non-compiling pile of crap" and "this ecosystem comes as a pile of crap".

I think you work at the same company as me!

I can find maybe only one or two things from this list related to Java, both not reflecting current state of affairs. You do not make the need to have a run configuration a language or platform fault, do you?

You say this as if python could do any better

I've interacted to a ton of Java projects. And with a ton of various other commercial and Open Source projects.

In my experience (sample size of hundreds of projects), Maven projects are <<by far>>, on average, easier to build and manage than most other languages. The repo structure is standardized, the build cycle is standardized, the built packages are always in the same place, etc.

The Java installation is a 1-time think. It's also a quick google away. I see this complaint a lot, I never understand it. It's plain and simple bikeshedding: someone complaining about a 1-time task that takes 5 minutes in total.

The rest of the stuff is IDE specific and I have no idea why you'd have to configure "getting the actual logs"?

Well Java is arguably one of the most IDE-reliant mainstream languages. It's completely unusable and unergonomic without the myriad of constant code generation and fixing as you write.

Treating the IDE like a separate problem is not realistic with java.

With e.g. Rust, I can use vim, vscode, whatever floats your boat.

> no idea why you'd have to configure "getting the actual logs"?

Some genius java logging library, I can only guess.

You can write java with vim, without any trouble. It’s just that Java has the very best IDEs available and I would be stupid to not use them.

> Well Java is arguably one of the most IDE-reliant mainstream languages. It's completely unusable and unergonomic without the myriad of constant code generation and fixing as you write.

True, but with IntelliJ 99% of what you wrote is auto-configured.

Eclipse needs a bit of hand-holding, but almost everything you described is due to project configuration.

> Some genius java logging library, I can only guess.

That's your problem, you're assuming that Java is dumb and its libraries are dumb. They're not. I'm reasonably sure that <<all>> mainstream Java logging libraries log to console by default.

The previous engineer must have been a marvel of a developer to manage to screw even that up.

What I'd say is: Java, especially older enterprise code, had a ton of over-engineered patterns. These are slowly fading away, except for Spring, and even Spring's simpler these days. Java also tends to have some low quality and a ton of average but unmotivated working devs on Java projects. So yeah, there's that ecosystem problem.

But for example, if you'd start a new project and you approach Java with an open mind, it's one of the most powerful and flexible techs available. And it's much more modern than the stereotypes imply.

> with IntelliJ 99% of what you described is auto-configured

It is not, that's what I'm saying. And I don't understand why, if IntelliJ is the de-facto IDE. I'm sure that a fresh project is a one-click import, but an average project that isn't fresh anymore, always have similar IDE-setup-magic problems. At least in my experience over the years.

I code in Java, Scala and Python using an IDE just because I like to not remember the documentation to the code.

I know plenty of people who don't use a formal IDE. This "I don't use an IDE" is a stupid flex.

Me too, that's why I have vim setup with language-servers. I get inline type info, code completion and so on.

It's not a flex, simply a matter of preference: I find using the mouse too irritating, others prefer it because it's simpler.

But that's not my gripe, my gripe is that setting up IntelliJ, the one defacto standard IDE to actually work is some voodoo ritual of a dozen steps and restarts and so on.

In my experience even Eclipse does a pretty good job of importing Maven projects without needing to tweak things.

Surprisingly, or maybe not, slf4j requires an implementation to be provided at runtime to log anything.

IMO it's the other way around. Java has some of the best IDE's around so everyone uses them. You can code in notepad and compile with maven but why would you?

With Rust you code in VIM because there's nothing better. Improving Rust IDE's has been one of the teams big focuses for years. Everyone codes in random text editors because there's no amazing IDE.

With rust yoy don't need to constantly generate code. Vim is just perfect, if you have rust-analyze you get inline errors, code completion, jump to definition, autoimport, autoformat...

You don't need to constantly "extract method", move to class, generate all these mountains of classes by rightclick. This says a lot about the language.

But the main point is that with rust it actually just works: there is one way to set global toolchain and per project/dir. You're not required to also click through a million menus and dialogs and copypaste stuff and sync and rebuild and restart to get the editor barely working.

> With rust you don't need to constantly generate code.

In Rust, macros generate code on build. In Java, Annotations Processors do. I don't see a big difference between these approaches, besides Java's mechanism being older and crustier. Rust macros generate tons of code.

> if you have rust-analyze you get inline errors, code completion, jump to definition, autoimport, autoformat...

Last time I checked, Rust analyzer was in alpha/beta. Java has all this and it's battle-tested. Including language server for VScode and other lightweight editors, and several options for build plugins that auto format.

> You don't need to constantly "extract method", move to class, generate all these mountains of classes by rightclick.

Lombok, or one of the popular Builder libraries, gets rid of all this boilerplate. I don't generate anything using IDE. In my experience, most Java user don't. There's many mature libraries for geting around class ceremony.

> there is one way to set global toolchain and per project/dir. You're not required to also click through a million menus and dialogs and copypaste stuff and sync and rebuild and restart to get the editor barely working.

You are right. Rusts build system is simpler and better integrated. But Java has amazing build tools, maybe the best out there. A properly configured maven project will compile, test, and run with one command. And it will import into your IDE with a simple "import project, yes"

I have worked on a ton of Java projects over the years and never had to deal with most of what your describe.

100% lombok can only be developed on eclipse. They have an open issue to make it compile on intellij...

I don't know if something changed, but for sure 2 years ago I used Lombok with IntelliJ.

I don't mean lombok as a library, which works. I mean developing lombok itself

Ah, sorry, haven't done that. It sucks if it's Eclipse only :-(

However I'd expect Lombok users to outnumber Lombok devs 100k to 1 :-)

Specially since it can only done in Eclipse xd

> The Java installation is a 1-time think. It's also a quick google away. I see this complaint a lot, I never understand it. It's plain and simple bikeshedding: someone complaining about a 1-time task that takes 5 minutes in total.

I frequently find that the particular constellation of build tooling and libraries particular projects have requires a particular version of the jvm and all the environment variables set up correctly. In some other languages there is widely used tooling to manage the install and use of the appropriate platform version for the project.

If you're working with a variety of projects, fiddling with the installed jvm and java_home and friends is not a one time thing.

> I know you will say, this is all fault of whoever made this crap project, or my fault for not fixing it. However this was my experience, and it has been similar every time in my life that I had to touch some java project

Yeah that’s a crap project. My experience was always that mvn install / ./gradlew run will just work. On all three OSs. And if I happen upon a decade old random jar file, and run java -jar something.jar, it will just work in the great majority of time. I think no other platform has it this smooth.

If you're using IntelliJ from JetBrains, there should be an option to store the run profiles for the project in the version control system.

Even if your project is so messy, then you should be able to just to version a working setup (maybe needing to figure out how to make the correct JDK version be used across different machines) and after that it'd become a question of just having to check out the project from the version control system and click a green button in IntelliJ.

This most likely wouldn't solve all of your issues, but at least would help make onboarding slightly less painful and slow.

You left out Gradle installed/downloaded options, Gradle version, Gradle JVM version, Gradle plugin auto-upgrades and the unholy mess this lot can create.

This is why we virtually all use the ./gradlew script that manage all of that by itself on every platform... No global installation of gradle should ever be done.

Isn't this rather overblown ? The first 5 steps appear to be a ONE-time framework and system setup for your IDE - and this assumes you have multiple JDK's. You certainly don't need to repeat this for every project.

The run configuration can be automatically created by Intellij by attempting to execute a runnable class. You only need to set program args and env vars if needed - and that would be true for ANY language or program.

The spring loader path appears to be specific for this project.

The run logs are independent of Java. Thats just getting program logs.

There is no need to maven-sync and rebuild-project multiple times, unless you are explicitly refactoring the project module/build structure. Once you have imported a maven project, you can simply click build.

Guess I'm nobody. I created a simple Swing (desktop) app in Java a couple of weeks ago. The process was something like:

Create new Java project in IDE Add no dependencies Use the built in IDE tooling to create a runnable JAR and tick the box to export the Ant build to a build.xml file.

Why would I want to complicate things by adding Maven/Gradle to the mix?

I will give you the secret trick of senior developers: don't be the one who manages the build process, otherwise leave the project asap.

And I will also give you the secret trick of superninja senior developpers: be the one who manages the build process, so you will never EVER be fired.

This method is useless for most projects, so yes, you are nobody.

And what if you want/need to add dependencies? A lot of them.

Sure _then_ you might opt to use Maven or Gradle.

My point is there are a wide variety of application types out there, not everything is a web application, not everything needs to use a complex build and dependency management system.

Yeah, but if you do know Maven, it scales down to small projects. The configuration for a simple project is trivial.

So just use Maven and save anyone coming after you a bunch of trouble when they have to extend your project ;-)

The complexity of maven is well understood and battle tested. The complexity of a custom build is not. In my opinion it is of a far more concerning sort of complexity.

It's an Ant build file - nothing complex at all. If you no or few dependencies and are comfortable managing them manually, Ant is fine.

Non-networked applications don't tend to need to be so paranoid about keeping up to date with the latest versions of dependencies.

We're probably optimizing for different things. I'm normally optimizing for easy onboarding of someone new. If this is a personal project with few dependencies then ant is probably good enough.

Well, this could be true for some legacy projects with old tooling, but this is not how things work today. You install the IDE, checkout the code from Git and let IDE get a proper version of JDK for you and fetch dependencies from Maven central to local repo. You really should not copy jars anywhere or set environment variables.

> Oh and now I have to make it work in the IDE too, click through a billion menus and dialogs, sync the build system, sync the dependencies, configure runtimes and classpaths, try to rebuild for the 10th time, pray.

That is not my experience at all. I dont have to rebuild work project many times. Creating new scratch project for playing in is easy and fast. Dependencies are managed by maven and I rarely have to deal with that beyond added dependency tag.

Huh, as a consultant having touched many java code bases, they are all basically the same. Import the maven pom into intellij and run the main class. The folder structure, the code conventions etc are basically the same at every client and every project I've ever seen in java.

All needed is having the correct version of Java, and maybe give maven a password so it can access private libraries. It's dead simple.

And more importantly: it's the same everywhere.

Yeah, people under-estimate badly how useful the Maven conventions are, because they've never seen them in the C/Python/whatever world.

I haven't used Maven in a bunch of years.

But source code goes in src/main/java. Test code goes in src/test/java, resources (images, files, whatever) go in src/main/resources, test resources go in src/test/resources. There is a pom.xml file that contains dependencies, and if it doesn't, it points to a parent which <<does>> contain them.

There might be submodules and they all are folders containing another pom.xml. That pom.xml generally points to the main pom.xml.

Etc. There's a bit of tedium in there, but <<it's always the same>>!

The level of consistency is something most ecosystems dream about.

And you know what? That means that you can use your spare brain cells for actual creativity or hard work.

I have done 10+ years of Java development and 1 year of Python. And I disagree, Java packaging is better. You can package all your dependencies in a fat jar like Spring boot does and you only need to run it with java -jar lib.jar and you are good to go.

Dependency management is a mess in the Python world and my number 1 complaint. You have pip, pipenv, poetry, anaconda, to manage your 3rd parties. And if you want to avoid conflicts and have consistent runtime you need to create a virtual environment to install these libraries and redownload everything.

Otherwise libraries are installed user or system wide, in a directory that depends on the Python version that you use but they are not versioned there.

With Maven you don´t have the issue. Your 3rd parties are downloaded in a global cache directory (the Maven repo) and are all versioned. For the delivery you can either provide them all and have the runner set the CLASSPATH or have them packaged into a single jar.

Great, but you will actually want to write some code, so you have to do this insane IDE setup clickfest.

I am not saying Python's packaging is any good, it sucks. But you can't realistically ignore Java's IDE shitshow.

It’s pretty much an import project and done. If not, then someone really fcked up that build file.

But this happened to me every time I had to use any java project over the years. Maybe I'm just unlucky.

Please try OSS java projects and educate your build guy. Java the language may have lots of complaints but from the build/packaging point of view its actually one of the best languages out there. And very, very consistent.

I have no clue what you're talking about.

Maven can build a nice fat executable jar, with just one XML block added to the pom.

I've opened a 5year unmaintained project with IntelliJ yesterday - click package and got a working executable.

I opened another project built with SBT... and guess what? No errors!

Let's try that with a Python project (LOL!)

> I can't stand python - too wishy-washy.

That's an interesting take; my experience has been (at least the community) is quite proudly dogmatic on the One Right Way To Do It, for all values of "It".

I also don't like Python, but I think for different reasons and perhaps a bit less than you.

Never used R, but I understand it to be quite fiddly.

> my experience has been (at least the community) is quite proudly dogmatic on the One Right Way To Do It, for all values of "It".

This hasn't been true lately. Python is growing ever more complex.

I hear you and agree to an extent; my suspicion is the influx of non-CS self-styled "data scientists" using Python for all the ML hype has diluted the historical rabidness.

But, for people still working in "computer science-y" domains, I haven't seen the historical dogmatism lessen; in many cases people are digging in.

R is an analysis tool. You don't write software in R.

Well, maybe _you_ don't write software in R. Others can write a pretty capable server[0] using just that.

[0] https://github.com/opencpu/opencpu/tree/master/R

Of course you can write anything in R but that doesn't mean it will be possible to maintain the codebase for the next 10 maybe 20 years including dev team changes.

In other words R is wrong tool to use for building the software just from maintainability standpoint.

I agree that in general one would not use R to engineer software. However I disagree regarding low maintainability.

I've seen very complex R packages that have been around for 30+ years. They are actively maintained. Applying good design principles, unit testing, modularity - all that is very much doable in R.

And you will want to write software in R, especially if the purpose is to support the very activity of working with R packages and to expose the R statistical environment over HTTP. Case in point, the software I link to, in my comment above.

Most python software use the standard venev setup with pip and requirements.txt, and most java software are distributed as fat jars. Nobody manually sets up dependencies anymore.

As a Java/Python/Go developer I can say that packaging is a solved problem as of 2021.

That's really not true.

I like to point out Airflow as an example. It's still a complete nightmare to deal with and it has many steps for just packaging the application.

https://github.com/apache/airflow

I have less trouble packaging and running a 3 year unmaintained Maven Java application, than actively maintained Airflow. And let's not forget that I can target Java 8, without having to install Java 8 on my machine. (Which only adds to Python's complexity)

What do you mean about R? Its packaging and distribution system is great!

Python's packaging is a lot better with Poetry. Use Poetry and build the wheel with poetry build. The lockfile lets you create an exact virtual environment.

Like you mentioned, wheel files are not nearly as customizable as JARs. You can easily pick & choose the dependencies to include in JARs (fat JARs) and change namespaces of dependencies via shading to prevent collisions.

I've generally found it easier to add JARs to Spark clusters compared to attaching wheel files to PySpark clusters.

I'm all for a good project-based package manager for Python, but I have tried Poetry several times over the last years and not once has it worked as intended. It always failed to install packages for some reason.

I have to say that I use Python a great deal and I just never run into these issues. Nor have I ever even used Anaconda, I just use virtualenvs and pip and it all works.

It's my guess that you are using a lot of Python libraries that are wrappers around C or C++ libraries which have dodgy cross-platform support, because I can't see another way this is happening. If so, it's hardly fair to blame Python for that...

You are kind of right. Install non standard stuff in python is sometimes pain. BUT you can do it absolutely right. Also most of the time only few packages make you trouble. So you just need sometimes a fix for them. That's it. You even can download it and make it completely offline. Instagram is running on python... so it is battle tested. By the way, I would always try to avoid anaconda in production.

> main complaint about Python is trying to use someone else's software. Then it degenerates into a maze of twisty little passages of trying to install the correct set of dependencies, not knowing where those files are installed

Java is far worse on this front. Let's take a look at JEE or Spring - The entire experience is akin to summoning a demon with waving reciting correct incantations in correct places while having no idea about what even goes on in the background. There are MessageChannels, MessageProducers, ClientFactories that interact in mysterious ways and initiated in also very mysterious ways. And then there's EJB's and their containers which are just blackboxes of automagic.

Or how about Hibernate?

Well, it does considerably more things than a hello world python app. So at least compare it to Python’s full blown backend server with ORM and the like.

Django: zero dependencies for years, you could literally copy paste the code and it worked, ORM included.

It now has 2 dependencies. It is a 1000 times easier to use than hibernate, despite being one of the worse ORM in Python.

That the problem with old Java libs like Swing or Hibernate: they have been designed for the 90' and don't leverage the potential of what a modern Java version can do.

You can use lean Java persistence frameworks with lesser dependencies if you wish. I think mybatis only has 4/5 dependencies and 2 of them are the the standard slf4/logback dependencies.

There are many modern java libraries that are very lean and mean out there and package into small-sized redistributable assemblies.

Pointing to Swing or Hibernate is akin to pointing to GoLang of 2012 or Python of 2001.

Reply on news.ycombinator.com