Hacker News Reader

Comment by Mahn | Hacker News Reader

> How to use downloads?

> 1. Extract the archive.

> 2. cd into the directory just created.

> 3. Execute ./gogs web and you’re done.

Can't beat the simplicity of running Go applications. It's funny because running a compiled binary is so incredibly basic to computing, and yet 90% of the time installing a new shiny toy in a server involves dealing with 342525 dependencies, half of which broke because god knows what dependency wasn't targeting the right version. And you sit there debugging dependencies and errors and pulling your hair for two hours, trying to get someone else's mess to work, wondering how did we get this so wrong.

View on HN

A lot of Java software is "(1) download the jar file, (2) java -jar thefile.jar". Agree it's good for the user, but when people talk about distributing a statically-linked program the "what about security updates" question invariably comes up.

It is still easier with static compiles if you ask me:

"Make sure you're running Gogs 1.64.2 or above"

"Make sure you're running Gogs 1.64.2, OpenSSL 1.2e, libcrypt 3.73, leftpad 2.0..."

Especially when the deployment of said binaries doesn't involve anything installed on the host OS.

"Make sure you run Foo 1.7, Bar 2.1, Baz 2.3, ..."

"Make sure you update to OpenSSL 1.2e and restart all services."

If you think of more than one program using the same module, shared libraries make a lot of sense.

They do, but static linking means the developer actually does have control of the environment. What happens if you ship the latest and greatest binary but the shared libraries aren't updated by the user?

That is why libraries guarantee API/ABI compatibility within a version number (SONAME). The package manager of your distribution will be responsible for updating the required dependencies if needed.

Of course, if the user chooses to install software outside of the package manager, it is their responsibility to upgrade dependencies accordingly.

You do run into false dependencies causing needless headaches with dynamic libraries though. There's a vulnerability in hash X in OpenSSL. One (or worse zero) of my 15 installed applications that depend on OpenSSL actually uses hash X, but in order to upgrade, I have to carefully manage those 15 dependencies. With statically linked dependencies, the hash X code wouldn't be in any of the other binaries and I only update the one (or perhaps don't have to update at all).

What do you mean by managing dependencies? You upgrade to an OpenSSL version that fixes hash X and be done with it? I don't understand your point.

Can't upgrade if the fix was dropping the hash because it's fundamentally flawed.

That would be an ABI incompatible change. These are only allowed when increasing the compatibility version (SONAME). You would immediately notice such a change. For the usual package managers that would even require to create a new package.

I guess under the assumption of everyone following sane versioning practices, maintaining alias symlinks, and/or really solid package management and compatibility testing, this isn't such an issue. If the system admin was compiling a lot of their own libraries and binaries that don't necessarily follow those best practices, it becomes more of a problem. The system admin doesn't know what functionality in the library is used by each binary and weather or not the upgrade is really necessary. If the binaries are self-contained and statically linked, the maintainer of the actual software can stay abreast of security updates in the libraries that they are using and ship updates only when vulnerabilities affect functionality that they use. The sys-admin then only has to download updated binaries if truly necessary and tracking any dependencies on the end-user system becomes unneeded. Not really arguing that it's a better setup. It does mean that software maintainers have to stay active and on-top of security patches. The complexity is inherent, it's just a matter of shifting it around onto different people.

> It is still easier with static compiles if you ask me

You'd be surprised how many users have no idea whether they are running the 32bit or 64bit version of their OS. For them, running a Go binary will be anything but easy.

You'll have the same problem with dynamic linking. It's even worse when your user has to know which version of Python/Java/etc they need to run your software on. The best case for your user is a static native binary behind a package repository.

Now I have to install java and keep it up to date, in addition to the jar.

Yeah, the missing bit is that this should be done by OS vendors which sadly is not true.

If package management is important to you, you might want to try an operating system that integrates it natively.

The point of "run anywhere" is to not have to search for a special operating system.

I wouldn't conflate the "run anywhere" aspect with OS-level package management.

How operating systems handle updates is unrelated to Java, and affects other applications on the system as well. Think about other "run anywhere" technology like Flash, PDF (Adobe Reader), Microsoft Office, Browsers -- they also don't have a great security story and need constant updates. Adobe and Sun decided to ship their software with "Update Managers". Everybody loved them (not), but at least they fulfilled their purpose. 10 years later, Google added background updates with binary patching to Chrome on Windows and Mac OS (not needed on Linux). They decided to just install updates without the user's consent, because that will at least keep them safe. Google can get away with it because everybody expects them to know what they're doing. But I certainly don't want downloading code at runtime to become the norm.

That's why distributing updates for third-party apps should be an operating system feature, as it is in all popular Linux distributions. Microsoft and Apple, please feel free to copy.

> That's why distributing updates for third-party apps should be an operating system feature, as it is in all popular Linux distributions. Microsoft and Apple, please feel free to copy.

That's exactly what I meant when I said

> Yeah, the missing bit is that this should be done by OS vendors which sadly is not true.

Only if you don't deliver it alongside the application, which is what statically compiling the Go runtime means.

Are there many java applications that bundle the jre?

Yes, it is quite common in desktop applications to do so.

Usually you get the option to use the bundled one or whatever is available if one is detected already on the system.

To the point that is one of the options of the new Java application packer introduced with Java 8.

Also in all server deployments that we do for our customers, Java is just another configuration of the whole application packaging.

Also many companies don't mind paying for commercial Java AOT compilers to native code, which then puts their Java code in the same league as Go.

I really wish more applications would do it. The situation here landed with running two different RDS servers, and users having to manage running applications from different environments simultaneously.

Because some Salesforce product required a specific version of Java and Oracle's PMS software required a specific, different version of Java. And both vendors only support their products by having it run with only the correct version of Java installed.

And then I have this VM with this ancient version of Java I can't even find for download any more so that I can use the management interface for Brocade FC switches.

A lot of Java software is also "(1) download the tar, (2) run bin/whatever which is a 500 line bash script for setting up the jvm environment"

6 of one, half a dozen of the other.

As long as it's behind and simple interface and it works, I don't care what it actually does.

And..., just like that, you are part of a bot net.

The truth is, nobody personally vets the source of all the software they use. Even people who compile everything themselves aren't reading the source code that came over the wire. It baffles me when people fret over "don't blindly run that curl|bash installation script because security!" when nobody would bat an eye at a .deb downloaded from the same web site.

There are many that take an interest in securing their workstations, just as there are many negligent individuals who blindly run things they download from somewhere.

A minimum amount of care when installing things (checking signatures/checksums, reading install scripts, etc) should be common sense and not doing that is grossly negligent.

What's the point of reading an install script if you don't also read the source of what the script is installing? That's like a proofreader only editing the intro paragraph and then signing off on the entire essay.

While I'm all for downvoting the grandparent, I'd say reading the installer script is wise even if you trust the source. Developers aren't good at shell, much less portable shell, and such scripts are usually far inferior in quality to the main application code (written in pretty much anything that's not shell). Since an installer typically manipulates privileged locations on your system it's also playing with fire. Taking few minutes to establish whether it was written competently can save a lot of pain down the road!

While that's true, you can also say the same about the preinst/postinst in that .deb package hosted on someone's PPA. Also, while every .deb/.rpm must be installed using root privileges, many (not all) shell script installers don't require root and just perform a local install, making them less dangerous by nature.

The point is that many install scripts do insecure things such as downloading over unencrypted channels, demand sudo without a good reason and are generally not careful to have a minimal impact to the machine.

At the minimum I want to know what it does and where it puts stuff.

This assumes that the software package itself is trusted, which is (still?) a reasonable assumption to make for open-source and big name companies.

By the way, it's poor form to downvote someone just for disagreeing with them.

Sure, but if I'm really worried, egress filtering will catch it or the VM won't be allowed to connect out to any non-whitelisted hosts.

Security is about defense in depth, not just "don't run anything you haven't read in detail" - have you read the code of the web browser you posted with?

Unless audit the code you download, you still could be pwned. Compiling or downloading from github isn't some magic talisman.

Well, it's more like:

(2) run bin/whatever which is a 500 line bash script for setting up the jvm environment that has a 50% chance of not working if you are on a distro that's not Ubuntu or RHEL, and probably also fails if it takes a filename and you give it a filename containing a space.

Well that sounds like a personal problem. Who puts spaces in a filename on UNIX? That's just bad form, and anyway, if you did decide to throw prudence and best practices out the window, you should properly escape it.

Don't forget (0) install the right JVM. This has caused me much grief in the past, at least on Linux.

Assuming you have the JVM installed.

Yes. Digging up dependencies is the worst. Managing another fucking build tool is the worst. To be honest, anything short of clicking a thing is the worst. Because there's no reason it can't be as simple as clicking a thing.

I've always believed that 100% of all dependencies should be included in projects. Whether that is raw source code, amalgamated source code, static libs, or dynamic libs I don't care. Give to me the things I need to run and/or build your project. Don't make me hunt shit down.

> I've always believed that 100% of all dependencies should be included in projects.

Including libc, Mesa, and libX11?

It's pretty untenable to do this in the limit. Eventually you're forced into using a good package manager, and once you have one you might as well use it for the rest of your dependencies too.

I make video games. There's no such thing as a package manager that supports Linux, OS X, Windows, iOS, Playstation, and Xbox. So from my perspective no. No you aren't forced into something that doesn't exist. ;)

There is a line. You shouldn't include operative systems. Nor language tools (C++ compiler, Python installer, etc). Nor hardware SDKs (DirectX, OpenGL, etc). But beyond that? My vote is strongly in favor of including dependencies.

Compiling OpensSSL on Windows is a god damn nightmare. I shouldn't have to fucking install StrawberryPerl. SQLite is beautiful in comparison. It's two .c files and two .h files. You can also download pre-compiled libs if you want.

Almost all open source projects could easily be single file. If it can be single file then, in my opinion, it should be.

I don't know if I'd go quite that far, but yeah if you have to do stuff cross-platform, packaging your own dependencies is a great solution.

As itself, bundling dependencies is not necessary a bad thing, but it often leads to really bad practices.

Once you bundle dependencies, you make the choice of having to track those dependencies for security fixes and bug fixes. Otherwise you end up exposing bugs and security issues to your users.

Too often, I've seen applications (specially in the "Enterprise" Java World) with hard pinned versions from 2, 3, 4 and even more years old, with tons of bugs and tons of security issues.

Bundling make sense in many cases, for example if you distribute a product, it permits to control most of the stack and to factor maintenance across several platforms.

Not bundling also make sense, for example if you are a distribution, you have tight control over what put in, and it reduces the effort to maintain stuff because you don't have to track x, y, z each in versions n, m, o.

Bundling or not is a trade-off, either way, it must be done properly.

Dependency handling is a complex task, and yeah, it sucks. And it's not technically that the problem will be solved, it's with proper organization, conventions and normalization.

Gitbucket [1] a git hosting system built in scala is as simple as typing "java -jar gitbucket.war"

[1] https://github.com/gitbucket/gitbucket

40MB download + requirement of Java Runtime (~100MB) to be present is not quite same. Though to their credit they have packaged all jar dependencies besides JRE in WEB-INF/lib

That's how WAR files get built. It's a very standard mechanism for deploying Java apps.

I think having Java on your machine is a reasonable requirement. I mean the equivalent in Ruby would be to have Ruby, install all gems and THEN launch the app.

Except with go I don't have to have anything else installed. And that is the way it should be.

Why Java on machine is reasonable requirement? The Gogs service does not need Go to run. So having Java is extra requirement for Gitbucket.

I know about packaging and running Java web apps. I am pointing out that compare to Gogs, Java based solution has extra requirement of JRE which I have to keep patched and updated. And this is independent of any fix/improvement of Gitbucket itself.

Your patched and updated argument is an equivalent one.

Are you saying if go gets updated, you will not have to rebuild your gogs binary? Go does not provide a better security infrastructure than Java. I grant that just running a binary is cool, but as I said, having Java is OK as far as requirements go IMHO. others may have different opinions.

Atleast its better than installing a ruby app.

One runtime dependency isn't bad, but it's still worse than zero. :/

Did libc stop being a runtime dependency when I wasn't looking?

I don't know if libc ever was a Go dependency, but it isn't today.

I spoke imprecisely; you are correct. That said, I don't think, of the software that has escaped the Golang pits and weaseled its way into systems I am forced to maintain, any of it does not have a cgo, and thus a libc, dependency.

If you actually want to run this software, JVM is a reasonable requirement: it's well-known, well-tested, easy to install, and comes in one package.

It's not zero requirements, though. (Go binaries still need platform-specific libc, AFAIK, but it's usually also a reasonable requirement.)

Actually, they mostly don't. Go doesn't use libc, fortunately.

Runtimes like ruby and python are quite commonly found in the default installation of many distributions (and OSX, IIRC). Java is not.

It's mostly a result of trying to avoid duplication. It's a non-issue for a server running one service but quite relevant on, for example, a dev workstation with dozens of projects or for smaller tools – imagine if every system command brought it's own dependencies.

But the trend seems to go towards sacrificing disk space for isolation, i. e. NPM which makes it more or less impossible to use a system-wide 'node_modules'.

> Can't beat the simplicity of running Go applications.

... as long as you downloaded the right binary.

This is a dev tool, devs usually know which arch they are running. Besides, you can detect the arch using javascript like google with google chrome download. Here's how you do it: window.navigator.platform.indexOf("64") and then show the right link checking window.navigator.platform.indexOf("Linux") for each OS. Easy as pie :)

This is true of any application; still better than a dynamic binary

If this is so desirable, why haven't people been shipping open source C and C++ software this way? It's not like Go invented static linking.

Not just C and C++, either -- Python, Ruby etc software could be shipped as statically linked executables.

People often _do_ ship C and C++ software this way with dpkg.

Static linking is sometimes discouraged because it makes it harder to identify what needs to be upgraded in case of (for example) security vulnerables in core libraries like libc.

Could you elaborate on how Python software can be shipped as a statically linked executable? I'm not familiar an easy way to do that, and it would help me quite a bit.

It's not really a static Python .exe, but PyInstaller can create a single executable that embeds all of its dependencies in the file. Except for a small delay at startup it works really well.

This is how I distribute Grow (grow.io), a Python program. It's worked really well so far, but it's also produced a few issues in development, for me.

For example, PyInstaller itself changes drastically from version-to-version, and I've previously had to spend hours picking away at why "compiled" Grow worked before but no longer works post-PyInstaller updates.

Overall happy though, and much happier to write in Python and distribute a single executable without requiring folks to muck around with Python versions, pip, dependencies, etc.

The python distribution story is far behind Go. This article goes into more details: https://glyph.twistedmatrix.com/2015/09/software-you-can-use...

PS: I'm the original author of PyInstaller. I obviously love it but it's a gross hack around the fact that Python doesn't have a serious distribution story.

Reply on news.ycombinator.com