Hacker News Reader

Comment by ypcx | Hacker News Reader

Well, Go seems to have enabled a new wave of software renaissance - Kubernetes, Docker, you name it - Go seems to have filled the space that Java was too fat for, JavaScript too light for, and C/C++ too difficult/fun-less to use for (which is everything).

After 20 years of coding, and having just recently discovered LISP and learned Clojure, I believe the perfect language will be a Clojure compiled to native with the ease of Go (e.g. w/ similar import system).

One such direction seems to be Carp (https://github.com/carp-lang/Carp), but I haven't tried it yet.

View on HN

>Well, Go seems to have enabled a new wave of software renaissance - Kubernetes, Docker, you name it

Let's name it. I wouldn't call either of these a "software renaissance". Docker is a wrapper on top of Linux kernel containers, and Kubernetes is a 10000 pound gorilla on top (and port of a C++ app, some say even through a Java rewrite/transpile to Go).

Go was just at the peak of being fashionable at the time (and had some basic features not semantics/syntax related like easy static compilation and cross platform-ness) so got adopted by these projects, the same way many new projects now use Rust.

Docker is a wrapper on top of kernel containers in the same way that Emacs is a wrapper on top of the kernel filesystem API.

I've written a 150-line container manager in shell before (because we had a use case where both Docker was not the right technical fit out of the box and the organization had barely any operational experience with even normal use of Docker), and it's great that containers are built into the kernel and you can just use unshare and a bunch of bind mounts if you'd like, but Docker is its own quite substantial product and brings its own approach to building systems (Dockerfiles, container networking, etc.) that aren't implied by or required by the kernel API.

Docker is a bad design. It is monstrously overcomplicated for what it does. In historico-aesthetic analogies, it's baroque, not renaissance.

This sounds like dogma. While I do agree there are some pieces which are over complicated or just plain bad code, on the whole it's more that the API's aren't quite right specifically for people who want to integrate with it rather than use it. containerd exists pretty much because of this observation.

And what does it actually give that is new? What does Docker give that you can't get with a normal VM, with Terraform to spin up however many instances that you need? Put an app on an AMI, save it, if you need one instance, or a 1,000 instances, spin them up with Terraform. You get to stick with normal operating systems, such as Linux and Windows. You don't have to learn a bunch of new technologies.

http://www.smashcompany.com/technology/docker-is-a-dangerous...

Terraform is written in Go as well.

Terraform needs an extensible DSL, so Python or Groovy or Kotlin would have been much better.

> What does Docker give that you can't get with a normal VM

There is a difference between Linux container (Docker underhood) and full blown Virtual Machine. In case of docker you are not running new operating system, you are just running new processes in different namespace but still using the same kernel. Linux containers are lightweight, you can start them as easy and fast as other processes, without overhead. VM (in most cases) is separate operating system running, which means startup time is different, you have overhead (CPU usage for keeping OS running and all its services, additional RAM usage, additional disk space usage etc) of whole operating system + virtualization overhead (in a lot of cases but can be minimzed).

Yes, I know, but what does that give me? When I'm setting up the infrastructure for my company, what can I do with Docker that I can't do more easily with a standard VM and Terraform? In terms of security, isolation, orchestration of services, managing concurrency, I get everything I need with ordinary VMs and Terraform. I don't see what Docker gives me.

Docker was written in go before go was fashionable. It was mainly written in go because it wasn't python or ruby and didn't want to participate in the language wars. K8s is written in Go because Docker was in Go and sharing that dev community made a whole lot of sense.

> Well, Go seems to have enabled a new wave of software renaissance - Kubernetes, Docker

Go turned out to be a pretty bad choice for container tech due to its awful multithreaded runtime. Things could have been different if it focused on a single threaded runtime or provided control over OS threads, but it did neither.

Also neither of those projects were enabled by Go. One is a pretty low quality but strategic push into the cloud and other one is a pivot from an attempt to solve real problems of building a PaaS hosting platform.

Modern Go solved the problem of control over native threads threads. But it is puzzling indeed that it took almost 10 years despite popularity with containers.

Only partially... what's available at least prevents leaking the context unexpectedly to other goroutines (due to thread re-use), however anything that spins up a new goroutine from the locked goroutine will end up in a new thread in a totally different context.

Even stdlib spins up new goroutines.

> Go turned out to be a pretty bad choice for container tech due to its awful multithreaded runtime.

Can you please elaborate on this? My experience with Go is that it is able to fork/exec programs and manage cgroups via sysfs on Linux at least as well as any other programming language can.

You pretty much need to do everything from a separate process, which is the main issue.

Certain things like setns(mountns) will straight up fail in a multithreaded program, so there are some nasty hacks in runc to handle such things in C prior to the go runtime firing up.

> You pretty much need to do everything from a separate process, which is the main issue.

I’m confused. fork()/spawn() are the proper means to launch child processes on Linux. What language would do things differently if the syscall determines the means here?

Because go is always multithreaded. You literally cannot do certain system level things in Go. For instance trying to "setns" to a mount namespace will yield EINVAL from the kernel in all cases with Go.

In fact, all namespace related operations are really annoying to do from Go because the runtime is always multithreaded and you have no control over the threading except for locking the active goroutine to its currently assigned thread. This also makes that thread unusable for any new goroutine (even goroutines that are spawned from code running in that thread), which has the side-effect that the new goroutine will always be in the original namespace context.

> Go turned out to be a pretty bad choice for container tech due to its awful multithreaded runtime.

What you mean by that? What makes it a bad choice in context of containers ? I ask because I know how Go is using M:N threading model in its runtime and what linux containers really are, but I can't find a reason in which I would use word awful to describe using Go in this context.

Because container tech is often per-thread which you have no control over in Go.

There is runtime.LockOSThread(), but then if something happens to spin up a new groutine from there it will be done on a new os thread and not running in the container context.

It's not totally impossible to work around, however it does make many things incredibly frustrating.

Kubernetes was transpiled to Go from Java, so I wouldn’t use it as an example in support of your point. (Read the source code; it’s pretty obvious — and it’s been confirmed elsewhere.). Some components such as etcd, however, are pure Go.

And is etcd significantly better than Zookeeper ? Not really. So not a great example for Go's supremacy.

Also with GraalVM you can equally support low memory, fast startup use cases.

I’m not sure that people are arguing about “supremacy” as much as they are arguing that Go was an attractive choice for the authors vis-a-vis other languages.

Getting into language or tool superiority arguments without deeply analyzing the environmental factors and use cases seems like a pointless exercise to me, and ought to be discouraged here IMO.

Etcd may not be, but Consul is in my experience substantially easier() to run in production than ZK, and is also written in Go. I’d say this is nothing to do with implementation language and much more to do with a more understandable consensus model.

( based on having spoken to hundreds of operators of both over the last few years, while (disclaimer) working for HashiCorp and other distributed system vendors, but also having personally run both at serious scale)

Sure, Go didn't turn out to be well suited for distributed systems either. But neither are JVM languages.

Sandboxed network classloaders were a big win for Hadoop and Spark.

Is the JVM's sandboxing still supported, patched and believed to be secure? I had assumed they gave up on it when they removed applet and webstart support.

Thought it over and you're right, Spark has some SecurityManager glue I've vaguely glanced at, but HDFS relies more on Kerberos than anything in-proc with the workers. IPC into an opaque worker binary would be a lot more painful but can be made to work.

> And is etcd significantly better than Zookeeper ? Not really. So not a great example for Go's supremacy.

ZK is using Paxos and Etcd is using Raft. It's difficult to compare languages in this case as you will compare consensus algorithms mostly.

> Also with GraalVM you can equally support low memory, fast startup use cases.

How is GraalVM performance this days? Last time I've checked[1] it was a lot slower than JIT.

1. https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

ZooKeeper uses a custom atomic broadcast protocol, Zab, not Paxos: https://cwiki.apache.org/confluence/display/ZOOKEEPER/Zab+vs...

(Atomic broadcast can be reduced to consensus, but frames the problem differently.)

> C++ too difficult/fun-less

C++ would be a lot more fun without the obligation to maintain header files all the time.

This is one thing that modern languages get right.

Modules are in C++20 and you can already play around with them.

aren't header files quite useful for yourself and especially other devs as a quick summary? I've found myself wishing my IDE would automatically make a header file equivalent for my python code (VS Code actually does this, but gives a lot of other info that's extraneous in my opinion)

I've been learning a bit of Go and I like what I've understood so far but one thing that seems somewhat lacking in the Go Eco system is mathy projects. I know there are some but I couldn't find many that were both active and to my taste. I was looking to learn and hopefully contribute. I was mostly looking for something mature for either mathematical optimization (stuff like linear, quadratic or integer programming) and (non-DL) machine learning. For C++, Java, Scala, Python and Julia (less sure about this), there seems to be much more.

You can compare Go's gonum to Python's numpy. For Java's Weka, Scala's MLlib, Python's scikit-learn, C++'s Dlib to what Go is offering.

Maybe Go leans more on its C-interop for these sorts of things which is a bit like numpy.

Am I missing something? Will this change?

What are you not sure of with Julia here? There are places in the ecosystem to not be sure of, but this isn't one. Julia has probably the most advanced mathematical optimization right now with JuMP (http://www.juliaopt.org/JuMP.jl/v0.19.2/) and some of the most advanced post-DL machine learning with the full language differentiable programming tools (Zygote, Tracker, ForwardDiff) which have showcased applications like quantum machine learning and neural stochastic differential equations (https://arxiv.org/abs/1907.07587). Some of it is still in flux, but in terms of ecosystem there's a lot of stuff there that you won't find in other languages.

I meant personally I am less sure about Julia, that my knowledge is less sure. I've neither used it myself nor spent a few hours browser project source code unlike most of other stuff I mentioned.

Since I am not finding what I am looking for in Go then maybe I should ought to try out Julia.

Thanks for the JuMP reference. That looks cool.

> I've been learning a bit of Go and I like what I've understood so far but one thing that seems somewhat lacking in the Go Eco system is mathy projects.

I think this has to do with golangs poor FFI performance.

Considering it took you that long to be a Lisp convert, I like to know more on how you came to the see the light. I’ve been doing coding for over 15 years, too. And, every time I tried to pick up a Lisp dialect (Clojure was the last) I couldn’t help but wonder what with all the buzz about this. I really wish that I could see the light someday, though.

Depends where you are coming from. What languages do you work in now? If the language you are working in now has excellent support for concurrency, then that part of Clojure is not going to impress you. Or if you currently work with a language that makes it easy to write DSLs, then that part of Clojure won't impress you.

Mostly in Java and now picking up Rust in my spare time.

I wish I had a good answer to this. I suppose I've never looked at anything else than what I was working with (C, C++, Java, JavaScript and much later CoffeeScript) out of "religious" reasons. I guess being really good at what you do can do that to you. I hold CoffeeScript in a very high regard. It has freed me from two things - the spaghetti verbosity of JavaScript, and the abrupt syntax changes of ECMA spec introduced in the recent versions, ones I don't particularly see as a positive development of the language, _especially_ compared to the timeless cleanliness of CoffeeScript. So and I think that CoffeeScript may have thawed me a little towards LISP-like syntax and behavior. I don't know. I think at some point, you realize that you have written (and managed) enough of C/Java-like verbose code and that it's time to try something completely different. So and then it hits you - programming should be data-centric, not language-construct-centric. And that a too big codebase must be a result of ill-specified project scope, more often than not.

> Java was too fat for

I think it should always be mentioned when people call java fat, that the core language is usually fast enough. What makes java "fat" is the mentality of "Frameworks".

For me, Go seems like java, but without the fucking frameworks.

I have been wondering for some time now, when exactly is a language "fat" or "light"?

The idea seems to be there and "obvious" in a way, but haven't been able to pinpoint exactly what it is.

Is it a feeling or an actual metric?

e.g. When I work with Java in IntelliJ, things don't feel significantly slower than developing Node in WebStorm. Is it perhaps the way the code looks, in terms of verbosity?

I do agree that frameworks contribute to this feeling of Java, but again, is it because a Spring Boot webserver boots slower than a Node one?

Or maybe, say a Java cli program vs a C++ one. Is it the speed at which things run?

Could it be that, because we know Java uses a VM then it "should" be slower than compiled code and thus it's fatter?

What is that "thing" that makes us determine that a language is fat or light?

The one area where Java feels fat is the memory usage overhead due to 64-bit pointers being used a lot. So if you end up holding and processing a lot of data in RAM, the process allocation and the garbage collector metrics won't look awesome. That said, recently Openj9 improves on this literally by miles. I agree with all the other points - Java and its tooling are very powerful and by that nature they are enabling us to put that much more load on it, so yes, that's where it will be predominantly visible.

Edit: that said, Java was created when we didn't have containers. (Just as Clojure was created before Java had lambdas and aggregate streams). So it may be the time to come back closer to the metal again, but as you point out, we need to maintain a clear mind while contemplating that.

Yes we did, HP-UX vault and Tru64 sandboxes.

Well,"fat" is an unprecise term anyway, but the main concern when evaluating languages like that is mandatory overhead.

What exactly constitutes overhead is debatable, but generally anything that is not strictly necessary for the functionality is considered as such... Even if it has useful tradeoffs.

Examples from Java are the GC, compound types need to be objects and thus are wasting space with type and vtable pointers, and the jvm is actually an interpreter (one with only stack semantics at that).

Since C++ has none of that, it is quite natural to say that it is more lightweight than Java.

But that in itself is not necessarily a judgement about execution speed, although it is heavily implied. If you write Java in a way that it is heavily jitable and doesn't stress the GC, you can end up with a faster program than a naive C++ version would achieve.

But all else being equal, the C++ version will be ever so slightly fester, because less stuff to do means more stuff done.

I agree that this is the picture I get in my mind when talking about "fat" languages.

So I was wondering if it was something that people felt differently, or if it was mainly the same theme, as it seems like it is.

> What is that "thing" that makes us determine that a language is fat or light?

In the same way the word modern is thrown around in the context of languages that have evolved a bit, I think heaviness is a notion that is an indicator of how committed the person is to the tradeoffs made by the tech: a heavy language made tradeoffs erring on the side of technical debt and the light one has not.

But it's easier to present one's own preferences as technical facts when trying to flame or troll.

In the case of frameworks, they often make choices for you, they are opinionated. It's easier to get unhappy about it, in a bike shed sort of way

It's just that some people repeat what others say without fully comprehending them (this is especially evident in the golang community). People parrot what the golang authors claim, even though many of such claims are baseless and incorrect.

Not just frameworks, but the JVM as well. Having go compile to a reasonably small statically linked binary (which can be shrunk even more by things like UPX if size really matters) is great.

Needing to bring a very large, very complex virtual machine to execute your program adds a lot of “fatness” IMO.

Something that any Java developer was able to do since 2000 with commercial JDKs.

And for those that want the free beer version instead, using a fat jar with linked jvm.lib is hardly different than packing Go's runtime in an executable file.

Kubernetes was written in Java, only got re-written in Go as another team took over and they were into Go bandwagon.

Reply on news.ycombinator.com