Hacker News Reader

Comment by javajosh | Hacker News Reader

Yes, if I was to learn a systems programming language, I would only pick from those that allow manual memory manipulation. Either that or the ability to trivially integrate with C functions that can do manual memory manipulation.

View on HN

I fear you are confusing systems programming language and operating systems programming language (or conflating).

Go is a great systems programming language especially for modern concurrent systems — http servers etc. It talks to C easily so you can integrate all kinds of system level code into your apps. Having a Garbage collector makes it trivial to write long-running daemons.

Modern operating systems have traditionally been written in C (and C++) plus some assembly language code. D can do everything C & C++ can do but it would still no doubt need the assembly language code.

Go lacks manual memory management. Some say that this would be a barrier for writing an operating system while others don't. The fact that you would have to use some assembly language code to talk to the hardware and you might need to add some manual memory management via assembly language code. After that I'm sure the garbage collector would make the OS more reliable and potentially a little quicker in places.

Either way I don't see why Go gets criticised for needing a bootstrap layer when operating systems written in C and C++ also need this.

There are tons of user-level applications that need manual memory management and that don't have anything to do with operating systems, like for instance video encoding/decoding/streaming or high-frequency trading.

What really bothers me about Go is not really that it's garbage collected, but rather that its garbage collector sucks so badly. Really, not all garbage collectors are created equal.

For example I'm working on a startup and we've been integrating with various bidding exchanges for serving targeted ads. All the bidding exchanges want the response to be generated in under 100ms, which includes the network roundtrip. This means on the server-side, the average must not be higher than 10ms per request, preferably lower. Scala on the JVM can handle it, but when I tried out Go, it was a disaster ... as that garbage collector stops the world and it's totally unpredictable, so you end up with spikes of latency that can upset your partners and given enough incoming requests, it can also blow up your buffers/queues, crashing your servers. It's also non-compacting, but that's a given, as it's not even fully precise yet.

Which is the reason for why integrations with bidding exchanges are usually written in C++ too (in case it's not clear, we are talking about B2B web services). We've gone with the JVM because it provides a good productivity/performance balance, but eternal vigilance is needed in profiling the memory allocation patterns and tuning the garbage collector to handle the load. And Go requires even more tuning. Which is why sometimes I fantasise about a high-level language that allows for manual memory management, as things would be so much easier ... although I'm rooting more for Mozilla's Rust, than I am for D.

Sociomantic is a German based company that does online real-time bidding and is using D to do it: you can find more info in their blogpost about Dconf 2013 with links to two talks talking about their company and how they dealt with the GC issues you mention.

https://www.sociomantic.com/blog/2013/06/dconf-2013-review/

Interesting you chose Scala. I have built and managed Jetty based adservers to talk to bidding exchanges and wondered how would the system perform if the adserver was written in Scala instead. Going with Jetty initially was the right choice as none on the team were experts in Scala, and the base assumption was that Jetty is battle tested, has good documentation and is already optimized to perform great from the get go. So, we simply chose Jetty. I didn't get a chance to build an equivalent system in Scala to compare performance. Will be great, if you can throw some light on how you went about building highly performant adservers in Scala, and scaled them. Does coding in Scala make it a lot more easy to deal with concurrency (for development and debugging), keeping a small codebase, help in rapid iteration, and in general make it fun to program and manage the servers? Do you have a blog post or a write up on this topic? I encourage you to make one, if you don't have one already. Good karma, and an opportunity to showcase your engineering chops to hire other great engineers.

I wrote a simulation software to test a product in my previous job which used akka mostly but also used AHC which is a plain java http client library without problems. Size-wise the code would periodically get more features but by refactoring often it never got beyond 2500~3000 LOCs. Readability was never harmed; Scala - contrary to what some people think - allows for very clear code. People that inherited the code is working on adding features. Obfuscated code can be written in PHP if you want (or are unable to do otherwise). Another piece of scala software was an internal web service written using unfiltered on jetty. It's so stable sometimes I just forget it's there.

I think that Objective-C would qualify as a reasonably high-level language and offers a choice between manual and automatic, but very predictable memory management.

Plus, it's also pure C whenever you want.

I'm building a similar infrastructure (different purpose). Can you give a few more details about the problems you ran into?

What version of Go?

Message rate per second?

Message size?

What was the % increase over average response time during GC?

What class of hardware?

I did notice the Go's GC improved between 1.0 and 1.1.

Thanks!

> Modern operating systems have traditionally been written in C (and C++) plus some assembly language code. D can do everything C & C++ can do but it would still no doubt need the assembly language code.

Actually, contrary to C and C++ language specification, in D, support for inline assembly is part of the language specification.

> downvote

sambeau 50 minutes ago | link | parent | flag

I fear you are confusing systems programming language and operating systems programming language (or conflating).

While I am on the D and Rust field, I support Go's ability to do this.

Go is no different from Oberon in system capabilities. And Oberon was used to write quite a few desktop systems used at Zurich's ETHZ during the mid to late 90's.

The OS bootloader and the kernel package for hardware interactions were written in Assembly, with the remaing parts in Oberon.

You can read all about it here, http://www.inf.ethz.ch/personal/wirth/books/ProjectOberon.pd...

> Either way I don't see why Go gets criticised for needing a bootstrap layer when operating systems written in C and C++ also need this.

Because many tend to assume compiler extensions to C and C++ are part of the language.

It is a fun quiz asking C and C++ developers what is part of the standard and what is compiler specific behaviour.

>I fear you are confusing systems programming language and operating systems programming language (or conflating).

Systems programming doesn't seem to be a very well defined term. My understanding is that it is certainly not application programming and it requires pretty tight management of hardware resources. That includes things like operating systems, database systems, embedded systems, networking software like firewalls, etc.

The way I like to think of it is that a "systems langauge" is something you could use to implement a "real"[0] GC. Does this ring true to anyone else?

[0] You know what I mean.

A systems language is one with enough features to allow developers to write kernel code and device drivers, even if with a little help from assembly.

A GC is just a special case of a memory manager.

my own take :

systems language => for building systems, many parts, many interconnections

operating system language => ability to use the hardware with no limitation. even for a monolithic part.

> systems language => for building systems, many parts, many interconnections

So... every language?

I guess it's a difference in terminology. I've never heard anyone refer to writing an HTTP server as "systems programming"; I'm used to "systems programming" meaning the same thing as "operating systems programming". Sometimes people refer to writing higher level OS components (like init or libc) as "systems programming" as well, but I would think that a server that isn't a core part of an OS would just fall under the category of "server programming".

I would argue go has a terrible ffi due to their tool chain being awkwardly integrated (at best) with the native c abi. Calling into c is expensive.

Reply on news.ycombinator.com