Hacker News Reader

Comment by CyberShadow | Hacker News Reader

I don't understand, why reimplement this if it is already implemented with adequate performance/quality in OpenSSL? I thought Go didn't have issues calling C code?

View on HN

Sure, you could do that but imagine the nightmare of modifying the Go standard libraries for stuff like TLS to hook into the OpenSSL C interface. We actually did that for a while and it was horrible: http://blog.jgc.org/2013/01/integrating-openssl-crypto-funct...

Calling C code from Go is something you do as a last resort. It breaks some desirable properties of Go programming, like fast compilation, cross compiling (at least without effort), and static binaries (without serious effort). Code called through cgo will never be idiomatic Go code, unless perhaps if hidden under an abstraction layer which takes time and effort to write and adds overhead.

On top of that, the cgo overhead is significant, not only in time, but in threads, so you'd want it only for big chunks of data and program accounting for it.

Another advantage of Go crypto is having been written in a memory safe language, and with runtime bounds checking. The assembly fast paths break some of this, but not most of it.

Go has moving memory so it generally means calling C requires a bunch of copying. This is slow and annoying.

> Go has moving memory so it generally means calling C SHOULD/WILL require a bunch of copying.

I've been working on a few Go frontend to C libraries and researching what's the exact memory model when calling C: Yes, Go has moving memory and you _shouldn't_ pass a Go pointer to C, and it's actively discouraged [1], but since many libraries just ignore that advice [2] for performance reason Go isn't currently AFAIK not moving any pointer passed through cgo.

That will probably be changed in Go 1.5+, I reckon an official post from the Go developer about the current and future state of C/Go memory interaction would help clarify this.

1: https://github.com/golang/go/issues/8310

2: comment #31 of https://github.com/golang/go/issues/8310

Are you sure that's true? Perhaps they have reserved the ability to do that in the future (can you point to doc on that? ), but right now I'm fairly certain that go does no such thing. The only copies necessary when calling C code are for strings since they aren't guaranteed / required to be null terminated byte arrays in Go. And cgo has C ABI compatibility so function call overhead is non existent.

Function call overhead in cgo is considerable. You're right that it's not from copying, but the runtime scheduler still has to coordinate the blocking call, and the stack needs to be switched out to the C stack, kind of like a context switch.

True, just benchmarked it and found the function call overhead to be about 1.85834729e-7 seconds (185 ns). Which isn't much, but the pure C version would obviously be single nanoseconds for the handful of instructions needed depending on the function call.

185 ns is worse than function call overhead in Python (about 100 ns on this machine)

Python calling Python, Python calling out to a native module, or Python calling through ctypes?

Python calling a no-op Python function def foo(): pass

185 ns is enough to compute over 17k floating point operations on a single 3 GHz CPU (Intel Haswell/Broadwell) core.

No, the slow part is switching stacks, and coordinating with the scheduler. Go's GC doesn't yet move memory, and even so there will probably be allowances for passing pointers from Go to C when it is implemented.

The stacks currently get copied when they run out of space, that may break things already.

cgo is slow due to Go's stack growth and userspace threading model. (The limitations are basically fundamental to the way Go handles concurrency.)

I think it's the overhead of the call or maybe their distaste for the state of repair of OpenSSL, e.g. Heartbleed, that drove them to it. Alternatively it could just be the chance to write some crypto in Go assembly and have it be included in their code base.

I don't think that OpenSSL is a great example of OSS, there are several areas where they don't follow bad practices for no reason and you can just look at their history of serious security flaws. Why would you want to integrate with that? In this case it is better and cleaner to implement the functionality in your code (in this case Go).

Having a history of security flaws is better than having a future of security flaws.

Arguably OpenSSL has both.

there are the same amount of security bugs in almost any software, and the number shows some correlation with the lines of code count. Strictly from the code point of view you can follow best practices and actively training the stuff on security. This cost money and time and nobody really wants to do it. The companies started to do this invested serious amount of money into the project and it shows in the statistics.

http://www.gfi.com/blog/most-vulnerable-operating-systems-an...

In open source this is more of a community thing with little discipline, the nature of the software development is less tight, this yields to mediocre results.

I guess the at Apple security is as far is from design as something can be, probably not a high priority.

Knowing the historical flaws is only useful if invest into mining it and act on the results.

OSS?

http://en.wikipedia.org/wiki/Open-source_software

OpenSSL no longer really qualifies as "adequate safety".

I agree, ECDH and AES-GCM are sufficiently complex to implement it makes sense to call into OpenSSL. OpenSSL has had its problems, but these tend to be with TLS protocol handling. The underlying cryptographic constructions have had a lot of attention.

Well, to tell the truth, the underlying crypto functions are also the ones harder to get wrong, using test vectors. The hardest part is probably trying to make them constant-time, but AGL is also the author and maintainer of Go crypto library, and he did extensive research on constant functions, and contributed code to OpenSSL as well. He even hacked valgrind to check for constant-time at runtime, which blows my mind (https://www.imperialviolet.org/2010/04/01/ctgrind.html).

So I think the general advise not to rewrite a TLS library doesn't fully apply to the Go team like it would apply to us.

The more C code that gets replaced out there increases the system security.

Because assembly is safer than C? Did you read the article?

There is a difference between writing a well contained piece of code inside a memory safe language, and using a memory unsafe language for user space applications.

Additionally removing the dependency on C proves the point that C isn't the only game in town, specially given that the same approach would require Assembly with C anyway.

Agreed, but this is for go only, so it's replacing a mix of go and assembly with a mix of go and assembly. How does that make anything safer?

By not introducing a dependency into OpenSSL, which is the point being discussed.

Missed the indent. I agree then.

Reply on news.ycombinator.com