Swift has (had?) the same issue and I had to write a program to illustrate that Swift is (was?) perfectly happy to segfault under shared access to data structures.
Go has never been memory-safe (in the Rust and Java sense) and it's wild to me that it got branded as such.
This is just two groups of people talking past each other.
It's not as if Go programmers are unaware of the distinction you're talking about. It's literally the premise of the language; it's the basis for "share by communicating, don't communicate by sharing". Obviously, that didn't work out, and modern Go does a lot of sharing and needs a lot of synchronization. But: everybody understands that.
It's been in usage for PLT for at least twenty years[1]. You are at least two decades late to the party.
Software is memory-safe if (a) it never references a memory location outside the address space allocated by or that entity, and (b) it never executes intstruction outside code area created by the compiler and linker within that address space.
Doesn’t NASA have an incredibly strict, specific set of standards for writing safety critical C that helps with writing programs that can be formalized?
C and C++ always defaults to minimum amount of safety for maximum allowance of the compiler interpretation. The priority of the language designers of them is keeping existing terrible code running as long as possible first, letting compilers interpret the source code as freely as possible second.
That's why many military and aerospace code actually uses much safer and significantly more formally verifiable Ada.
If you assume the entire lang, yes. If you use a large subset, no. Furthermore, compiler interpretation might actually be sane! There are more compilers out there than GCC, Clang or MSVC. I suspect many assumptions are being made on this claim.
But a memory-safe program != memory safe language. Memory safe language helps you maintain memory-safety by reducing the chances to cause memory unsafety.
> the issue here is that the "Rust and Java sense" of memory safety is not the actual meaning of the term
So what is the actual meaning? Is it simply "there are no cases of actual exploited bugs in the wild"?
Because in another comment you wrote:
> a term of art was created to describe something complicated; in this case, "memory safety", to describe the property of programming languages that don't admit to memory corruption vulnerabilities, such as stack and heap overflows, use-after-frees, and type confusions. Later, people uninvolved with the popularization of the term took the term and tried to define it from first principles, arriving at a place different than the term of art.
But type confusion is exactly what has been demonstrated in the post's example. So what kind of memory safety does Go actually provide, in the term of art sense?
If you were engaged to do a software security assessment for an established firm that used Go (or Python, or any of the other mainstream languages that do shared-memory concurrency and don't have Rust's type system), and you said "this code is memory-unsafe", showing them this example, you would not be taken seriously.
If people want to make PLT arguments about Rust's correctness advantages, I will step out of the way and let them do that. But this article makes a security claim, and that claim is in the practical sense false.
It's not even always the case that corrupted data structures (or even pointers) in C code are exploitable. You need attacker control of data and where it goes in memory. It's far less often the case in Python or Go --- in fact, it's basically never the case. As evidence for that claim: the zero memory corruption RCEs in all of shipping Go code, of which there is a lot.
I’ll take tptacek’s word over most FAANG type on such topics if we’re doing appeals to authority. The guy is very practical, unlike the Rust community which is incredibly focused on theoretical correctness instead of real-world experiences.
Denial of service can absolutely be a security issue, as can any correctness bug if it leads to unintended behavior or corrupted data.
Sure, those mainframes from the 80's weren't bullet proof either. But you first had to get to them. And even if the data traveled in plain text on leased lines (point-to-point but not actually point-to-point (that would require a lot of digging), no multiplexing) you had to physically move to the country where they were located to eavesdrop on them, and injecting data into the stream was a much harder problem.
It is trivial to change this example into an arbitrary int2ptr cast.
> Go (or Python, or any of the other mainstream languages that do shared-memory concurrency and don't have Rust's type system),
As the article discusses, only Go has this issue. Python and Java and JavaScript and so on are all memory-safe. Maybe you are mixing up "language has data races" and "data races can cause the language itself to be broken"?
> If people want to make PLT arguments about Rust's correctness advantages, I will step out of the way and let them do that. But this article makes a security claim, and that claim is in the practical sense false.
This article makes a claim about the term "memory safety". You are making the claim that that's a security term. I admit I am not familiar with the full history of the term "memory safety", but I do know that "type safety" has been used in PLT for many decades, so it's not like all "safety" terms are somehow in the security domain.
I am curious what your definition of "memory safety" is such that Go satisfies the definition. Wikipedia defines it as
> Memory safety is the state of being protected from various software bugs and security vulnerabilities when dealing with memory access, such as buffer overflows and dangling pointers.
My example shows that Go does not enforce memory safety according to that definition -- and not through some sort of oversight or accident, but by design. Out-of-bounds reads and writes are possible in Go. The example might be contrived, but the entire point of memory safety guarantees is that it doesn't matter how contrived the code is.
I'm completely fine with Go making that choice, but I am not fine with Go then claiming to be memory safe in the same sense that Java or Rust are, when it is demonstrably not the case.
While you're wondering why I keep claiming Go is a memory-safe language, you can also go ask the ISRG, which says the same thing I am at (checks notes) https://www.memorysafety.org/.
And yet Go violates the definition they give -- it doesn't prevent out-of-bounds accesses. (And just to be sure we're talking about the same thing, I'm specifically talking about Go here. All the other languages on their list are actually memory safe, as far as I know.)
> you have to demonstrate a plausible scenario in realistic code where an attacker controls both the value and the address it's written to.
So your definition of memory safety includes some notion of "plausible" and "realistic"? Neither https://www.memorysafety.org/docs/memory-safety/ nor Wikipedia have such a qualification in their definition. It would help if you could just spell out your definition in full, rather than having us guess.
I am honestly curious here. I am a PLT researcher so I am in a bubble where people use the term consistently with how I use it. You are the first person I meet (for some notion of "meet" ;) that uses the term differently. But without external sources it's hard to judge how wide-spread your definition (that you still haven't spelled out...) is.
I think we actually agree on all of the factual points here, we just don't agree on how languages should be categorized/labeled according to their guarantees in both a theoretical and a practical sense, and that's largely a subjective matter anyway. So, happy to agree to disagree here.
U.S. and International Partners Issue Recommendations to Secure Software Products Through Memory Safety
They recommand Go among other language in their paper.
https://media.defense.gov/2023/Dec/06/2003352724/-1/-1/0/THE...
> Memory safety is a property of some programming languages that prevents programmers from introducing certain types of bugs related to how memory is used. Since memory safety bugs are often security issues, memory safe languages are more secure than languages that are not memory safe.
That is the definition they give. Since Go does not "prevent programmers from introducing certain types of bugs related to how memory is used." it does not fall under this definition. They can list go as memory safe, but then either they disagree with their own definition or made the mistake of adding Go to that list. Memory safety is not a spectrum. You are either memory safe or unsafe. The spectrum is in the unsafety. Go is obviously less unsafe than C for example.
if there is one takeaway from this discussion, i think it must be that memory safety does not have any single, commonly-accepted, or objective definition -- and that it is pretty obviously a spectrum, not a boolean
"Memory safety denotes the degree to which a programming language guides and protects developers from memory‑related errors—ranging from minimal, manual checks to comprehensive static and runtime enforcement—through mechanisms like strong typing, ownership or borrow checking, and garbage collection."
And then also include modern C++ in their lists. because by all accounts it is memory safe by that definition.
This is a strawman argument, you're arguing semantics here. You're a smart person, so you know exactly what he means. The perception created by your article is that people shouldn't use Go because it's not memory-safe. But the average developer hearing "not memory-safe" thinks of C/C++ level issues, with RCEs everywhere.
Unless you can show a realistic way this could be exploited for RCE in actual programs, you're just making noise. Further down the thread, you admit yourself that you're in a PLT research bubble and it shows.
Seriously, why are we bashing a researcher for being academic? This makes no fucking sense. Nobody claimed anywhere that people should stop using Go.
Yes, semantics — what do things mean, exactly? — is the subject of the discussion here and is actually quite important in general.
Uh, where exactly am I saying or implying that? I am, in fact, saying that Go is much closer to memory-safe languages than to C, safety-wise.
But I am arguing that the term "memory safe" should only be used for languages that actually went through the effort of thinking this problem through to the end and plugging all the holes through which memory safety violates can sneak in. Go is 99% there, but it's falling slightly short of the goal. I think that's a useful distinction, and I am disappointed that it is regularly swept under the rug, which is why I wrote this blog post. You are free to disagree, I never expected to convince everyone. But I think I gave some people some new food for thought, and that's all I can hope for.
Everybody does not understand that otherwise there would be zero of these issues in shipping code.
This is the problem with the C++ crowd hoping to save their language. Maybe they'll finally figure out some --disallow-all-ub-and-be-memory-safe-and-thread-safe flag but at the moment it's still insanely trivial to make a mistake and return a reference to some value on the stack or any number of other issues.
The answer can not be "just write flawless code and you'll never have these issues" but at the moment that's all C++, and Go, from this article has.
- in other languages, it’s understood that perhaps the language is vulnerable to certain errors and one should attempt to mitigate them. But more importantly, those errors are one class of bug and bugs can happen. Set up infra to detect and recover.
- in Rust the code must be safe, must be written in a certain way, must be proven correct to the largest extent possible at compile time.
This leads to the very serious, solemn attitude typical of Rust developers. But the reality is that most people just don’t care that much about a particular type of error as opposed to other errors.
> in Rust the code must be safe, must be written in a certain way, must be proven correct to the largest extent possible at compile time.
Only for the Safe Rust subset. Rust has the 'unsafe' keyword that shows exactly where the former case does apply. (And even then, only for possible memory unsoundness. Rust does not attempt to fix all possible errors.)
Woe to those that use unsafe, for the scorn of the community shall be upon them. :)
Opposite really. I like rust because I can be care free and have fun.
I had to convince Go people that you can segfault with Go. Or you mean the language designers with using everybody?
Of course unsafe Rust is not memory safe. That's why it is called like that. :) Go has unsafe operations too (https://go.dev/ref/spec#Package_unsafe), and of course if you use those all bets are off. But as you will notice, my example doesn't use those operations.
Hence the focus on fearless concurrency or other small-scale idioms like match in an attempt to present Rust as an overall better language compared to other safe languages like Go, which is proving to be a solid competitor and is much easier to learn and understand.
~130k LoC Swift app was converted from 5 -> 6 for us in about 3 days.
Swift is strating to look more like old java beans. (if you are old enough to remember this, most swift developers are too young). Doing some of the same mistakes.
Anways https://forums.swift.org/t/has-swifts-concurrency-model-gone... Common problems all devs face: https://www.massicotte.org/problematic-patterns
Anyways, they are trying to reinvent 'safe concurrency' while almost throwing the baby with the bathwater, and making swift even more complex and harder to get into.
There is ways to go. For simple apps, the new concurrency is easy to adopt. But for anything that is less than trivial, it becomes a lot of work, to the point that it might not make it worth it.
A much bigger problem I think are the way concurrency settings are provided via flags. It's no longer possible to know what a piece of code does without knowing the exact build settings. For example, depending on Xcode project flags, a snippet may always run on the main loop, or not at all or on a dedicated actor all together.
A piece of code in a library (SPM) can build just fine in one project but fail to build in another project due to concurrency settings. The amount of overhead makes this very much unusable in a production / high pressure environment.
Rust avoids all this entirely, by using its type system.
In practice, Java programs tend to pick up on data races very quickly because they mutate some collection and the collections framework has safety checks for this.
Is it? It will depend on the code, but my gut feeling is that you typically would get a few (if not lot of) unnoticed non-segfaulting issues before you get the segfaulting one that tells you straight in your face that you have a problem.
On 64-bit systems, and even then, it depends on the system’s memory layout (I think most integer values in programs are < 2³²)
Inconsistent data is pretty bad, but it's not as bad as memory corruption.
In C# For example, if a structure is over CPU arch Word size (i.e. 32 or 64 bits) then you could have a torn read if it's being written. However object refs themselves are always word size so you'll never have a torn pointer read on those.
However, in either case there is still a need in multithreaded environments to remember the CPU's memory ordering rules and put proper fences (or, to be safe, locks, since memory barrier rules are different between ARM and x86 for example).
But that second bit is a fairly hard problem to solve for without having the right type of modelling around your compiler.
Rust on the other hand solves that. There is code you can't write easily in Rust, but just yesterday I took a rust iteration, changed 'iter()' to 'par_iter()', and given it compiled I had high confidence it was going to work (which it did).
Not synchronizing writes on most data structure does not create a SEGFAULT, you have to be in a very specific condition to create one, those conditions are extremely rares and un-usual ( from the programmer perspective).
In OP blog to triggers one he's doing one of those condition in an infinite loop.
Or put another way what is the likelihood that a go program is memory unsafe?
Not going down the same road is the only reason it didn't end up on the pile of obscure languages nobody uses.
Case in point, Limbo and Oberon-2, the languages that influenced its design, and authors were involved with.
Dart ended up on the pile of languages nobody uses. And Carbon? What's Carbon? Exactly!
> Case in point, Limbo and Oberon-2, the languages that influenced its design
Agreed. Limbo and Oberon-2, as primitive as they may look now, had the kitchen sinks of their time. Why wouldn't they have ended up on the pile of languages nobody uses?
Dart was a victim of internal politics between the Chrome team, Dart team, AdWords moving away from GWT wanting AngularDart (see Angular documentary), and the Web in general.
Had Chrome team kept pushing DartVM, it might have been quite different story.
Carbon, good example of failure to actually know what the team purposes are. It is officially a research project for Google themselves, where the team is the first to advise using Rust or another MSL.
One just needs to actually spend like a couple of minutes on their wiki, but I guess that is asking too much on modern times.
Limbo and Oberon-2 were definitely not kitchen sinks of their time, their failure was that neither Bell Labs in 1996, nor ETHZ in 1992, were that relevant for the programming language community in the industry.
Trouble with that line of thinking is that Google never pushed Go either. It didn't even bother to use it internally (outside from the occasional side project here and there). Google paid some salaries. I'll give you that. But it has paid salaries for a lot of different languages. That is not some kind of secret sauce.
> It is officially a research project for Google themselves
It's not just a research project. It is officially "not ready for use", but its roadmap has a clear "ready for use" plan in the coming months. Rust was also "not ready for use" when it hit the streets, it officially being a Mozilla research project, but every second discussion on HN was about it and what is to come. And that was without Google backing. If what you say is true, why isn't Carbon being shouted from every rooftop right now?
I know you're struggling to grasp at straws here, but let's just be honest for a moment: If it hasn't caught attention already, it isn't going to. Just another language to add to the pile.
It is officially "not ready to use", it isn't a strawman as people keep complaining about nothing.
At one time, Go maps were not thread-safe. Was that fixed?
sync.Map was added, but isn't intended to be a general purpose map.
——
The Map type is specialized. Most code should use a plain Go map instead, with separate locking or coordination, for better type safety and to make it easier to maintain other invariants along with the map content.
The Map type is optimized for two common use cases: (1) when the entry for a given key is only ever written once but read many times, as in caches that only grow, or (2) when multiple goroutines read, write, and overwrite entries for disjoint sets of keys. In these two cases, use of a Map may significantly reduce lock contention compared to a Go map paired with a separate Mutex or RWMutex.
This is a killer combination for any team looking to write code for auto-scalable microservices, to run for example on Kubernetes. Java is not great in this niche because of its slow startup time, relatively large memory overhead, and the need for warm-up before code actually starts executing fast (so scaling up and down has a very large cost for Java services). .NET has similar problems, and also a huge container size. Python is far too slow, and not typed. TypeScript is single threaded, and still has a pretty hefty runtime. OCaml doesn't have any large org behind it, is quite obscure syntax, and was still single-threaded at the time Kubernetes started. Haskell has similar issues, and is also large and slow starting. Rust, C++, C all require manual memory management.
So, it's no surprise that Go was used for Kubernetes services themselves, and it's no surprise that people designing for Kubernetes mostly chose to write their new stuff in Go. Go the language, with its antiquated design, is actually quite secondary to all of that. But Go's runtime is completely unmatched in this space.
> .NET has similar problems
s/has/had/
https://blog.washi.dev/posts/tinysharp/
The issue is that some people still fighting against the concepts ML family languages (primarily SML) introduced. Go implemented go routines and channels from CSP (https://en.wikipedia.org/wiki/Communicating_sequential_proce...) but dragged a lot on influence from C (understandable) into the language.
I think Rust opted for the best combinations (some CSP, a lot of ML and a bit of C++).
The article you quote is a toy example - if you write a C# or F# web API server, you'll see that it takes up way more space than a Go one with similar functionality (and has way higher memory overhead as well). A Go API web server is maybe 10MB on disk, with no dependencies (that is, you can run it perfectly in a container that is defined as `FROM scratch; COPY my-go-exec /my-go-exec `). The equivalent Java or .NET container is somewhere around 2-400MB at the minimum.
As for the syntax and constructs, I don't care so much. If OCaml or SML had comparable support and a comparable ecosystem to Go, I'd bet plenty of people would have chosen them instead.
Native AOT compilation didn't exist for .NET then, and .NET was, for all intents and purposes, Windows-only.
(Last I checked, the AOT story for C# is not that great even today. No idea about F#.)
To be fair, everyone was fighting against ML concepts at the time. Ruby on Rails was "in" and "doing situps" was "out". Go was built for the time it was created. It was, quite explicitly as told at its launch announcement, made to be a "dynamically typed" language with static type performance. It is unlikely it would have had a static type system at all if they knew how to achieve the same performance optimizations without a type system.
> I think Rust opted for the best combinations
But built in another time. Ruby on Rails was "out" and static typing (ML style in particular) was "in" by the time Rust finally got around to showing up to the party. Looking back, it may not seem like there was much time between the creation of Go and the creation of Rust, but on the tech scale it was created eons later. The fashion of tech can change on a dime — as captured in the humorous fable about JS having a new "must-use" framework every week.
The fashion trends will change again at some point. They always do.
Go is not particularly fast. People often see that Java gets faster as it runs and thinks, oh, it must be slow at the start then. But when you compare like with like, Go ends up being stuck at the bottom of the curve that Java then sharply climbs. The difference in GC quality is big, or at least, used to be? The only place where you really notice the difference is command line tools, and Java has GraalVM for that.
There are some kubernetes services that scale up and down. And even for those that don't normally, if they have some kind of failure, the difference between taking a millisecond to get back up and taking a second can actually matter for a web host.
> Go is not particularly fast. People often see that Java gets faster as it runs and thinks, oh, it must be slow at the start then. But when you compare like with like, Go ends up being stuck at the bottom of the curve that Java then sharply climbs.
Go starts up much faster than Java. And Go code runs measurably faster than interpreted Java code, even though it's slower than the JITed code you'll eventually have if your JVM runs long enpigh. But un-JITed Java code is very slow, more comparable to Python than JITed Java or with Go . This has nothing to do with the GC - where I do agree Go is mediocre at best.
The fact that it can do this mostly concurrently (it's not fully concurrent, actually) is not that special - Java's latest GC is also mostly concurrent, without having any of the other previous drawbacks.
Go's big advantage in terms of memory management is that it doesn't produce as much garbage as most GC languages. The fact that it natively supports value types, and actually uses them extensively in the standard library (unlike C#, which also supports these), and it's excellent stack escape analysis which allows many allocations to stay on the stack and outside the GC heap entirely, is what makes Go have such little memory overhead.
I personally appreciate Go as a research experiment. Plenty of very interesting ideas, just as, for instance, Haskell. I don't particularly like it as a development language, but I can understand why some people do.
Is there? When you get down to it, it is really just a faster Python. Which is exactly what it was said to be when it was released. Their goal was to create a "dynamically-typed" language that was more performant. It is likely that it wouldn't have had a static type system at all if they figured out how to achieve on the performance end without needing types.
You can tell who is clueless when you hear someone say its type system is lacking. I mean, technically it is, but it is supposed to be. Like saying Javascript or Ruby's type system is lacking.
- using zero values as an optimization mechanism;
- (non-)pointers and passing self by copy.
I mean, I hate both mechanisms, but intellectually, I find them quite interesting.
Also, I'd not classify it as a faster Python. It's more of a cousin of Obj-C if the authors of Obj-C had fallen in love of Erlang instead of Smalltalk.
I will grant you that Carbon is still in its infancy, but when Rust was in the same youthful stage we never heard an end to all the people playing with it. You, even if not tried it yourself, definitely knew about it.
You've made up a fun idea, but reality doesn't support it. Google has not shown its weight carries anything. They have really struggled to get any for-profit business units off the ground since they gained the weight, never mind their hobbies! If anything, Google is detrimental to a project.
If only Google put their weight into a watch, maybe you'd have one?
Oh wait. They did! Google can't successfully turn their weight into much of anything. Go's success, if we can call it that, clearly happened in spite of Google.
Like who? Outside of Go itself, which is really more of a community project — albeit with the chief maintainers still on Google's payroll, almost nothing at Google is written in Go. In fact, Pike once gave a talk reflecting on why it didn't succeed in that space, and noted that it was the "Python and Ruby programmers" who actually ended up adopting it.
Google makes money selling services (i.e. Google Cloud) that run Kubernetes, Docker, etc. If it weren't for that, it is unlikely that Google would even be continuing to maintain it at this point. It was an interesting experiment, perhaps, but ultimately a failure within Google. As before, it was the Python and (probably most especially) Ruby communities that ended up leaning into it.
Which isn't surprising in hindsight. Go offered those who were using Python and Ruby a language that was in the same kind of vein, while solving many of the pain points they were experiencing with Python and Ruby (awful deployment strategies, terrible concurrency stories, trouble with performance, etc.) These developers were never going to use Haskell. They wanted Ruby with less problems.
And that's what Go gave them — at least to the extent of being better than any other attempt to do the same. Since it solved real problems people had, without forcing them into new programming paradigms, it was adopted. Choosing a technology based on starry-eyed fandom and arbitrary feelings might be how you go about navigating this world, but that doesn't extrapolate.
This got to be a joke right. The only thing I hear is at Google no one likes Go. Most software is in C++, Rust, Java or Kotlin.
Lets also not forget Rob Pike famous quote regarding simple minds, as target audience.
As for Go, Kubernetes made it unavoidable, it is like UNIX for C, Web for JavaScript, and so forth.
While you fairly point out that many fall into Python because they learned about it in school and never bother to look beyond, Go has had no such equivalent. For you to choose it, you have to actively seek it out, where you are going to also see all the other programming languages you could also choose.
> As for Go, Kubernetes made it unavoidable, it is like UNIX for C, Web for JavaScript, and so forth.
UNIX's programming interface is a set of C functions. You are right that C is the path of least resistance to use it.
The web's programming interface is Javascript. You are right that Javascript is the path of least resistance to use it.
Kubernetes' programming interface is a "REST API" – or your program running inside a container, if you want to look at it from the other direction. In what way is Go the path of least resistance to use it?
TFA's point is that (safe) Rust is also like that, but achieves it by restricting all cases where a torn write could be observed through its type system instead of VM's memory model.
Java has a very specific memory model, so the behavior of variables across threads is quite well defined. Basic variables can tear however (a 64bit long on a 32bit architecture) without the volatile keyword and that is quite different than rust.
What Rust prevents is very specific.
Usually, but not always! https://jcdav.is/2015/10/06/SIGSEGV-as-control-flow/
Is guaranteed that every offset you can try to read is guaranteed to create a segfault?
The offset is fixed as part of the compiled code; the JVM can enforce that it's less than 4k (otherwise it can use an explicit NULL check), and that the first 4k page is always unmapped.
Memory safety is just the source of bugs that we've figured out how to eliminate. It's a significant source of really bad (hard to debug due to action at a distance, high impact, etc) bugs so that's worth a lot, but it's not perfect. And even then we have a more frequently used escape hatch to the memory-unsafe world than would be ideal from a safety perspective for practical reasons.
A more complete version of safety would be achieved with a language that proves code correct to arbitrary specifications. We aren't there yet for there being such a language that is practical for every day use. Personally I'm increasingly optimistic we'll get there sooner rather than later (say, within 20 years). Even then there will probably be specification level bugs that prevent a claim of complete safety...
Do you have any good examples? Not trying to argue, just genuinely curious as someone who hasn't been in this field for decades.
It was designed with contempt for developers, for example disallowing developers to create generic data structures, or lacking a decent way of error checking that is not extremely error prone and verbose.