Go 1.17 Release Notes
golang.org
golang.org
> Go 1.17 implements a new way of passing function arguments and results using registers instead of the stack. Benchmarks for a representative set of Go packages and programs show performance improvements of about 5%, and a typical reduction in binary size of about 2%. This is currently enabled for Linux, macOS, and Windows on the 64-bit x86 architecture (the linux/amd64, darwin/amd64, and windows/amd64 ports).
I love how they're doing it in such an iterative fashion: even assembly functions don't have to be rewritten. Then again, I guess doing it progressively like this is the only feasible way to avoid reworking all the low-level assembly routines in one fell swoop.
$ time goawk_go1.16 'BEGIN { for (i=0; i<100000000; i++) s += i; print(s) }'
4999999950000000
real 0m10.158s ...
$ time goawk_go1.17 'BEGIN { for (i=0; i<100000000; i++) s += i; print(s) }'
4999999950000000
real 0m6.268s ...
I use VSCode a lot, and in VSCode you can press "F12" to jump to the location of a function definition. In Go I find myself deep diving into github libraries all the time just because F12 takes me there. That never happened with C/C++ - the tooling was just too disparate and fragile.
My biggest complaint with Go is lack of generics. I mean technically you can achieve generics with an unholy empty interface/reflection incantation but it's a huge pain compared to C++.
Truly, it is night and day. I can get Real code intelligence on almost anything, including large C projects with custom Makefiles that compile cross-arch like Wine. It only ever has trouble with unusual translation unit patterns. (You can see this in Higan for example, which has .cpp files including other .cpp files; clangd seemingly fails to put them in the context of the intended translation unit.)
It’s not as good as Go, but it is life-changing for me.
It's not like Go's tooling is that much better than the best modern C++ tools, but the ecosystem is so much more unified by having 1 blessed set of tooling. Rather than 20 different package managers/(meta)build systems. No need to worry that I am using build system X, but I want to use a library using build system Y.
(The one and only case I've ever found it doesn't work is to find references to a function that's only called via ADL in a widely used template that's instantiated with thousands of different types.)
* Error handling is error prone and awkward.
* No proper enums
* No sum types/pattern matching
* Null pointers exist
* Its interfaces are very awkward. There are superior implementations of what they tried to do in other languages.
* Subpar IDE experience, made worse by how interfaces are implemented. You need an IDE for any non-trivial project.
* Very awkward choice to have visibility be determined by the case of the first letter in the function or variable. Simple changing of visibility to public will require larger diffs if it's used a lot.
* No private visibility, only package private.
* Very crude import mechanism. Try to rename a package and see how the IDE fails to figure out what to change.
Wouldn't any visibility change require a large diff in all languages?
But if you're coding C/C++ in a "proper" IDE like Visual Studio, CLion or Xcode, such features are expected to work out of the box. IME, Intellisense-like features are usually only a pain to setup in "non-integated" development environments like Vim or other text editors.
After a while I actually understood why Go is such a successful language / ecosystem:
Go's priority is to make projects easier. It is doing so by all it's smaller and larger features respectivly skipped features. But one soon understands the big picture of the Go team. Go is designed by very experienced devs who knew what is important and what not.
In my Go projects, I don't have to worry about:
- Memory safety
- Tooling
- Performance
- Structure (once I understood the package desgin philosophie)
- Difficult syntax
- Concurrency
- Libraries, as we can do most with the standard lib
- Maturenes and stability
Instead I focus on the things that count:
- Solve the problem at hand
- Create correct, stable and maintable software
And as this was not enough, the Go team comes around the corner with an 5 % average performace gift.
Awesome.
It's enough in 95% of cases and it works every time in every environment. In the time it takes me to get a debugger running and attached to a process, I've print-debugged it and fixed it already =)
In any modern editor or IDE, getting a debugger running and attached to a process usually amounts to opening a folder, adding a breakpoint and pressing F5
I tend to reach for a debugger when doing post mortem debugging or when I work with C where that's usually the easiest way to get a stack trace when a program crashes.
That said, time travel / reverse debuggers can be quite useful indeed: see http://choly.ca/post/debugging-go-with-rr/
You have no idea how amazing that feels.
And even if you take Win16 out of the picture, Win32 exists since 1995.
Borland C++ or Watcom, probably not much help.
Delphi, C++ Builder, Visual C++ for Windows 95/2000 mostly work, naturally you might need to use the IDE wizards to upgrade project files, or fix tooling paths on the makefiles.
The problem is not JS, more the dependencies you decided to take for your project.
By 3.6 it was pretty good though and even migrations and dual code bases were easier because they went back and allowed things like u"" for unicode strings to continue to mean unicode strings in python 3 - before that they'd actually blocked unicode in 3 that was working well in 2.x code bases! It was a total in your face move to break the ecosystem even though supporting u"" would have been almost no cost in terms of improving compatibility with existing code. ASCII handling (important for things like web tech stacks header handling etc) also improved thankfully. 3.x made things a lot harder initially (and slower too).
Also a lot of code doesn't look the same over time - the conventions (formatting) are either very poorly defined or change. Go seems to have pretty much one format from fmt
etc
The fact that they went out of their way to break python 2 unicode when running on python 3 was just totally nuts. Especially after making such a big deal about unicode!
I've never seen anything like it I don't think? Maybe the new Perl that never really landed?
Imo it's infinitely worse than that.
The big deal about Unicode is its nature, as defined in the "Summary Narrative" from 1991[0]. To wit:
> The Unicode character encoding derives its name from three main goals:
* universal (addressing the needs of world languages)
* uniform (fixed-width codes for efficient access), and
* unique (bit sequence has only one interpretation into character codes)
The Unicode folk realized that it would take decades to shift developers worldwide to doing that properly, so they adopted a three stage plan for software (eg the string types of programming languages) to get from where things were, to where they needed to be:
* Stage #1: Character = byte
* Stage #2: Character = code point
* Stage #3: Character = what a user thinks of as a character[1]
Python 1 was a Stage #1 language -- Character = byte -- like most others of its time.
In Python 2 there were tweaks to try move toward Stage #2 -- Character = code point, again, like most other PLs of its time.
In Python 3, they dictated a full switch to Stage #2 --- Character = code point. That was an unnecessarily painful break relative to Python 2. But -- and this is what really matters -- they entirely ignored Stage #3, which is the whole point of Unicode in the final analysis.
For some reason this was not good enough for the python 3 folks - they actively broke this code which was from folks who had SPECIFICALLy addressed unicode in their apps.
And yes, they could have supported u"" (and a number of other things).
They went - unicode is so critical we will break the world, and then for folks who had already supported unicode well or wanted to dual target a library they said your u"" approach to unicode is so bad we will break it.
Total BS in my book.
I challenge anyone to upgrade a project from Angular ver 5 to the latest. It's possible but you'll also question whether its possible for you to quit software engineering and learn woodworking.
The more Go code I write, the more I like the language, the tools, and the standard library.
Low-friction, high-quality software development.
If only I could use it professionally, instead of C++.
When you have a team of varying skill levels who need to deliver shit that works on a schedule and still have some kind of performance, you use Go.
Even more so if you need to hand off the maintenance to a team of randoms from god knows where and THEY need to be able to keep it running and add new features.
It doesn't have the fanciest new EXPRESSIVE operators or anything exciting really. It just does the job, albeit verbosely.
You can do fancy shit on your own time.
It's verbose as all hell, but on the other hand you'll notice immediately if it's missing.
Go is not a "fun" language by any means, but that doesn't mean it's not a productive one.
You instantly notice it because the if err != nil -template is missing after it =)
https://github.com/golang/go/issues/20803
In contrast, e.g. Zig requires you to use _ if you want to ignore.
I don’t know what to think about the upcoming generics. It feels late to make such a big change so long after the language has been established. At least a plan for it should have been integrated into the language from the start.
It feels like a missed opportunity - an effort with similar funding but a more sound theoretical foundation than being Newsqueak 3.0 could have become an industry game changer. Instead it fills a niche, which is a success, albeit smaller.
I agree with that. I think looking back on it, Go arrived on to the scene at a perfect time where lots of people were rearchitecting so there was lots of natural appetite in the industry for a new language. Go vacuumed up a lot of this opportunity but ultimately hasn't really evolved the practice of software engineering all that much.
This is recoverable. The Java world was also saddled with a decade of awful pre-generic code, but we burned almost all of it to the ground and started over.
That's a pretty good description of Rust 0.x. Might ring a bell.
Once generics are finally added the language should basically be a no-brainer for any serious production code.
a = append(a[:i], a[i+1:]...)
// or
a = a[:i+copy(a[i:], a[i+1:])]
Is that code just idiomatic, and Go programmers recognize it instantly? Or maybe they don't deal with slices that often?
a = slices.Delete(a, i, i+1)
Compare
strings.ToUpper(strings.Replace(strings.Trim(s), "a", "b")))
s.Trim().Replace("a, "b").ToUpper()It's "to avoid complicating questions about the interface (in the Go type sense) of basic types". It also allows separating the builtin functions, of which there are very few (they have to be in the core language spec), from the stdlib functions like strings.ToUpper, which there are many many more of and are added to more quickly than the builtins.
I didn't get the last point. builtin functions are there (for a big part) because the language doesn't have generics. Functions on types can be added without affecting the interface, since strings don't implement any interface (at least not on purpose, another problem with golang). Java constantly enriches standard library types with more useful methods.
You can restructure it if you don't need to preserve order:
a[i] = a[len(a)-1]
a = a[:len(a)-1]
*: because the Internet is a nitpicky place.
You might like the slices proposal for utility functions like these: https://github.com/golang/go/issues/45955
I'll admit freely that I'm not a proficient Go programmer, so it's easier for me to see the things I don't like than the things that Gophers praise. But even this solution leaves me unsatisfied, in contrast to what I'd reach for in Rust or even C++: it requires that I know that struct{} is zero-sized, and I still have to do the manual legwork of writing a CS101 dedupe function.
Like, if you tell Rust you want 4 million empty tuples in a vector, it will give you a vector with exactly 4 million empty tuples in it and no heap allocation because those empty tuples don't take space, so, 4 million of them also doesn't take up any space.
Because Rust has generics, it is more ergonomic though.
https://doc.rust-lang.org/std/collections/struct.HashSet.htm...
For every complain like that in Go I'll find a hundred in C++, ten in Java / JavaScript / Python.
As to your question, as a full-time Go programmer: this doesn't come up often.
When it does, I google "slice tricks" and copy & paste the formula.
It'll be fixed by next release because generics will enable writing a function that does that and that function will be in standard library.
Why wasn't this fixed before generics? Because the only way to do it would be via a compiler built-in function and Go team is rightfully reluctant to bloat the language / compiler for minor functionality like that.
It seems strange to not provide slice.erase as a primitive operation, and instead to recommend implementing it in terms of append. Usually removing from a vector does not require allocation. Still it may be for the best with upcoming generics, fewer builtins is good.
On the second glance, one can see why probably it was left out: there are two fundamental different ways to implement the deletion. You can copy the last element into the slot of the deleted element, or you can move all elements after the deleted one place to the left. The latter is what you implemented and preserves the ordering, which often is desirable. The former is way more efficient, if you don't need to preserve the order. A default implementation would probably choose to preserve the order and thus introduce a systematic inefficiency which is easy to be avoided.
Fortunately, with the introduction of generics and especially the new slice package, this discussion becomes moot.
For now you just have to accept these quirks as part of the tradeoff of working with the language. I don't find it that big of a deal esp with the solution on the horizon.
Have you ever had to remove elements from a list/slice?
I didn't completely expect it, but I now find myself reaching for Go first when writing something like a small script or utility that I'd previously have written in something like Ruby or Python. It all feels much more solid.
That said… I really chafe against the anaemic type system when writing anything a larger than that. I hate not being able to communicate things like "this can never be nil and if it's is then it's an error" to the compiler. I resent having to write three lines of boilerplate every time I want to reverse a flipping 10-item slice, and the design decisions leading to the existence of `time.IsZero()` are bordering on criminal.
I don't know if I'm the odd one out for feeling like that – parts of the language and ecosystem are just absolutely wonderful to work with, but the unergonomic sharp edges bother me so much that I end up finding it really _annoying_ to write a lot of it.
Up above I talk about gopls's postfix completions, and reversing a slice is one: https://github.com/golang/tools/releases/tag/gopls%2Fv0.7.0
What's the rationale for Time.IsZero()? Seems like every use case for Time.IsZero() I think of is a code smell. The 1970 epoch is an implementation detail that should be hidden or configurable. People probably use IsZero() as a sentinel value to indicate "undefined time", even though the 1970 epoch is a valid point in time.
Go has a notion of "zero value". When you don't assign a value explicitly it'll be set by the compiler to "zero value" of that type.
This is much better than C/C++ of "random value".
For primitive types, the compiler decides what "zero value" is. For structs, each component is set to its zero value.
For good reasons (language simplicity) Go doesn't allow the user to declare what the zero value is (something that you can do in e.g. C++ via a constructor).
Time is a struct. Its zero value is the same as for any other struct and not meaningful time value.
It's a pragmatic necessity to be able to query "is this time value an unset time value?". A pragmatic solution for this need is to provide IsZero() method on Time struct.
// IsZero reports whether t represents the zero time instant,
// January 1, year 1, 00:00:00 UTC.
func (t Time) IsZero() bool {
return t.sec() == 0 && t.nsec() == 0
}Go 1.17 Beta - https://news.ycombinator.com/item?id=27462884 - June 2021 (118 comments)
Go 1.17 is deprecating the traditional use of 'go get' - https://news.ycombinator.com/item?id=27630625 - June 2021 (215 comments)
Very good for the ecosystem. Nudge folks to upgrade from broken/insecure versions
This is a very welcome change and make the go.mod much more obvious to understand. Hooray!
[1] https://groups.google.com/g/golang-dev/c/U7eW9i0cqmo/m/ffs0t...
Now add pattern matching please ! Well done to everyone that worked on Go ! It's still a pleasure to use.
The way I see it, both Go and Java are crawling towards the same ideal: a compiled, statically typed, garbage collected, modern enough program language targeted for a majority of back-end concerns. Go has to close the gap mostly just with generics; and Java with memory layout (Valhalla), concurrency (Loom), and ergonomics (Amber); but they will both be there in 3-5 years.
Take this with a grain of salt if you like, but while I probably find it more fun to program in Go, I don't think there's a compelling reason to switch a large codebase from Java to Go. It often feels like my company is needlessly divided by programming languages even though they solve fairly similar problems. It's one thing to move off of a dying language, but Java is certainly not that.
Not so much with Rust.
There is a huge class of problems that don't need the precision (especially around memory) that Rust demands from the programmer.
Otherwise try a "go mod init" in the root of your project.
Yay!
It doesn't encourage you to do fancy language tricks, it kindly directs you to get your shit done, compiled and delivered.
Frankly, people are making boatloads of money off of software written in Python and JS--languages which are both far less safe than Go and far slower and yet Go is on par with those languages with respect to productivity (I argue it's more productive due to its static typing features, but others argue it's less productive presumably because of the learning curve). Most software doesn't need to be so fast or correct.
Also, I work in distributed systems and SaaS (software as a service). In this world, most errors are silly type errors ("undefined is not a function", forgetting to `await` some async function, etc) that would be caught with a flat-footed type system like Go's. Beyond that, the next largest bucket are issues that even a sophisticated type system like Rust's wouldn't help with, such as infrastructure issues (missing/misconfiguration of some cloud permission, networking rule, etc), misconfiguration of the service, race condition with many processes accessing the same networked resource, deployment error, etc. This means that Rust's stated advantages aren't really as nice as they initially sound.
As a programmer, I can appreciate a super fast language with a strict type system; however, as an engineer or a technologist, Rust is rarely a good fit for me. Go seems to be a lot closer to the sweet spot, which makes sense because it was developed expressly with distributed systems in mind.
I can’t speak to python, but typescript has basically fixed this problem overnight in the javascript ecosystem. The typescript compiler finds almost all small bugs like this while I’m coding. And as an added bonus, type hints allow the IDE to be much more helpful - adding to jump to function support, autocomplete, method parameter suggestions (or documentation on hover). And typescript is easier to read than javascript because you don’t have to guess what data type some variable is.
I love javascript’s quick and dirty nature, but I still use typescript instead of javascript now for any code I write that I expect to survive the week. (And as a bonus: you still get IDE type hints when calling typescript functions from raw javascript!)
Typescript’s type system is also more powerful than Go’s. It supports enums, genetics and type unions (eg x: string | number).
Go sits in the awkward place of having a worse type system and no significant advantages over typescript for me. It’s awkward to use go on the web. Go is faster than javascript on the server but that usually doesn’t matter. And when it does I can reach for C or Rust. Both of which work really well with native code, or with JS through wasm.
Rust is much harder to learn than Go - but personally I’ve climbed that hill already. Once you’re over that hill, Rust is much more expressive. On purpose both ways - go isn’t trying to be expressive, and rust is. I love rust’s parametric enums and output types in traits. Which I now sorely miss in other languages.
I can imagine Go shining a lot more brightly when working with a team which has mixed skill levels. Most of my work lately has been solo, so I don’t need gofmt to enforce a consistent code style, or anything like that. I do miss Go’s green threads. Rust’s afterthought scattergun approach with threads and futures feels like a mess. But I can’t see myself ever really using Go. It’s weak in areas I want it to be strong (eg the type system). And strong in areas I just don’t care much about. (Eg consistency).
Rust can do some things Go simply isn't suited for right now; for instance, there is no way we're getting kernel Go in the foreseeable future. You couldn't reasonably build a browser, or browser components, in Go.
Both languages have their place. There is a lot of overlap, and the original comment about how one might prefer Rust if what they miss most in Go is generics makes sense. But the idea that the only thing you'd gain by choosing Go over Rust is easier development on big teams is just false. One language is GC'd, the other is borrow-checked. GC makes a lot of things faster to build. This debate ended in the mid-1990s.
Much harder, not really. Rust lets you add mutability, reference counting and thread safety to your data structures, you just need to know when to use these features. Yes, Go has fully general GC but very few problem domains have a real need for general GC.
No, no one needs a GC, but many problem domains need productivity, and GC is the most productive way to manage memory to date. As previously mentioned, Rust has made truly impressive strides in improving productivity of borrow-checking, but it still remains quite far behind GC.
enum Node<E: EntryTraits, I: TreeIndex<E>> {
Internal(Pin<Box<NodeInternal<E, I>>>),
Leaf(Pin<Box<NodeLeaf<E, I>>>),
}
I've been working on this code for months and I still have no idea if my internal b-tree functions should be taking a &mut self, self: Pin<&mut Self> or a NonNull<...> or something else. The compiled code is blazing fast, and being able to control behaviour so clearly with a few parametric type parameters is amazing.
But the process of figuring out the best way to code it up is awful, and it requires all of my attention and capacity. I doubt Rust will ever gain the sort of mainstream usage that javascript & Go have because of how complicated otherwise simple problems can become. Its a great language for the linux kernel and web browsers. But I can't imagine many normal programmers will want to build regular websites and apps in rust.
GC languages are slower, but credit where its due - they make it so much easier to just dive in and spend all your braincells thinking about your problem domain.
Just like consumers don't care if their favourite music application is written in Electron, as long it plays their favourite album on their newly acquired speakers, they also don't care what type system was used to deliver such experience.
With Go finally getting generics, it will be pretty alright, 99% of distributed computing applications need not care about zero GC code, only proper use of value types.
If you want to make a data structure holding the equivalent of a parametric enum (or tagged union from C), go is very awkward to use compared with richer languages like Swift or Rust. Go is also awkward if you want to implement custom generic data structures.
Eg, this[1] code I wrote a couple years ago for doing text based operational transform became about 1.5x longer in Go compared to rust or typescript because its so awkward to express a parametric enum in go. And it was much harder to read & more buggy as a result. Sadly I lost the go version of the code. I'd be curious if someone with more go experience could do a better job, but I'm skeptical.
Hopefully the situation improves somewhat when generics land.
[1] https://github.com/josephg/textot.rs/blob/03c84b7c35a375ba7d...
Thank you for expressing that thought. However: I came quite to the opposite conclusion. I´m currently working in a small team with members that are not so versatile in coding, coming from a JavaScript background. It has been surprisingly easy to teach them Spring Boot. You follow a typical layout and everything falls into its place. I have my reservation that the same holds true for Go. It feels like there is much more room for confusion. Do I use global functions or receivers? How do I inject my dependencies and where do manage them? How do I build constructors and do I really need them? I found surprisingly little advice on those topics.
The borrow checker and GC aren't really doing the same thing though. One very important distinction in some fields (but unimportant in others) is that GC only really cares about memory resources. So you are (or should be) doing explicit manual cleanup for all non-memory resources in a GC language. Rust isn't, in Rust we don't write explicit resource cleanup for a programmable interrupt controller, or a file, or a database connection -- the resource knows how to manage itself, and Rust promises to tell it immediately when it falls out of use, whereas a GC can't promise to ever clean up which is why Go doesn't even bother providing a means to do this automatically for your non-memory resources when they're garbage collected (Java does, but again, no promises it ever fires, so, don't rely on this).
It sounds like you are talking about finalizers? If so, they exist in Go in a similar way to the Java feature [1].
I don't use them for resource cleanup, but I think they probably make sense for library authors as a back-up mechanism (for example, the standard library has a finalizer to close file handles, and the current runtime does GC every two minutes in the absence of memory pressure). You can also explicitly run the GC in Go, and while generally there's a lot of problems with that idea I could maybe imagine it being viable for certain unusual workloads.
- defer (possibly add a go vet check for when missing it)
- when Go 1.18 brings generics, the withFunc pattern from FP where lambdas get given a resource released on function exist, thus having regions/arena like resource management
So what you're doing there is adding a control to try to mitigate the consequences of a language failing. Go doesn't actually prevent you from getting this wrong, but a linter such as go vet can flag cases where it suspects you screwed up, and maybe you'll catch the worst mistakes most of the time this way and by having usage conventions.
We shouldn't mistake this for equivalent capability. The complicated cases that tempt people to ignore or switch off such linting are exactly the cases most likely to have an undetected problem.
So this is "RAII like" only in the sense that "Just remember to do it properly" is RAII like, and we could make exactly the same claim for C.
In the "no breaking changes" case I agree that C++ objects that live in the free store are in the same place as Go, the programmer has a responsible which they may not fulfil, to manage this resource and a linter can only help mitigate this problem.
But plenty of people including Stroustrup want to do lifetime management, despite potential breaking changes from that, and under lifetime management the compiler has visibility into your object lifetimes and can reject programs which inadvertently leak.
Now, that doesn't (can't) make leaks impossible, but it means any leak is now in some sense "on purpose" and would happen for GC'd resources too, it isn't just an accident. For example Rust's mem::forget will prevent the drop happening for the object you're forgetting, but it's not as though you type mem::forget() by mistake. You clearly wanted to achieve that (e.g. you stole the underlying Unix file descriptor from a File and sent it over a socket to a separate process, so now cleaning up that descriptor is the Wrong Thing™) and incorrect usage is not the same category of error as forgetting a with clause in Python.
RAII destructors, on the other hand, do not permit the coder (at the use site of the type) to forget, as the destructor is invoked automatically when the variable goes out of scope.
In my time reviewing Python (also GC, "with" provides a similar functionality), this is a very common error.
Same applies to using on .NET, where forgetting to call using on an IDisposable type can trigger a compiler error, via Roslyn plugins.
Also seem to be unaware how closure based RAII works on FP languages.
RAII isn't triggered on heap allocated objects unless smart pointers are used everywhere.
Python doesn't have a static analysis tool for with, nor does it support value types with stack allocation. Not all GC languages were born equal.
It's triggered unless you have memory leaks.
Similar to forgetting memory in Rust, and then there is no miracle RAII.
Even if it weren't for the performance I would use Rust over Go due to the lack of generics alone, as well as the associated general philosophy of Go that boilerplate is good and one should repeat oneself as much as possible.
It has to do with that one sometimes has to write 80 lines of code in Go to to the same as 3 lines of Rust code due to having to repeat oneself all the time.
It also gets in the way of reading code.
Yes you have cargo flamegraph for profiling locally and you now have pprof-rs to mimick Go's embedded pprof support. But allocation heap profiling is still something I struggle with.
I saw there was a pprof-rs PR with a heap profiler but there was some doubt as to whether it worked correctly; to get a feeling of how that approach would work but without having to fork pprof-rs I implemented the https://github.com/mkmik/heappy crate which I can use to produce memory allocation flamegraphs (using the same "go tool pprof" tooling!) in real code I run and figure out if it works in practice before pushing it upstream.
But stuff you give for granted like figuring out which structure accounts for most used memory, is very hard to achieve. The servo project uses an internal macro that help you trace the object sizes but it's hard to use outside the servo project.
The GC makes some things very easy, and it's not just about programmers not having to care about memory; it's also that the same reference tracing mechanism used to implement GC can be used to cheaply get profiling information.
Have you tried this one? https://github.com/koute/memory-profiler
I tried `cargo install --git https://github.com/koute/bytehound.git`, but that results in "error: multiple packages with binaries found: bytehound-cli, bytehound-gather, interrupt, linking, lz4-compress, simulation".
For the time being I'll stick with heaptrack.
Switch to a dumb allocator and then profile mmap calls or page faults? That should get you large allocations at least. It's a pretty crude proxy. The other allocation profilers I'm aware of cause significant slowdowns.
I currently intercept calls to malloc/calloc/realloc/... and capture stack traces. This way I know how much memory gets allocated for each allocation site. Since allocations usually go through constructor calls, the presence of a constructor in the stack trace can let you infer how many structures of a given type are been allocated. Knowing how big they are is more tricky since allocation for the whole struct and its parts doesn't have to happen entirely in the constructor (some structures like vectors and hashmaps can grow, some structures can collect data from other sources and then hold onto them, etc)
Furthermore to know how the live memory is broken down between object type and allocation sites, you also need to track freed memory. This is significantly more tricky to do efficiently. I currently take an allocation sample every N bytes being allocated and use a poisson process estimator to scale the total allocated bytes.
The only ways I know to account for in use memory is to track every single allocation or to add some extra space for every allocation where we record whether a block had been sampled and of yes, what was its corresponding allocation event.
Can you please elaborate more on your suggestion?
The missing feature I was comparing was the ability of Go to provide good estimated heap and allocation profiles using minimal overhead on a production workload.
If you're writing a compiler, or some other project where extreme high performance is not necessary (e.g., if you're willing to be, say, a factor of 2 slower than C) then Go is a good choice.
Go's performance is excellent, and should be sufficient for all but the most demanding applications. It's not a crummy scripting language like Python/JS.
I think Go is a good replacement for the things people used C and C++ for twenty years ago, but less of a replacement for the things people use C and C++ for today.
Back then, C/C++ was your default "write big server program that needs to go relatively fast" language, and Go is targeting that. But in the meantime, Java got fast enough and hardware got cheap enough that Python, Ruby, and JavaScript have also eaten into that domain.
Today, I see C and C++ used primarily for embedded work and games. I don't see Go as being a great fit for either of those.
1) Get request
2) Serve request
3) Get GC'd
Ie. amortize expensive work over many requests.
It introduces a whole distinct type of complexity into operating the server.
The latency of a request to your server is not just caused by the code that is ran to serve that request (which is how people intuitively would think about it), it can be caused by GC pauses that are happening because of the memory pressure caused by previous requests.
The Go team originally wanted to replace C++, especially inside Google, but they didn't succeed. According to googlers on Hacker News, very few projects inside Google actually use Go.
The reason Go attracted these kinds of developers is precisely that it isn't a systems programming language. Systems people want something like Rust instead.
You don't return to 1992 C++, thinking nah this is fine.
Companies like F-Secure are using Go for real system programming just fine.
The problem is more of mentality and not having management willing to push it down unbelievers no matter what, that prefers to recycle UNIX clones instead to save money.
Midori powered Asian Bing for a while and even then the Windows team did not believe it was possible.
Im glad you said that and I was not just imagining that myself. I remember following the Go language closely in its earlier days and it was often spoken about as a "systems language" but it actually seems to have ended up settling as a language to write servers for people who are sick of OOP but still like their imperative C style code.
- Java has lots of brittle reflection in some libraries and JDK can be finicky, especially with GC tuning
- .NET needs a runtime, historically there's Mono, now there was .NET Core and now there will be just .NET, though in some cases there's also IL2CPP and so on
- Python not only generally runs slow, but also has problematic package management, especially with vent
- JS (Node in particular) has similar package management woes as well as really fast package deprecation
Lots of applications and some tools will get written in Go because of this, because it's pretty reasonable to use in most cases.
In comparison, C, C++, Zig and Rust would all be better suited for systems level programming or embedded development - they typically let you write more performant and less memory hungry code, at the expense of foot guns and slower development.
> .NET needs a runtime
It doesn't. With a single command .NET Core can produce stand-alone single-file cross-platform deployables needing no SDK, Framework, runtime, or other dependency on the server.
> Go at least partially solves some of those problems, by being compiled, having decent performance, somewhat rich ecosystem and passable package management, all while the language remains usable.
It does, and I've been a big fan of Go for a fair few years now. However every point made in that sentence applies equally to .NET too.
The one area where Go beats C# (and most others) hands-down for me is the build time. It's a whole order of magnitude (possibly several) faster than most alternatives.
I'd argue that most systems that won't die out eventually will need a rewrite, or alternatively, in the modern day we'll see more and more polyglotic systems popping up.
Therefore it always makes sense to explore the best options for a particular bit of development, regardless of whether it's Node, Python, Java, Go or something else.
Like building what?
Rust is great for language enthusiats doing hobby projects or for learning. The Rust book is great. Rust has great language features.
Having that said, Rust is the right tool for the job for a very small niche. Basically, if you would have used C++ before and don't need much developer reach or mature libraries. And only for the rare cases you really cannot affort a GC (even though Go's GC is highly optimized).
Go on the other hand, is an industrial strength proven and mature general purpose language. It is the best fit for various kind of networking application. Especially APIs, but also infrastructure where you can live with an GC. CLIs are great with Go as well.
If you're working on a professional grade project (where the GC is acceptable), Go is much superior than Rust in all regards.
Rust is advertised for years and years and didn't have it's breakthrough yet. This empirical fact cannot be ignored. There're reasons for this, of course. Some are:
- Writing Rust consumes so much more mental power with so little gain. That mental energy should be directed to solving the problem.
- Go makes everything besides your problem at hand easy. You can focus on solving your problem, not fighting the Borrow Checker
- Rust's ecosystem is not reliable. Many essential libs are one-man-shows. Version 0.1 everywhere.
- Rust is for and by language enthusiats. If you need to rely on libs for longer than a couple of years it is a hight risk for your project
- In terms of real world performance: Go is so close to Rust that there're very very very few use cases that really need that marginal gain