typical182

It's been discussed for a long time, and the related proposals were heavily upvoted, including various older proposals.

As I understand it, part of the reason it took a while is that the core Go team was generally of the opinion that doing user-facing SIMD APIs the right way was to design a high-level, cross-platform API that would stand the test of time, and that was then punted a few times given its complexity and need to do other things.

Part of what helped the current approach take off was switching to a philosophy of designing a lower-level architecture-dependent API first (the 'simd/archsimd' package), and then later doing a higher-level portable API (the 'simd' package, which is topic of this blog post).

That two-level approach I think also gave some additional freedom for the design and implementation of the friendlier / high-level 'simd' package, including because the lower-level 'simd/archsimd' package is available for people who need or want to drop down.

It's a nice design.

From the former head of the Go security team [1]:

> I have never seen real Go code (i.e. not code written purposefully to be exploitable) that was exploitable due to a data race.

And from tptacek in that same discussion [2]:

> The fact is that Go doesn't admit memory corruption vulnerabilities, and the way you know that is the fact that there are practically zero exploits for memory corruption vulnerabilities targeting pure Go programs, despite the popularity of the language.

[1] https://news.ycombinator.com/item?id=44672003

[2] https://news.ycombinator.com/item?id=44672371

Go is broadly considered to be a memory safe language.

See for example comments from tptacek like:

https://news.ycombinator.com/item?id=43335748

https://news.ycombinator.com/item?id=46028232

https://news.ycombinator.com/item?id=44672371

(The gist: memory safety is a term of art coined by security practitioners. Go, Python, Rust, Java, others: memory safe. C/C++: memory unsafe. Periodically, people in different slices of industry or academia come up with new definitions of memory safety that declare Rust or Go or other languages to be memory unsafe, but that is not by the broadly accepted definition across industry.)

FWIW, there is some pretty substantial autovectorization work that is already in-flight for the Go compiler.

There's a CL stack here:

https://go.dev/cl/791740

It's hard to make predictions with an open source project, but my personal guess is some flavor of it will land (including it is already demonstrating good results without an enormous level of code complexity in the compiler and without overly slowing down compile speeds), but I guess we'll see.

It's being driven by an external contributor who has landed some good changes in the past to the Go compiler. (I think the autovectorization work might be part of their PhD or other academic research, but not sure.)

FWIW, every commit for the Go project is tested on multiple 32-bit architectures, including pre-commit tests and post-commit tests.

You can see a list here:

https://build.golang.org/

(That particular build dashboard just shows post-commit tests. '386' there is 32-bit x86, 'arm' is 32-bit ARM.)

I can't speak to what Google runs in production or what Google tests internally, but the Google-run CI for the Go project definitely does test 32-bit, and those tests essentially match the tests run against 64-bit.

And to answer the second half of your request, here is that exact same code as above, but now using the 1.27 generic methods feature (with a runnable example using tip at https://go.dev/play/p/1YK62tGetsm?v=gotip):

  // Map maps from a slice containing type In to a slice containing type Out.
  func (s MySlice[In]) Map[Out any](f func(In) Out) []Out {
      var out []Out
      for i := range s {
          out = append(out, f(s[i]))
      }
      return out
  }

In short, you could always have methods on a generic type since Go first introduced generics in Go 1.18, but with 1.27, the methods on the generic type can also introduce their own additional type parameters.

(Previously, you could achieve the same net effect with a top-level generic function, but then the code would not be grouped as nicely as hanging it off of the type, and arguably it now can have slightly better ergonomics in some cases. You can see more of the rationale from Robert Griesemer at https://github.com/golang/go/issues/77273.)

It's not a great example.

I think it's trying to show a mapping operation for a generic container where the container values are of one type and the mapping function is allowed to return a container with values of a different type.

Without generics, something along the lines of the following (with runnable example at https://go.dev/play/p/KHBI1uAhbO0):

  type MySlice []int

  // Map maps from a slice of ints to a slice of float64s.
  func (s MySlice) Map(f func(int) float64) []float64 {
     var out []float64
     for i := range s {
         out = append(out, f(s[i]))
     }
     return out
  }

From a quick search, this seems to be better explanation of this new 1.27 feature:

https://www.gopherguides.com/articles/golang-generic-methods

(That uses an example that seems similar in spirit to the Interactive Tour's example, but with a more useful type of a Stack[T] and corresponding explanation seem clearer.)

FWIW, I do remember that quote.

It doesn't seem particularly damning to me, though, especially written by Russ Cox in hindsight. It's almost a truism that if you had spent less time on approaches that didn't pan out and more time on approaches that did eventually pan out, you likely would have arrived at a workable solution sooner.

For example, if the core Go team had spent more time exploring C#-like approaches (as some suggest they should have), it's possible that would have delayed the whole thing.

(btw, consider me a "long time listener, first time caller" -- I pretty much always pause to read your comments while flicking through discussions here, including I appreciate you often bring in historical context, even if I might have a different take, or even if I might have expressed your take differently ;)

> it finally took convincing Philip Wadler (of Haskell and Java generics fame) to help. Without that lucky break [...] Go probably still wouldn't have generics.

Small correction, I think:

My understanding is the core Go team was already well on its way to doing generics before Philip Wadler became involved, though he and his colleagues were later very helpful.

From Wadler's "Featherweight Go" paper [1] (the first draft of which seems to have been uploaded to arxiv.org circa May 2020):

Recently, the Go team mooted a design to extend Go with generics [Taylor and Griesemer 2019], and Rob Pike wrote Wadler to ask: Would you be interested in helping us get polymorphism right [...]

and:

It took us several tries over many months to formalise it correctly.

Well before that, doing generics was one of the main focuses in the GopherCon keynotes/announcements in 2017 and 2018 by Russ Cox [2].

The August 2018 generics design [3][4] was very detailed, and my recollection is most people in the Go community at that point believed the core Go team was going to make generics happen, even if the team and community were still iterating on the exact design.

Of course, that design improved over time, including thanks to Wadler and crew, but I think it would be overstating things to say that probably nothing would have happened without Wadler.

[1] https://arxiv.org/pdf/2005.11710v1

[2] https://go.dev/blog/go2draft

[3] https://go.googlesource.com/proposal/+/master/design/go2draf...

[4] https://go.googlesource.com/proposal/+/master/design/go2draf...

From what I've seen, I don't think the core Go team was ignoring the lessons of Java or C#.

Here's a sample quote from Russ Cox from 11 years ago on this site: [1]

We have spoken to a few true experts in Java generics and each of them has said roughly the same thing: be very careful, it's not as easy as it looks, and you're stuck with all the mistakes you make. As a demonstration, skim through most of http://www.angelikalanger.com/GenericsFAQ/JavaGenericsFAQ.ht... and see how long before you start to think "was this really the best way to do this?"

And of course, they asked other experts for help, including Philip Wadler (of Featherweight Java and Haskell fame): [2] [3]

We’ve been thinking about generics since work on Go began, and we wrote and rejected our first concrete design in 2010. We wrote and rejected three more designs by the end of 2013. Four abandoned experiments, but not failed experiments, We learned from them, [...]

Last year [2018] we started exploring and experimenting again, and we presented a new design [...] and we’ve been working with programming language theory experts to understand the design better.

[1] https://news.ycombinator.com/item?id=9622417

[2] https://go.dev/blog/experiment

[3] https://go.dev/blog/generics-next-step#acknowledgements

Go modules did not get rid of vendoring. You can do 'go mod vendor' and have been able to do so since Go modules were first introduced.

How long the google-run module cache (aka, module proxy or module mirror) at https://proxy.golang.org caches the contents of modules is I think slightly nuanced.

That page includes:

> Whenever possible, the mirror aims to cache content in order to avoid breaking builds for people that depend on your package

But that page also discusses how modules might need to be removed for legal reasons or if a module does not have a known Open Source license:

> proxy.golang.org does not save all modules forever. There are a number of reasons for this, but one reason is if proxy.golang.org is not able to detect a suitable license. In this case, only a temporarily cached copy of the module will be made available, and may become unavailable if it is removed from the original source and becomes outdated.

If interested, there's a good overview of how it all works in one of the older official announcement blog posts (in particular, the "Module Index", "Module Authentication", "Module Mirrors" sections there):

https://go.dev/blog/modules2019#module-index

> They may be an expert in Go, but from their writing they appear to be misunderstanding (or at least misrepresenting) how things work in other languages

Thanks for that link.

Based on reading through that whole discussion there just now and my understanding of the different ecosystems, my conclusion is that certainly people there are telling Filippo Valsorda that he is misunderstanding how things work in other languages, but then AFAICT Filippo or others chime in to explain how he is in fact not misunderstanding.

This subthread to me was a seemingly prototypical exchange there:

https://lobste.rs/s/exv2eq/go_sum_is_not_lockfile#c_d26oq4

Someone in that subthread tells Filippo (FiloSottile) that he is misunderstanding cargo behavior, but Filippo then reiterates which behavior he is talking about (add vs. install), Filippo does a simple test to illustrate his point, and some others seem to agree that he is correct in what he originally said.

That said, YMMV, and that overall discussion does certainly seem to have some confusion and people seemingly talking past each other (e.g., some people mixing up "dependents" vs. "dependencies", etc.).

There’s some more context in a proposal from the folks behind this project to upstream the needed Go runtime hooks into Go proper.

From what I can tell, the core Go team seems generally favorable to it, so seems like a decent chance it will happen.

From:

#73608 proposal: all: add bare metal support

https://github.com/golang/go/issues/73608

> Go applications built with GOOS=none would run on bare metal, without any underlying OS. All required support is provided by the Go runtime and external driver packages, also written in Go.

And:

> These hooks act as a "Rosetta Stone" for integration of a freestanding Go runtime within an arbitrary environment, whether bare metal or OS supported.

Draft 1.26 release notes:

https://tip.golang.org/doc/go1.26

The arena experiment was essentially placed on indefinite hold:

> The proposal to add arenas to the standard library is on indefinite hold due to concerns about API pollution.

I think the parent comment was using arenas as an example that GOEXPERIMENTs don't always move forward (like arenas), or can change while still GOEXPERIMENTs in a way that would normally not be allowed due to backward compatibility (like synctest).

The arena GOEXPERIMENT has not yet been dropped as of Go 1.25, but as I understand it, the plan is to remove arenas from the runtime when 'regions' are introduced, which have similar performance benefits but a much lower API impact:

https://github.com/golang/go/discussions/70257

As discussed there, seeing how people actually used the arena GOEXPERIMENT influenced the 'regions' design.

> The biggest problem is any string you pass as an argument to the fmt functions is moved onto the heap

FWIW, that's not quite correct. For example, a string literal passed as a fmt argument won't be moved to the heap.

The upcoming Go 1.25 release has some related improvements that help strings in more cases. See for example https://go.dev/cl/649079.

Bluesky and atproto seem to be built to be hackable.

Someone in the community recently built a searchable directory of Bluesky "Starter Packs" (which are a way for a user to publish a set of interesting people & feeds to follow, primarily to help newcomers bootstrap their experience):

https://blueskydirectory.com/starter-packs/all

Dan Abramov posted about it earlier today, saying he liked it and:

"the fact that it can be done in the ecosystem is awesome. let the ecosystem cook" [1]

And maybe more poignantly:

"seeing random projects pop up in the atproto ecosystem reminds me just how much public web common were stifled by social companies closing down their APIs. an entire landscape of tools given up on and abandoned" [2]

[1] https://bsky.app/profile/danabra.mov/post/3lar3sdna222d

[2] https://bsky.app/profile/danabra.mov/post/3lar3xpuu4c2d

> you can simply forbid pointers into the stack. This means that your GC doesn't need to be moving. I believe that's what Go does

I might have misunderstood your comment, but FWIW, Go does allow pointers into the stack from the stack.

When resizing/moving/copying a stack, the Go runtime does indeed find those pointers (via a stack map) and adjust them to point to the new stack locations. For example:

https://github.com/golang/go/blob/b25f5558c69140deb652337afa...

(The growable stacks I think replaced the segmented stacks circa Go 1.3 or so; I can't speak to whether they were contemplating growable stacks in the early days whilst considering whether to start their project with the Plan 9 toolchain, LLVM, or GCC, but to your broader point, they were likely considering multiple factors, including how quickly they could adapt the Plan 9 toolchain).

The “Personal” in PDS doesn’t mean it is only for self-hosting.

Bluesky has a main PDS instance at https://bsky.social that serves almost all of the Bluesky user base.

There is a good overview of the architecture here:

https://blueskyweb.xyz/blog/5-5-2023-federation-architecture

Here’s a snippet from the protocol roadmap they published 3-4 weeks ago [1]:

Multiple PDS instances

The Bluesky PDS (bsky.social) is currently a monolithic PostgreSQL database with over a million hosted repositories. We will be splitting accounts across multiple instances, using the protocol itself to help with scaling.

[1] https://atproto.com/blog/2023-protocol-roadmap

If I understand the question, I think that is rsc.io/unix/v6unix, which is at https://github.com/rsc/unix/tree/main/v6unix:

  "Research Unix Sixth Edition (v6) kernel written in Go and using rsc.io/unix/pdp11 to run user-mode code"

For example, you can see the process struct definition starting here:

https://github.com/rsc/unix/blob/main/v6unix/proc.go#L25-L67

Effectively yes.

A fix for the loop variable closure problem is now in the official proposal process, which is how language changes happen in Go.

It’s a concrete proposal from the core Go team and seems to be on track for acceptance:

https://github.com/golang/go/issues/60078

An implementation is already available on tip and in the upcoming Go 1.21 release behind a GOEXPERIMENT flag.

The community reaction has been extremely positive. As one approximate measure, an earlier draft of the proposal had 671 upvotes and with 0 downvotes:

https://github.com/golang/go/discussions/56010

People should essentially never set GOROOT. It's mostly a holdover. For example, Ian Lance Taylor on the core Go team wrote [0] somewhat recently:

"It's a special purpose hook that is almost never needed".

[0] https://groups.google.com/g/golang-nuts/c/qDhJbkE1QeY/m/JoV2...

> Generics seemingly split the community. May be some libraries won’t get used because they picked the wrong side.

I haven't really observed that at all.

One thing that is going on is there hasn't been a massive disruption while everyone stops to rewrite the world in generics, and generics are not suddenly everywhere, which is what some people had predicted would happen. I think part of the reason is that in some cases another solution (closures or interfaces or whatever) can be a better fit, and the evolutionary approach to generics that Go took means you can use generics in conjunction with non-generic libraries or other pre-existing approaches without suffering from an ecosystem split.

goda[0] is excellent. You do do things like:

  # show the impact of cutting any package
  goda cut ./...:all   

which prints a sorted ASCII table with stats like 'size:4.4MB loc:134171' for each package, which is an estimate the savings you'd get if you eliminated that package from your binary. That is a great way to see what is unexpectedly large compared to its value.

goda has a bunch of other capabilities around dependency analysis, and was written by long-time Go contributor Egon Elbre. The examples in the README are the best way to get started after 'go install github.com/loov/goda@latest'.

[0] https://github.com/loov/goda

There is a pretty complete description of the origination of UTF-8 here:

https://www.cl.cam.ac.uk/~mgk25/ucs/utf-8-history.txt

Rob Pike opens with:

> Looking around at some UTF-8 background, I see the same incorrect story being repeated over and over. The incorrect version is: 1. IBM designed UTF-8. 2. Plan 9 implemented it.

> That's not true. UTF-8 was designed, in front of my eyes, on a placemat in a New Jersey diner one night in September or so 1992.

FWIW, this is maybe an area where Go goes against the grain a bit and goes out of its way to not allow code you just downloaded to execute anything while you are building.

For things like 'go generate', the convention is to check in the results, which means a consumer of a package has the results without executing code:

https://go.dev/blog/generate

This seems to be a nice example of maintainers of a popular open source project listening to feedback from the broader community:

> Thanks to everyone who took the time to write constructive, helpful feedback. Those discussions are open source at its best.

FWIW, there is a proposal to add telemetry to LLVM [0] and Rust used to have telemetry [1], both off by default. Some things in the node.js world have telemetry enabled by default, like Next.js [3].

Some people are posting here as if this is already decided -- AFAICT, that's not the case. It's not even a formal proposal yet, and the stated intent was to start a conversation around something concrete. (For context, this is standard for how the Go project approaches large topics, including for example I think there were something like ~8 very detailed generics design drafts from the core Go team over ~10 years).

It sounds like the Go team is going to take some time to look into some of the alternative approaches suggested in the feedback collected so far.

In any event, this is obviously a topic people are very passionate about, especially opt-in vs. opt-out, but I guess I would suggest not giving up hope quite yet.

[0] https://discourse.llvm.org/t/rfc-lldb-telemetry-metrics/6458...

[1] https://github.com/rust-lang/rustup/issues/341

[2] https://nextjs.org/telemetry

Some additional context in this comment:

https://news.ycombinator.com/item?id=34754632

FWIW, people had started to post obscene ASCII drawings, were using various curse words, and there were many, many repetitive comments across the ~400 comments posted.

From what I understand of the conversation there, the core Go team said they were going to take some time to digest the feedback, including [0]:

> The goal here is a productive conversation that aims at better understanding of different positions. Many comments here have contributed to that, and I am grateful for them. To be extra clear, the people who have been discussing opt-in vs opt-out respectfully and with reasoned arguments are most welcome here and have been an important part of the signal, not the noise. Thank you to them in particular.

> Much of the moderation is being done by volunteer contributors working valiantly to keep the conversation on track, polite, useful, and non-repetitive. I appreciate their efforts.

> This discussion has in fact scaled somewhat beyond what GitHub discussions can reasonably manage (I just spent a while clicking every "load more" link on the page to make ^F work again), which is causing even more repetition, so I will probably lock the discussion at the end of the day and take some time to think about the feedback we've gathered so far.

[0] https://github.com/golang/go/discussions/58409#discussioncom...