Proposal: expression to create pointer to simple types
github.com
github.com
s := S{}
sp := &s // Works
_ = sp
_ = &S{} // Works
i := int32(1)
ip := &i // Works!
_ = ip
_ = &int32(1) // Doesn't work!
https://play.golang.org/p/fdgvbEwJWghIt seems odd that you can't apply & to a function's return value. I think the best approach would be making & work in basically any scenario. For example the following also doesn't currently work.
&(int32(1) + int32(1))
It seems like it should be possible to "desugar" &X to `_tmp = X; &_tmp` and solve this weirdness.Here's an example with a bit of explanation: if you pass a value to fmt.Println it will escape. The raw println builtin does not cause values to escape. So calling the first function twice and seeing the same address for the value strongly implies stack allocation while calling the 2nd function twice and getting different addresses implies heap allocation.
Thank you for explaining what's going on there! :)
And btw, compiling the above example with -gcflags="-m" (which I learned about earlier today) proves you right.
:= kinda smears the clarity by not allocating if you have a variable on the left that is already allocated, and there's some other places where it kinda smears things up, but at the core, Go makes you explicitly allocate.
Most people probably think "heap allocation" when you say this. Go doesn't do dynamic allocations within a stack frame (alloca in C), so when you say "it's an allocation", what does that mean? It could be a stack allocation that occurred at compile time as a reservation in the size of the stack frame for that function. It could be a heap allocation. Only the compiler knows!
The Go compiler is the ultimate authority on what becomes a heap allocation. It tries to make everything into a stack allocation when possible, and stack allocations are "free".
Beyond that, a sufficiently smart compiler can reuse stack "allocations" within a single function as certain values become "dead" (never used again). So there isn't even guaranteed to be a 1:1 correspondence between "stack allocations" and variables that you declared inside the function.
So, I completely disagree with your statement about Go being "relatively explicit about allocations." It's one of the least explicit compiled languages in that regard.
Go makes a distinction between declaration and assignment, which is the syntax you're talking about. It really has nothing to do with allocations.
Also, allocations are just... allocations. Go qua Go doesn't have stack vs. heap, and it's a mistake to care except when optimizing. So in Go qua Go, it isn't an issue that it may "reuse" a particular address, because in Go qua Go you can't witness that anyhow. (If you try to keep a pointer around to witness it with, you'll keep the thing pointed to alive.) From Go's perspective, it's still an allocation even if the implementation manages to re-use a particular memory address to do so.
I'm talking about the runtime Go implements here, not the implementation.
This actually took me some years to correctly internalize, for what it's worth. It does a "good" job of glossing over things. However, if you really poke at it, allocations are still explicit. They just may not look like what you are used to from other languages.
temp := someFunc()
p := &temp
is any less explicit than p := &someFunc()
It seems that the `&` is still required to put something onto the heap.The simpler way to think about it is that in Golang everything is on the heap. However the optimizer will move things to the stack if they don't have their address taken. I think the point about explicitness is that if you don't use `&` then it will be able to be put on the stack. So `&` doesn't cause a heap allocation but lack of `&` (or new()) confirms that there isn't one. (I don't actually know if that is true but I can't think of any counterexamples)
I think assigning to a pointer would cause an escape.
Just taking a reference wouldn't though, the reference still has to escape (of course you'd usually take a reference so that it can escape but that's not always the case, especially with inlining).
*x = y x := make(map[int]int)
x[0] = 5
y := &x[0]
*y = 10
print(x[0]) // 5 or 10?
x[0] = 6
print(*y) // 6 or 10?
// force the map to grow and reallocate the buckets
for j := 1; j < 100; j++ {
x[j] = j
}
*y = 11
print(x[0]) // 5, 6, 10, or 11?
The crux of the problem is answering what y actually points at: the value in the map bucket, or some freshly allocated value? There are problems with whichever one you pick.edit: changed the second print to *y instead of x[0]. thanks masklinn for catching this error.
Do you mean `print(*y)`? You just assigned to `x[0]` so its value should not be in question.
Also
> if it's the latter, resizing the map invalidates the pointer, so it must be updated somehow.
It doesn't (have to) invalidate the pointer though. When resized the map's content get copied to a new backing buffer, the pointer can keep pointing to the old buffer. That's basically the same behaviour as slices: when a slice resizes, a new backing array is allocated, the contents get copied to the new array, and the slice is retargeted to the new array. There can be other slices pointing to the old array (it's of course a very bad idea to update slices to shared arrays, but Go will let you do it).
That's true, but I don't think it's very comparable to slices. With slices, you have to explicitly reallocate either by creating a whole new slice or using append. Reslicing, indexing, or other operations do not reallocate. On the other hand, maps may end up resizing on any operation that involves them, or even theoretically in the background without any operations (during GC, for example). It would be unfortunate to lose that implementation flexibility, and keeping it means that you're essentially picking the "make a copy" option.
It's exactly the same.
> With slices, you have to explicitly reallocate either by creating a whole new slice or using append.
That's a distinction without a difference. `append` does not "explicitly reallocate", it may or may not reallocate, you've no idea. Even if the backing array is full, it might be realloc'd in-place.
> On the other hand, maps may end up resizing on any operation that involves them, or even theoretically in the background without any operations (during GC, for example).
So?
Also technically nothing prevents a GC from reallocating the slice.
> It would be unfortunate to lose that implementation flexibility, and keeping it means that you're essentially picking the "make a copy" option.
I've never heard of a hashmap implementation which would do otherwise.
Trying to extend in-place and attempting to properly redistribute if that works sounds like absolute hell. Likewise trying to shrink in-place, though at least you've got some scratch space which you don't have in the other case: you'd have to segregate everything into one half of the map then insert them in the other half, before shrinking your allocation, which might give you a new allocation anyway, at which point you've moved all your values thrice whereas just creating a new allocation and reinserting your stuff there is a single move.
Maybe to you, but to me, a pointer going from modifying the value inside of the map to no longer modifying the value inside of the map during any operation is quite a bit different than requiring a reassignment of the slice header. In other words:
x := make([]int, 5)
y := &x[0]
x[3] = 8
*y = 5
print(x[0]) // always prints 5
as compared to x := make(map[int]int)
y = &x[0] // btw, is this even valid? let's assume it implicitly does x[0] = 0
x[3] = 8
*y = 5
print(x[0]) // maybe sometimes prints 5?
is meaningfully different. For slices, we know that x[0] will always print 5 until the value of x is reassigned in some way.> Also technically nothing prevents a GC from reallocating the slice.
It would have the same problem the map does: you'd have to update any pointers into the slice to point to the new slice, otherwise the semantics of the program changes. That is not something the GC currently does, and would require an awful lot of metadata and scanning.
> I've never heard of a hashmap implementation which would do otherwise.
I'm not sure what this is referring to. I agree every map implementation has to reallocate the backing store of values periodically. I was trying to say that keeping the flexibility to reallocate the backing store of the map during GC means that you cannot choose the "writes through pointer are observed in the map" option (at least without a lot of complication around updating pointers) because as a programmer, you would not be able to know if it would do that or not, which is a fairly useless primitive.
Yes, I avoid making assumptions about invariants across mutation calls, that's just a bad idea.
> For slices, we know that x[0] will always print 5 until the value of x is reassigned in some way.
Unless an other goroutine is stomping on your backing array anyway.
> It would have the same problem the map does: you'd have to update any pointers into the slice to point to the new slice, otherwise the semantics of the program changes. That is not something the GC currently does, and would require an awful lot of metadata and scanning.
Yes. So maybe we could ignore that useless strawman?
> I'm not sure what this is referring to.
To what I'm quoting.
> I was trying to say that keeping the flexibility to reallocate the backing store of the map during GC
That sounds less like flexibility and more like "let's make the GC slower and more complex for no reason".
> := [...] an error to use it when it doesn't allocate.
> := [...] not allocating if you have a variable on the left that is already allocated,
This appears to be a contradiction.
I suppose you mean something like "error to use it when there's no possible context where that line of code would allocate"; what's an example of that?
If either a or b (but not both) were already defined, this won't re-define (and reallocate space for) them.
foo := MyStruct{}I believe the person you are replying to was making a confusing point about some hand wavy notion of "any kind of allocation", which includes stack allocations... which are determined at compile time, not with "alloca".
foo := MyStruct{} // Could live on the stack
_ := &foo // Oh, now foo must live on the heap.No matter what syntax you write inside a function, the Go compiler always has the final say on what is stack allocated and what is heap allocated. Taking the address of foo will not cause foo to be heap allocated unless Go is unable to prove that the pointer will live for less time than the current stack frame. Look up "escape analysis".
Basically the only way to guarantee that something will always be heap allocated is to assign it to a global variable. Even returning a pointer to that object from the current function is not a strong guarantee, since the compiler could inline this function into the caller and determine that everything can live happily inside the newly inlined stack frame without heap allocation.
Otherwise the meaning of &f().x is different for f() returning pointer-to-struct and f() returning struct.
Similarly &m["x"] is a compile error today but would silently make a copy tomorrow rather than produce a pointer to the value in a map.
All of that would be incredibly confusing and the source of many subtle bugs.Offtopic: Surprisingly I was asking myself this question but if possible in C... Is it?
In C:
int *f() {
int x = 0;
return &x;
}
It works, but it is wrong. The C type system isn't smart enough to realize the lifetime of x in this case. It is not allowed for a function return because C does have the concept of a temporary value so it is disallowed because it is basically always incorrect to do so.Note that C++ does somewhat allow this with lifetime extension. It is somewhat like what I expected Go to do, except because lifetime extension only extends to the enclosing block it is more of a footgun. With a dynamic tracing garbage collector like Go it not a footgun.
int* ref1() { return &1; }
-> error: lvalue required as unary '&' operand
//
#include <stdlib.h>
int alloc() {
return *(int*)malloc(sizeof(int));
}
int* ref() { return &alloc(); }
-> error: lvalue required as unary '&' operand
You can still be unsafe though, by making a reference to a stack-allocated object and letting it go out of scope : int* make_unsafe_ref() { int a; return &a; }
-> warning: function returns address of local variableI mean it could implicitly allocate, that's what Rust does for instance.
Your second and third attempts would not compile though, the first would by returning a `&'static T`.
Maybe. I just meant that storage is created implicitly (static or stackframe depending on the case), then a reference is created to that.,
&(T[]){ f() }
The type in brackets needs to be an array so that if f() returns a struct then the initializer list has the right shape. If T is a simple type then you can drop the [].I tend to not declare variables when the pointer is used deep into a struct because I find the back-and-forth in the editor to be bad. I usually resort to a pointer to an inline anonymous function, e.g.:
a := SomeStruct{
Field: func() *int64 { x := int64(13); return &x }(),
}
It's ugly and verbose but after seeing it 2 or 3 times you immediately know what it's about the next time.> No.
Not sure why, but something about this being the (first part of the) last question he had to answer makes it quite funny to me.
Can you describe a possible implementation? (Yes.)
And the General who responds, "Right. Well done."
> What would change in the language spec?
> The new operator would get an optional second argument, and/or conversions would become addressible.
> [...]
> How would the language spec change?
> Answered above. Why is this question here twice?
util.StrPtr("hello")
util.BoolPtr(false)
util.Int64Ptr(7)
// etc
This is just a gap in Go's design, so I'm glad this proposal exists :) func PointerOf[T any](t T) *T {
return &t
}
But that has a nasty gotcha: func Process(x *int32) {
if (x != nil) {
fmt.Println(*x + 5);
}
}
func main() {
Process(nil); //ok
x := i32(5)
Process(&x); //ok
Process(PointerOf(5)); //BOOM: cannot use PointerOf(5) (value of
//type *int) as *int32 value in argument to Process
}
Go's type coercion is quite primitive. It strictly works inside-out (propagating types strictly upwards in the AST), with the only exception that a numeric literal can be coerced into a specific numeric type by considering the immediate parent in the AST. So when you have `func f(x int32)` and you call it as `f(5)`, the literal 5 gets coerced into int32 to match the context it appears in. (The same strategy is also applied to determine the type of a nil literal.)However, in `Process(PointerOf(5))`, the immediate surrounding of the literal 5 (the PointerOf call) does not coerce the literal into a specific type, so it takes on its default type, int.
The proposal (or, to be exact, both proposals) avoids this gotcha by requiring a type to be stated explicitly.
Process(new(i32, 5));
Process(&i32(5));> Fortran, C, Forth, Basic, C, C++, Java, Python, and probably more. Just not JavaScript
Forth is a "top-of-mind" language for Rob Pike. That's unexpected and incredibly cool :)
(I was slightly surprised when I learned that C99 compound literals are not just for structs: you can use any complete object type, and the result is an lvalue so you can take its address.)
Why not? In fact it already kind-of does: Go has "default types" for most untyped constants. When you write
i := 3
absent an explicit type, Go will fall back to "int".> Numeric constants represent exact values of arbitrary precision and do not overflow.
package main
import (
"fmt"
)
const tst = 1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
func main() {
fmt.Println("%v", tst)
}
Error:
./prog.go:16:17: constant 1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 overflows int> An untyped constant has a default type which is the type to which the constant is implicitly converted in contexts where a typed value is required.
The default type for a number (integer) is int. If you were to add a period in that long string of zeros, the default for floating-point is float64.
tl;dr: Go uses casts to coerce values to the correct types, which means you couldn't get pointers to number literals for non-default number types.
Love this one
> Would you consider yourself a novice, intermediate, or experienced Go programmer?
I have some experience.
How is that possible? At least in Common Lisp, all literal objects have types, and the same is true of C from what I have just checked.
Prelude> :t 3
3 :: Num p => p
> It is an error if the constant value cannot be represented as a value of the respective type. An untyped constant has a default type which is the type to which the constant is implicitly converted in contexts where a typed value is required, for instance, in a short variable declaration such as i := 0 where there is no explicit type. The default type of an untyped constant is bool, rune, int, float64, complex128 or string respectively, depending on whether it is a boolean, rune, integer, floating-point, complex, or string constant.
Practical benefit in the sense of "you get to express something in a shorter, more uniform way"? Some.