Comment by jerf | Hacker News Reader

Comment by jerf | Hacker News Reader

Parent Full thread

Python copies references by value.

    $ python3
    Python 3.12.3 (main, Jul 31 2024, 17:43:48) [GCC 13.2.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> def x():
    ...     v = 1
    ...     y(v)
    ...     print(v)
    ... 
    >>> def y(val):
    ...     val += 1
    ... 
    >>> x()
    1

A pass-by-reference language would print 2.

Everything in a modern language is passed by copy. Exactly what is copied varies and can easily be pointers/references. But there were languages once upon a time that didn't work that way. It's a dead distinction now, though, unless you go dig one of them up.

If you want a specific one to look at, look at Forth. Note how when you call a function ("invoke a word", closest equivalent concept), the function/word doesn't get a copy of anything. It directly gets the actual value. There is no new copy, no new memory location, it gets the actual same memory as the caller was using, and not as a "pointer"... directly. Nothing works like that any more.

C++ is a live language, C# has out parameters.... there's stuff out there.

The classic example of "pass by copy-reference is less expressive" is you can't have pass a reference to number and have the caller modify it. You have to explicitly box it. I understand you understand this, but it's worth considering when thinking about whether the distinction means absolutely nothing at all.

> The classic example of "pass by copy-reference is less expressive" is you can't have pass a reference to number and have the caller modify it.

This is really not true. Depending on how your language implements pass-by-reference, you can pass a reference to an int without boxing in one of two ways: either pass a pointer to the stack location where the int is stored (more common today), or simply arrange the stack in such a way that the local int in the caller is at the location of the corresponding parameter in the callee (or in a register).

The second option basically means that the calling convention for reference parameters is different from the calling convention for non-reference parameters, which makes it complicated. It also doesn't work if you're passing a heap variable by reference, you need extra logic to implement that. But, for local variables, it's extremely efficient, no need to do an extra copy or store a pointer at all.

Hmmm... yeah that's a good point. Though I would contend that the fact that languages do not do this is indicative of... something.

I would guess that the main reason is that the on-stack way only works for local variables. If you want to pass anything else by reference, you need to use some kind of address to it, since it's not in the caller's stack anyway.

uh I'd not say it like that

Python passes primitive types by value, out rather "as if by value", because it copies them on write.

if you modify your experiment to pass around a dict or list and modify that in the 'y', you'll see y is happily modified.

so Python passes by reference, however it either blocks updates (tuple) or copies on write (int, str, float) or updates in place (dict, list, class)

jerf is correct. Please read this:

https://stackoverflow.com/questions/373419/whats-the-differe...

> if you modify your experiment to pass around a dict or list and modify that in the 'y', you'll see y is happily modified.

No, you won't.

  x = {'a' : 1}
  foo(x)
  print(x)

  def foo(z):
    z = {'b' : 2}

You'll see that this prints `{'a' : 1}`, not `{'b' : 2}`. Python always uses pass-by-value. It passes a copy of the pointer to a dict/list/etc in this case. Of course, if you modify the fields of the z variable, as in `z['b'] = 2`, you do modify the original object that is referenced by z. But this is not pass-by-reference.

Is it not pass-by-reference by some technicality? In the mutation example you suggest, if a reference to x isn't being passed into foo, how could foo modify x?

I would sooner believe the example is showing you shadowing the z argument to foo, than foo being able to modify the in-parameter sometimes even if it's pass by value.

"In the mutation example you suggest, if a reference to x isn't being passed into foo, how could foo modify x?"

Because it is passing a pointer by value under the hood.

This is the part that messes everyone up. Passing pointers by value is not what passing by reference used to mean.

And it matters, precisely because that is extremely realistic Python code that absolutely will mess you up if you don't understand exactly what is going on. You were passed a reference by value. If you go under the hood, you will find it is quite literally being copied and a ref count is being incremented. It's a new reference to the same stuff as the passed-in reference. But if you assign directly to the variable holding that reference, that variable will then be holding the new reference. This is base level, "I'd use it on an interview to see if you really know Python", level stuff.

Everything in a modern language involves passing things by value. Sometimes the language will gloss over it for you, but it's still a gloss. There were languages where things fundamentally, at the deepest level, were not passed by value. They're gone. Passing references by copy is not the same thing, and that Python code is precisely why it's not the same thing.

Sure, but in the traditional sense of pass-by-reference you could say it's just always pass-by-value, and that value is always a reference. It's just not a helpful thing to say. (In those traditional pass by reference languages, was it impossible to pass a scalar to a function?)

Passing a pointer-by-value similarly seems to be avoiding the point; if you tell me Python is always pass-by-value I'll expect an object I pass to a function to be a copy & not a reference, thus not be able to be mutated, and that's not the case.

> Passing a pointer-by-value similarly seems to be avoiding the point; if you tell me Python is always pass-by-value I'll expect an object I pass to a function to be a copy & not a reference, thus not be able to be mutated, and that's not the case.

That would be a misunderstanding. It would only make sense if you think Python variables are Python objects. They are not: Python variables are pointers to objects. The fact that assignment to a variable never modifies the object pointed to by that variable is a consequence of that, and doesn't apply just to passing that variable to a function.

> In the mutation example you suggest, if a reference to x isn't being passed into foo, how could foo modify x

The important point is that it's not "a reference to x" that gets passed, it's a copy of x's value. x's value, like the value of all Python variables, is a reference to some object. The same thing applies to setting variables in Python in general:

  x = {1:2} # x is a new variable that references some dict 
  y = x # y is a new variable that references the same dict
  y[1] = 7 # the dict referenced by x and y was modified
  x = None # x no longer references the dict
  print(y) # y still references the dict, so this will print {1:7}
  y = None # now neither x nor y reference that dict; since y was the last reference to it, the dict's memory will be freed

This is what confuses me; it sounds like what you're saying is Python isn't pass-by-reference, it's pass-by-value, and that value is sometimes a reference?

Honestly, "x's value, like the value of all Python variables, is a reference to some object" makes me think it's more accurate to call Python pass-by-reference only.

Pass-by-reference means that your callee gets a reference to your local variables, and can modify them. This is impossible in Python. Pass by value means that your callee gets the values of your local variables and can't modify them. This is how Python functions work.

What those values represent and how they can be used is a completely different topic. Take the following code:

  x = "/dirs/sub/file.txt"
  with open(x, "w") as file:
    file.write("abc")
  foo(x)
  with open(x, "r") as file:
    print(file.read_all()) #prints "def" 

  def foo(z):
    with open(z, "w") as file:
      file.write("def")

Here x is in essence a "reference to a file". When you pass x to foo, it gets a copy of that reference in z. But both x and z refer to the same file, so when you modify the file, both see the changes. The calling convention is passing a copy of the value to the function. It doesn't care what that value represents.

So to be very clear:

  def foo(x):
    x['a'] = 1
  
  y = {'b': 2}
  foo(y)
  print(y)

foo can modify the object y points to, but it can't make y point to a different object? Is that what "This is impossible in Python" is referring to?

Yes.

yes! we agree how this works.

so we disagree on terminology?

in my CS upbringing, sharing the memory location of a thing as parameter was tagged "call by reference". the hallmark was: you can in theory modify the referenced thing, and you just need to copy the address.

call by value, in contrast, would create an independent clone, such that the called function has no chance to modify the outside value.

now python does fancy things, as we both agree. the result of which is that primitives (int, flot, str) behave as if they were passed by value, while dict and list and its derivatives show call by reference semantics.

I get how that _technically_ sounds like call by value. and indeed there is no assignment dunder. you can't capture reassignment of a name.

but other than that a class parameter _behaves_ like call by reference.

you replace the local binding z to the dict globally bound to x by a local dict in that z = ... assignment.

however if you do z['b'] = 2 in foo, then you'll see the global dict bound to x has been modified, as you have stated.

well, that's _exactly_ pass by reference.

There is no notion of variable binding in Python, that's a different thing. z, like any Python variable, is a reference to something. Initially, it's a reference to the same dictionary that x references. If we modify the object referenced by z (e.g. by adding a new item), we of course also modify the object referenced by x, as they are referencing the same object initially. However, when we assign something to z, we change the object that z is referencing. This has no effect on x, because x was passed-by-value to foo().

Pass-by-reference doesn't exist in Python. Here's what it looks like in C#, which does suport it:

  auto x = Dictionary<string, int >() ;
  x.Add("a", 1);
  foo(ref x);
  System.Println(x); //prints {b: 2}

  void foo(ref Dictionary<string, int> z) {
    auto k = new Dictionary<string, int>();
    k.Add("b", 2);
    z = k;
  }

Here z is just a new name for x. Any change you make to z, including changing its value, applies directly to x itself, not just to the object referenced by x.

Reply on news.ycombinator.com