Stunned by Go | Hacker News Reader

Stunned by Go

how-bazaar.blogspot.co.nz

320 pointssquidsoup108 comments

Go developer here. I posted this on the blog too.

""" This is a mistake, plain and simple. It was pointed out on the Go developer mailing list a few days before your post, and you can see my reply there.

NewRequest should take an io.ReadCloser. Unfortunately, due to backwards compatibility, a long standing semantic mistake like this is not something we can just fix, but we've at least documented it.

The tweet you quoted is wrong, and I want to explain why. If an interface value v is passed to a function f, then what does happen from time to time is that f will look for custom methods on v that are at least logically equivalent to the static interface type f is declared to expect. For example, io.Copy takes an io.Reader and an io.Writer and does the obvious thing, Reading into a buffer and then passing that buffer to Write. Of course, it would be nice to bypass the buffer when possible, and so there is a standard WriterTo interface that an io.Reader can also implement that, in contrast to Read, which copies into a buffer, says "read the data out of yourself and directly into this Writer". If io.Copy finds that method, it will use it, but the operation - reading - is the same. There is an analogous case for the Writer too. If you pass a value v to json.Marshal, Marshal will check to see if v implements json.Marshaler, meaning v knows how to marshal itself as JSON, in which case v is given that opportunity. Again, same operation you requested.

In contrast, if you have an io.Reader, while it might be okay to look for other Read-like methods, it is certainly not okay to look for Close. That's a very different operation, it's surprising, and it shouldn't be there. FWIW, I'm not aware of any other instances of this kind of semantic mixup in the standard library. But again, unfortunately, we can't change it until Go 2. All we can do for now is document it and move on. """

Out of simple idle language-design-bystander curiosity, can you recall any other times where this class of bug has bitten?

No.

A related bug would be if you implement Read and (say) ReadByte, but your ReadByte is buggy. Then when you get weird behavior passing yourself to something that expects a Reader (but internally needs byte-at-a-time and uses ReadByte to avoid double-buffering the input) it can take a minute or two to realize that ReadByte might be the problem and not Read. I've only seen that once or twice.

I know of atleast one instance in C# where something like this happens. The IEnumberable<T>.Count() extension sees if the input is actually a ICollection and uses the count on that to get the count directly, rather than iterating over it.

I remember writing extension methods in C# for IEnumerable<T>, which would of course take in an IEnumerable<T>, but saw if the actual input was a ICollection<T>, IList<T> etc to optimize the operations.

I'm not a heavy C# dev, but I don't understand why your first paragraph would be so: isn't that the point of implementing an interface like IEnumerable<T> -- so that you can implement Count() in an optimized, specialized way that "gets the count directly"?

Some implementations of IEnumerable do have a Count property on them. He's specifically talking about the Count() extension method. Since this extension method applies to all IEnumerable<T> implementations, there may or may not be an optimized way of accessing the count, and this method would need to know about them. The code looks a little something like this (decompiled from the .NET framework, so the variable names are probably not true to what they're actually named in the real source)

https://gist.github.com/anonymous/7c2c65268feafa66fb02

Count() is an extension method for IEnumerable<T>. This means that Count() is a static method which takes an single argument of type IEnumerable<T> and ostensibly only depends on members defined by IEnumerable<T> to work (but manojlds points out that this isn't really the case). C# provides syntactic sugar to make calling a static extension method look like calling an instance method.

If you wanted to implement your own "Count" you would implement ICollection<T> which the IEnumerable<T> extension method would call into.

(I'm not a c# expert and I may be wrong) IEnumerable really only requires a

IEnumerator GetEnumerator()

method most of the real "meat" of ienumerable is in the static System.Linq.Enumerable class. You see c# doesn't have multiple inheritence or scala like traits or type classes but it does have a cool compiler hack called Extension methods(http://msdn.microsoft.com/en-us/library/vstudio/bb383977.asp...). You take a static class with static helper functions like you might make in java and you add a this keyword to the first argument then you can call it with a syntax that makes it look like it was a method on that class. For example something like

    public static class Enumerable
    {
        public static int Count<TSource>(this IEnumerable<TSource> source
        {
            int i=0;
            for(TSource t in source)
            {
                i++;
            }
            return i;
        )
    }

And if you use Extensions methods on an interface type you get all of the extension methods for free once you implement the minimal interface. One of the downsides is that you can only really define it in one place and can't override it and if you want even somewhat efficient type specialized versions you need to cast.

I believe in scala you can use traits in a similar manner for default implementation and specialize them for collections but traits are more complicated/powerfull feature.

This is not a compiler hack, there are quite a few languages out there that support this concept.

As for your Scala remark, extension methods in Scala are done via implicits.

I didn't mean hack in a derogatory way, I think its a fairly elegant solution in many ways just that to my knowledge it doesn't really change the static/dynamic type and is just a simple compiler transformation.

I should have mentioned that I know even less about scala then c#. I/m not quite sure how implicits are implemented/optimized but I'm pretty sure you can have inheritence/overiding traits as implicits.

This is not the same thing. The problem with the Go implementation is essentially that the method is semantically different from the expected method.

Closing a stream is semantically different from not closing a stream. It just is. The Go method was just wrong.

The problem was not the type introspection. There's nothing wrong with us attempting to do optimization for ICollection`1 or the rest of the collection interfaces because we have exactly the same semantic effect.

If you use different methods to achieve the same semantics, that's an implementation detail. If you use any methods to produce different semantics, that's a bug.

> This is not the same thing. The problem with the Go implementation is essentially that the method is semantically different from the expected method.

No-one is arguing about the Close() case. However, the io.Copy() case is comparable to the IEnumerable<T>.Count() implementation.

Unless your WriterTo method does something different than the caller assumes it's going to do.

Unless the special functions are predefined by the language (e.g. Python's __str__, __len__, __iter__, etc.) or specified in an interface that the class explicitly inherits from, making assumptions that the language developers and I meant the same thing is careless at best and dangerous at worst.

That's morally wrong, but at least the language provides an alternative: you could provide overloads for the function, and so it would be clear that the Count() on an ICollection was a different method from the Count() on an IEnumerable (at least, I assume you can - if C# extension methods make overloading impossible then this is a Bad Thing and should be fixed. (But even then, a problem with extension methods is much less significant than a problem with all functions)). In Java we see this with e.g. Guava's ImmutableList.copyOf(...), which has several overloads taking different subinterfaces (iterable, collection, list, ...) so you can reasonably tell that the method will use all the details of its input.

The semantics of ICollection<T>.Count and IList<T>.Count are well defined and known to you at the time your write your extension method.

You don't know the semantics of a method of an arbitrary type you've been passed in just because it has the same name. Go plays fast and loose with this idea, but at least the interface type of the argument should document "I'm expecting these methods to have a particular semantics". If they don't, that's then your fault.

The Go dev has acknowledged that this is a bug, but also pointed out why something like this was done elsewhere ( io.Copy ), and the Count example was pointed at that.

You adopted "looking for custom methods" without making it part of the contract in any way as a best practice in the standard library, but it is a "mistake, plain and simple"? Doesn't it, like, illustrate a serious language and/or standard library design issue? What can possibly motivate breaking encapsulation in the standard library? Why even have interfaces if even the standard library will abuse them? Your response is not very convincing...

Theoretically, if the standard library only optimizes about interfaces whose semantics are in the standard library (i.e., standard library interfaces) that's perfectly fine.

What worries me about this approach is that Go is essentially duck-typed -- implementing an interface isn't necessary for something to appear like it implements an interface, and a method signature doesn't implicitly carry its semantic information alongside it.

Something like the proposal for C++ Concepts captures some of the semantics via axioms (http://en.wikipedia.org/wiki/Concepts_(C%2B%2B)#Axioms) seems like a really interesting solution for this idea.

It’s not like that. It makes sense in Go. Say you have a type, and you implement “Read” function for it. You don’t do anything else but now you are implementing an interface and other functions can call your “Read” function. Having a “ReadByte” function and other functions calling that isn’t that much different from your perspective. Any type that implements “Reader” may implement similar functions too. And if they do, that’s probably for a reason. If your type has a specific “ReadByte” function you want others to use that instead of the generic “Read” for reading a single byte.

Unless your ReadByte function does something different than what the function you call assumes it does.

For example, it might have different behaviour in edge cases (end of file, I/O exception). One could call that bad architecture, but one could not even be aware of the existence of the broader interface that has a ReadByte member function, or one could (IMO rightfully) think the difference does not matter of you call a method taking an argument of interface type that does not include ReadByte.

In the absolutely worst case, it does something completely unrelated to what Read does. Unlikely? Yes, but it might be part of a badly factored class that also has code for reading from an A/D converter.

> what does happen from time to time is that f will look for custom methods on v that are at least logically equivalent to the static interface type f is declared to expect.

Interesting. This kinda sounds good in practice, assuming programmers respect the interface's semantics when downcasting. But from a language design point of view it strikes me as a bit hacky al least, and at odds with the "Tell, Don't Ask" principle.

Couldn't this need for downcasting be avoided by allowing something like default method implementations in interfaces? I'm asking this without any practical knowledge of the Go language; i'm sure different approaches have been considered, but i'm interested in understanding the reasons behind the preference for downcasting :)

Imagine the Reader interface declared, besides the Read method, a WriteTo method with a default implementation:

  type Reader interface {
      Read(p []byte) (n int, err error)
      WriteTo(w Writer) (n int64, err error) {
          // Default implementation in terms of Read and w.Write 
          // using a buffer just like io.Copy
      }
  }

Then any type with a Read([]byte) (int, err) could still be referenced as a Reader, and when seen as a Reader one could call the convenient WriteTo method without downcasting. io.Copy could then tell the objects what to do instead of asking them what they are:

  func Copy(dst Writer, src Reader) (written int64, err error) {
      return src.WriteTo(dst)
  }

A type that complies with the Reader interface could also implement a specific version of WriteTo if needed for performance reasons.

This way not only programmers could avoid resorting to downcasting, but the interfaces would become more explicit: instead Reader being "something with a Read method... and maybe a WriteTo if you want to, but this is not declared in Reader itself" it'd be "something with a Read method and a default WriteTo mehod".

The downsides of this would be that the dispatch semantics become a bit more complex for having to consider the default implementations, and maybe the interfaces could become more "bloated" (though i think that's subjective, as the semantics of an interface like Reader are implicitly mixed up with things like WriterTo; having all the methods in Reader would make that existing coupling more explicit).

Wow, this is pretty unfortunate. Also a very good example of why so many people maintain that Go is not nearly as statically typed as languages like Haskell or OCaml: casting (essentially subverting the type system) is very common, and limits most of the type system's advantages.

In large part this is because Go's type system is not very expressive. I would just like to point out that OCaml did the same things as Go (i.e. structural sub-typing). But with (global) type inference. And variants (including structural subtyping for those too). And generics (parametric polymorphism). Years before Go was developed.

In OCaml, this sort of thing probably would be much less likely to come up. After having used the OCaml object system a bit (but still more than most people!), I am actually very happy with it. It certainly has issues--oh boy, does it ever--but the issues are superficial. They are the result of very limited manpower in the project, not fundamental shortcomings of the design.

In OCaml, the compiler can figure out what methods you used without your telling it. So it would automatically realize that the function called the close method, and infer the appropriate type. You wouldn't even have to write an interface (called a class type in OCaml) for it. If you did write an interface, but it did not expose a close method, the code would give you a type error.

I suppose you could try to do some sort of runtime casting and duck-typing, but I am not sure how. It would certainly not be a common thing to do, at all! Instead, the type would reflect exactly how the object is used in the function. It would be more explicit, more predictable and safer.

And, harping back on the same theme, it would be fully inferred for you.

The best of both worlds, really.

> Wow, this is pretty unfortunate. Also a very good example of why so many people maintain that Go is not nearly as statically typed as languages like Haskell or OCaml

I'm confused why "so many people maintain that" -- not because its a position I disagree with, because its not really a controversial position such that you'd need to "maintain" it.

Go is, pretty overtly C-like, but with some language-level concurrency features, plus type system features -- interfaces and methods and reflection -- to support OOP (though its definitely not a class-based OO language) and dynamic typing, plus GC, plus filing some of the sharp edges off of pointers.

It doesn't even begin to pretend to be like OCaml or Haskell in terms of strict typing.

Thanks for the informative comment.

> So it would automatically realize that the function called the close method, and infer the appropriate type.

So, if i understand this correctly, OCaml would infer that the function takes "an object with write() and close() methods" because it uses both methods in its definition. That's reasonable. But wouldn't that prevent it from accepting objects that don't have a close() method?

Wouldn't the proper type for the parameter of this http library be something like "an object with a write() method or (an object with write() and close() methods)" (i should have used another notation for this hehe)?

I think the biggest problem in this case is no so much the weakness of Go's type system, but the design decision of mixing two different behaviours (that require two different type signatures) in the same method in the http library. It seems there should be one method that takes a Reader and only calls Read() and another method that takes a ReaderCloser and calls both Read() and Close(), it can even delegate to the first one (assuming, of course, that Go accepts Reader as a valid subtype of ReaderCloser (i would hope so!)).

> (assuming, of course, that Go accepts Reader as a valid subtype of ReaderCloser (i would hope so!)).

Yes it does: http://play.golang.org/p/OnBotL9zfF

Thanks for the example.

Playing with the example a little bit left me with a nice feeling about Go's structural typing:

- Passing a ReaderCloser reference to a function that expects a Reader is possible: http://play.golang.org/p/PZCKvmp5Sj

- Composing interfaces, though very useful, is not required in order to make one a proper subtype of the other: http://play.golang.org/p/fXn0zzM60D

You're right; OCaml would have to supply two different functions, one that takes an object that exposes < close : (); write : string -> (); .. > and another that takes < write : string -> (); .. >. Reproducing the Go system would require the ability to, at runtime, see whether an object of type < write : string -> (); .. > can be downcasted to < close : (); write : string -> (); .. >, which would require more runtime type information than OCaml currently supplies. Some people have made proposals to implement such downcasting (throwing an exception if the downcast fails), which would permit the Go-like implementation, but as far as I know, such proposals haven't been met with much enthusiasm.

Thanks for the examples of record types in OCaml.

> but as far as I know, such proposals haven't been met with much enthusiasm.

Well, if the proposed use cases are similar to this http Go library, then i think i can see why that's the case. As just adding a close: () method to a type could mean that code that uses that type now behaves differently.

I'm not sure if this is what you're talking about, but if you want to see an example of "runtime casting and duck typing" in Go, have a look at my torrent tracker project:

https://github.com/drbawb/babou/blob/master/app/filters/flas...

What I've linked to is a filter that gets put in front of a controller. -- To keep some modicum of type-safety while making use of interfaces, I've added these TestContext() methods to my interface.

Basically, when a controller is registered with one of these chains, the entire chain is checked to make sure that its dependencies are satisfied. (For example, this "flash" filter requires the "session" chain, because it uses session storage for the message being displayed to the user.)

The controller has a similar test; and these runtime penalties are only incurred once when the route is registered. So the app will panic immediately if the types needed for the chain to work aren't present.

It actually works out quite nicely: but you _need_ to write these tests or it falls flat on its face. -- I like to approach it as a form of test-driven development whenever I add a new filter or controller.

The Golang team (or at least, Russ Cox) agrees with you:

https://groups.google.com/d/msg/golang-dev/o9i_SbGRrzI/SSuqW...

This issue was recently discussed on the development mail list: https://groups.google.com/d/msg/golang-dev/o9i_SbGRrzI/gWvnB...

The fix is to document that Close is called: https://codereview.appspot.com/11432044/

Yes, Roger Peppe, one of the Go contributors, had exactly the same issue, but luckily Brad and Russ, two core devs, were able to solve the mystery immediately and updated the documentation.

It's now quite funny to read a blog post from someone completely different (Tim Penhey) who had encountered and solved the same issue with exactly the same usecase on the very same day, complaining about Go without even mentioning the discussion [1] or the latest changeset [2] that solved the problem.

[1]: https://groups.google.com/d/topic/golang-dev/o9i_SbGRrzI/dis... [2]: https://codereview.appspot.com/11432044

That is pretty funny, but it doesn't really change anything. It's unfortunate that it called Close() without at least giving proper notice, and most people will probably agree that in a perfect world where we could start over without breaking APIs, the implementation would be different.

I was just criticizing the publishing skills of some people... The problem itself is unfortunate and I am personally not a huge fan of the "no API changes" philosophy participated by the Go team. I can understand the reasons behind it, but it's still hard to read several CLs per day that are abandoned or done differently (in a non-optimal way) because of this contract.

It's much better for the core team to compromise a little than force everyone to update their code all the time.

In particular, small semantic changes that can't be caught statically will just break people's code with no warning whatsoever. That's bad.

The end user gets to decide when they upgrade the toolchain so it wouldn't be "no warning" if they wrote about the change in the release notes. I'd much rather they break code relying on undocumented weird behavior, than document the weird behavior and keep it alive indefinitely.

That might be right - in this instance - and the actual cost might (or might not) be low - in this instance, but that's not worth breaking the promise Go has made to developers...that code remains compatible across minor forward versions.

If we let small semantic changes like this in, it would be too onerous to upgrade, and so nobody would upgrade. Go users should not have to read and understand all these tiny changes, look through their code to see if anything breaks, and finally make the correct fix.

It's just not tenable, and we know this from experience (before Go 1 we introduced breaking changes with each stable release).

FWIW, this is the first time we've encountered this bug, and it's been part of Go since it was released. Hardly worth breaking people's code for.

It's not always obvious if a semantic change that's documented in the release notes will break your code. You might not know you use the code, the particular behaviour might not be covered by your test suite, etc. I like the fact I can upgrade to the latest Go version without having to worry things like that will happen.

Why can't this API be fixed, isn't that the whole point of the 'go fix' command?

http://golang.org/doc/go1compat.html

'go fix' was mainly used before Go 1, but it would be useful again for e.g. a Go 2.

Documenting this does not solve the problem.

Its solves the problem that side effects that are not expected from the documentation of the behavior of the function are produced.

It doesn't solve the "Go interface types specify minimum functionality that must be satisfied at compile time but do not limit the methods that can be called on an object received through the interface type at runtime" problem, but then, that's a fairly fundamental Go design decision, not a problem. (Especially in the context of Go 1.x, which has a no-backward-incompatible-changes commitment.)

The claim in the article, and I agree, and the Go maintainers agree, that it is a problem. It will never be fixed, but it is a problem (or, to put it another way, it causes serious maintainability and understandability problems, if this kind of behavior is common across API boundaries).

I also want to point out that I don't think typecasting itself is a bad thing in all cases. But you shouldn't type cast objects that don't belong to you.

> The claim in the article, and I agree, and the Go maintainers agree, that it is a problem.

I don't think that's accurate. What the Go maintainers seem to agree is that this particular use of the capability to use run-time interface querying on the received value to exceed the capabilities of the interface through which the object is received is problematic because it isn't using an interface of similar semantic role, and this is the kind of thing that would likely be fixed now but for the commitment not to make breaking changes in Go 1.x.

They do not appear to agree that "Go interface types specify minimum functionality that must be satisfied at compile time but do not limit the methods that can be called on an object received through the interface type at runtime" is a problem, just a feature that, while provide a valuable benefit, can also be used in less-than-ideal ways (of which this is an example.)

> But you shouldn't type cast objects that don't belong to you.

On this point in particular, I see no evidence that the Go maintainers agree.

Roger Peppe and Tim Penhey both work on Juju at Canonical. One of them wrote to the mail list regarding the issue that they encountered. The other vented his frustration on his personal blog.

This is caused by an unfortunate bit of code in the standard library, but is hardly representative of a fundamental problem in Go as the article implies (unless you view casting and interface coercion as a fundamental problem, and I know some people do, but that's basically a different debate since Go does not pretend to be a purely statically typed language by any stretch of the imagination).

I've been using Go regularly for nearly two years now and haven't run into an issue quite like this either with the http package (what he's doing that causes this to be a problem is legal and should work, but is not a common use case) nor with any other bit of the standard library.

It depends on what your definition of "fundamental problem" is. If being able to treat a value differently than it was received, or to perform computations that you shouldn't be able to, e.g. impure actions inside a pure function, then very few languages satisfy that property. Even Haskell isn't safe because of unsafePerformIO--you'd need Safe Haskell, at the very least.

Go has never tried to be, and the authors have never tried to give people the impression that it is that kind of language. Some of the safety comes from the language e.g. doing array bounds checking, not letting you manipulate memory manually, etc., but the rest comes from discipline and idiomatic code. (Same thing as not using 'undefined', 'fromJust', 'unsafePerformIO', etc. in Haskell unless you're absolutely sure they won't do harm--they are too useful to remove, even if they are the sources of many headaches.)

I feel like the author missed a great opportunity. "Stopped by Go"

Stop, Go

or, in "explosive tabloid headline style":

Go: Stop!

Do not pass Go...a ReadCloser?

This is pretty interesting, but I wonder if it's just a one-off problem with the http library.

It'll be interesting to see if someone familiar with Go has a response to this, but from the blog post it doesn't appear that this contract is being habitually violated, only that it can be, which is still a problem in practice.

It's a bit insidious since even if that situation is rare, just knowing that it's possible introduces a new dimension of doubt next time things go awry with any code you didn't write. Enter a "Seriously, No Groping" type modifier ;)

> The standard http library was calling Close on my io.Reader. This is not expected behaviour when the interface clearly just takes an io.Reader (which exposes one and only one method Read).

So if I implemented my own object with a Reader interface and passed it to this HTTP library, could it find some other method on my object named Close() and it could call it?

Is it pure duck typing happening here, or is the HTTP library looking for a commonly-defined interface with documented semantics for Close()?

In Go, a type doesn't have to declare that it implements an interface; it automatically satisfies an interface if it implements the appropriate methods. Most interfaces in Go only contain one method.

If a type has a Close() method, then it is a Closer. The code that tries to cast it to a Closer will succeed. So to answer your question, it doesn't matter what the semantics of your Close() method are, it will succeed solely based on the fact that your type has a method with that name.

> your type has a method with that name.

I would say with the same function signature, not just the name.

Go doesn't support overloading. Method dispatch is done by name only.

http://golang.org/doc/faq#overloading

It doesn't work :) http://play.golang.org/p/PVS5sarmHb

But interface satisfaction is by signature, not name, and that is what is being discussed, not method dispatch.

Sorry, I didn't mean to conflate the two. Thanks for the clarification.

> So if I implemented my own object with a Reader interface and passed it to this HTTP library, could it find some other method on my object named Close() and it could call it?

Yes, and that's what's happening here. It can try to convert your value to a value of type Closer, which is an interface that implements a method Close(), and it can use that method. This conversion will succeed if the value you pass in has a method Close() (taking no arguments) -- Go has a strong "loose coupling" philosophy; there is no "implements."

For example, you can do:

nfoo, ok := foo.(myCustomInterface)

where foo is any value. If foo implements myCustomInterface (indicated by ok being True), regardless of whether that interface is exported or not, or whether the person behind foo intended to implement it, then you will be able to call myCustomInterface's methods on nfoo. It's not exactly duck typing, but it does mean that taking a value implementing SomeInterface is no guarantee that you can only use the methods defined by SomeInterface.

See my explanation here: https://news.ycombinator.com/item?id=6060627

Even if there is agreement that this is a problem, it strikes me that it can become a problem.

The standard library should be exemplary, what do you think would happen in the hands of a developer who's "not as smart as he thinks he is?"

Wasn't that part of go's pitch? I never really considered that libraries would go so far as to inspect the types for methods rather than just make those methods a requirement. The fact that they are able means someone else is going to use it, and it's going to bite me. Don't like.

What was part of Go's pitch, exactly? And where did you see it pitched?

People outside the Go team have tried to pitch Go as a lot of things, most of them having no relationship to what Go was actually intended to accomplish.

The FAQ[1] may be enlightening. Nowhere does it say Go is supposed to be a straightjacket that makes bad programmers into good ones.

[1] http://golang.org/doc/faq#What_is_the_purpose_of_the_project

You'll have to forgive me, I'm at work right now so I can't pull up the talk Rob Pike did on Go, I think it was 2011.

Around the point where he explained Go was not a language to expand the boundaries of computer science. It's for programmers who "often are not as smart as they think they are"

I'll look it up when I get the chance. That might be later tonight.

I'm reasonably confident you're assigning too much significance to what was probably an offhand remark in a particular context. I just looked at slides from a 2010 talk[1] by Pike which was pretty explicit about what they were doing in this area:

"It's a form of duck typing, but (usually) checkable at compile time. It's also another form of orthogonality."

And this one from Russ Cox in 2010[2]:

"Compiled, statically-typed languages (C, C++, Java) require too much typing and too much typing:

* verbose, lots of repetition

* too much focus on type hierarchy

* types get in the way as much as they help

* compiles take far too long"

A common theme for Go from the beginning has been the balance between type safety and not getting in the way of the programmer. The analogy to duck typing has been a consistent observation throughout Go's public existence. This is incompatible with the sentiment you took from whatever talk you saw.

Go's architects didn't set out to make a Java clone. It's more like Java's evil twin.

[1] http://talks.golang.org/2010/ExpressivenessOfGo-2010.pdf

[2] http://talks.golang.org/2010/go_talk-20100112.html

Anyone proficient in GO would care to provide the minimal code that illustrate that extremely weird behavior ? I find that post so disturbing that i still believe i've missed something.

An example:

http://golang.org/src/pkg/io/io.go?s=11741:11801#L336

Line 336. io.Copy takes a writer and a reader, but actually asserts whether they implement ReaderFrom/WriterTo, then use their methods, ReadFrom and WriteTo, if they do. (Same thing with the http package checking if the reader implements Closer, and calling Close if it does, though admittedly 'we should be able to call Close' is a much more unsafe assumption than 'if it implements ReadFrom, and it has the right type signature, we can just call that function instead'.)

The reason it doesn't take values implementing the interfaces ReaderFrom and WriterTo is that io.Copy works even if you don't implement those interfaces.

It's an unfortunate and regrettable predicament, but it's not something that comes up a lot. As others have mentioned in this thread, Go certainly isn't a language with referential transparency by any means--the type system provides minimal sanity checking, and the language itself is memory-safe, but it's not trying to be a "high-assurance language."

This io.Copy behavior seems like an "optimization"... less explicit than I would like but sticks to the idea of an io.Copy.

Having a resource you own Closed on you isn't an optimization, it is a behavior change.

I agree. I think I edited my comment to specify that it's a much more unsafe assumption just before you posted your comment.

That is SO disturbing... They don't even check whether the parameter implements another interface, they do the check directly on the method by callin it... That's so ugly...

Reminds me of my surprise when i realized the math module only worked on float types, and that min/max only existed for some numerical types but not others. The thing in itself probably isn't a big deal, but it really questions the ability of the language to model complex relationships often found in business processes.

> That is SO disturbing... They don't even check whether the parameter implements another interface, they do the check directly on the method by callin it... That's so ugly...

That's incorrect. The code you see at the top:

    if rt, ok := dst.(ReaderFrom); ok {
            return rt.ReadFrom(src)
    }

means: "If dst implements ReaderFrom, then call ReadFrom on this value of that type."

You can't arbitrarily call methods that may or may not exist. You can check if a value implements an interface by doing "newVal, ok := val.(suspectedInterface)", and checking if ok is true, or by doing "newVal := val.(suspectedInterface)" and living with a runtime panic if it doesn't implement the interface, then call a method in that interface on the new value newVal, which will be of type suspectedInterface. (If val doesn't implement suspectedInterface--i.e. ok is false--then newVal will be the zero value of suspectedInterface.)

yeap, i read it too fast. Didn't notice the "er" after dst. thanks for clarifying.

Does Go not provide method overloading? Couldn't they have 3 versions of the copy method?

> Does Go not provide method overloading?

Yes, it does not permit method overloading.

I am just wondering whether you can write your own wrapper class that implements (exposes) the read method and doesn't expose a close method (or expose a close method that does nothing) and pass that in?

I am not suggesting this is a good solution, just curious.

Yes, easily:

    var rc io.ReadCloser // a file, whatever
    readerOnly := struct{io.Reader}{rc} // only a Reader

That's cool. The io.Reader is an unnamed field, so its method is effectively pushed to the surface of the wrapping struct instead of needing to be explicitly forwarded to.

Why dont the Go team do golang2 now.. There are a few issues.. already mentioned in open fire chat this year 2013.. Would be nicer to actually recongise flaws now.. and offer alternative.. even faster tht python 3000 could ?

Thats whats in my mind.. and fix the the documentation so the classes interlink and it a bit of a java/md/rst/doc style..

> Why dont the Go team do golang2 now.. There are a few issues..

I suspect that it is because Go 1 (issues and all) is useful enough that people are making productive use of it, and they'd like to flush out more of the issues that may exist -- and give more time for different approaches to those issues to be tried in experimental forks and, where possible, libraries, preprocessers, etc.

> and fix the the documentation so the classes interlink

Not sure what this means, and Go doesn't have classes. (Its not just an issue of calling classes different names, it divides up functionality differently than class-based OO systems -- packages are in a sense like classes [they are the closest equivalent to the function of classes as containers of static methods], structs are a bit like classes [in the role of containing members, and in that anonymous embedding approximates inheritance], every type is a bit like classes [in that any type can have methods attached to it, so long as they are attached in the same package as the type is defined]; but all these things are also unlike classes in various ways, too.)

for those not familiar with Go, here's the problem translated in C terms: "I passed a pointer to your function and it called free() on it. how dare you!"

or "I passed an fp to your function, and you called close() on it. How dare you!?" :)

More "I passed a pointer to your function and it called free() on it, without documenting that it was taking ownership of the memory". Ownership contracts are an important part of the API of any function that does stuff with resources, whether memory, file descriptions, network connections, etc. If the API implies that it takes a pointer to a resource but will only read from or write to it, and then disposes of it, that's a bug.

I think that's a good point. It's often said that garbage collection helps prevent memory leaks, and it's also said often that memory isn't the only kind of resource which could be leaked.

Here is another case where weak- and dynamic-typing language features have come up in the context of resource management.

I'm not saying this one example proves anything, just that it might be interesting to be on the lookout for other events of this pattern.

No, that's not what the problem is. Translated to C++, the problem is "you passed a pointer to A∗, you dynamic_cast<>'d it to a subtype B∗ without documenting that you did so".

Not really. You can't dynamic_cast across across protected or private inheritance boundaries, even if the function attempting it is a friend. As long as your public inheritance hierarchy maintains your contract (and they should) you're fine.

This behavior is explicitly defined in the godoc for io. I don’t see why the author should be "stunned" by a function doing exactly what it says it does.

That'll be because it's only been in the docs since they reported their issue.

You're both correct. It's been in the docs for io.Copy for a long time, and was only recently added to the http docs.

So what's the fix for this? Wrap the object in another that only defines Read? Or is there another way?

Supporting typecasting is hardly earth-shattering. Pretty much every statically typed programming language in actual use supports typecasts. That includes Scala, Java, C++, C, C#, Pascal, Algol, and even Ada.

This title should be "stunned by shitty poorly documented library function" not "stunned by Go."

There are a few languages out there that don't support typecasting. I think SML was one, for example. But they made a lot of other design choices to be able to support that choice, and so far that set of choices has not gained popularity. At minimum, you would need a fairly complex type system with generics to be able to completely forbid typecasting. You also run into problems where stable APIs completely block progress in a particular area.

> There are a few languages out there that don't support typecasting. I think SML was one, for example.

Haskell too, unless you explicitly opt in with Data.Typeable.

It's arguably true in a certainly commonly-used subset of C++ as well: this type of typecast doesn't work in C++ unless you use dynamic_cast or another RTTI system (like COM). dynamic_cast only works on classes with a vtable. Many C++ projects don't use any form of RTTI. (Note that the Go code does not assert that the Reader is a ReadClose; rather it depends on RTTI to determine whether the reader is a ReadClose.)

dynamic_cast downcasts in C++ only work across public inheritance. In particular a object that inheritance a Closable interface privately and a Reader interface publicly wouldn't allow a cross cast.

dynamic_cast isn't a loophole in the type system, it's there to strengthen it.

I don't think I've ever seen C++ that inherited an interface privately.

That said, it's extremely common for classes to lack any virtual functions and thus be unusable with dynamic_cast.

Well protected is the default.

The default is private for classes and public for structs.

Section 11.2 of the C++ standard.

By default it is private.

Usually private inheritance is mostly used for library code, as a way for code reuse but without exposing the inheritance relationship to the class users. Also coupled with mixins sometimes.

It's true that RTTI isn't used very much in C++, but I can't think of any non-trivial C++ project that doesn't include some traditional unsafe typecasts. And I have worked on a lot of C++ projects. And then there's the implicit type coercions, and the implicit constructors... ah, the good old days.

Also, used normally, the best you're going to get out of Data.Typeable is cast :: a -> Maybe b which means that if your two types don't have runtime equivalence you'll be handled a statically ensured runtime Nothing that you must handle gracefully. The semantics are extremely clean.

dynamic_cast only works on classes with a vtable This isn't technically true - theoretically, you could implement C++ without vtables whatsoever. Furthermore, you could use a static_cast if the pointer had originally been cast from something lower down the hierarchy.

> Supporting typecasting is hardly earth-shattering

I think the problem is the standard library using it extensively, apparently without doing it properly. As an example, C++ has casting but the STL doesn't need to use it (much) because it has generics in the form of templates.

I happily use a coercion-free language every day: Haskell. I don't miss coercion at all. Being able to trust the type contracts that each function portrays is an incredibly important part of the confidence that Haskell gives a developer, team, or leader.

Go didn't fail you, OOP failed you.

There's nothing about OOP that requires weak typing.

I was referring to the fact that this function mutates the state of one of its arguments.

But so does Read(), right?

Even in Haskell I'd be calling the HTTP thingy in the context of the IO monad.

<tldr>App author found a nasty example of a common antipattern in library code which caused a bug in his app. Documentation was updated to point out dragons.

Reply on news.ycombinator.com