Hacker News Reader

Comment by imiric | Hacker News Reader

I'm also not seeing why this is necessary. Range over int is pure syntax sugar, and while range over func could be useful, iterators can be easily implemented with current Go features.

I love Go because of its simplicity, and because it strictly adheres to the "there's only one way to do it" mantra. Adding syntax sugar muddles this, and introducing another keyword and construct makes the code less readable.

I hope that this proposal is rejected, and that it's not a sign of Go becoming yet another kitchen sink language.

View on HN

> iterators can be easily implemented with current Go features.

It's rare that can is relevant when it comes to language design. You typically can do anything in any language.

Adding dedicated support for iterators would promote it as the way to iterate. That would actually simplify the language - one way to express a common need as opposed to the current mess of ~5 different ways [1]:

"In the standard library alone, we have archive/tar.Reader.Next, bufio.Reader.ReadByte, bufio.Scanner.Scan, container/ring.Ring.Do, database/sql.Rows, expvar.Do, flag.Visit, go/token.FileSet.Iterate, path/filepath.Walk, go/token.FileSet.Iterate, runtime.Frames.Next, and sync.Map.Range, hardly any of which agree on the exact details of iteration. Even the functions that agree on the signature don’t always agree about the semantics."

Go would have been a far simpler language had it originally shipped with generics, iterators, and sum types.

[1] https://github.com/golang/go/discussions/56413

I agree with you to an extent. But adding more features is always a tradeoff between introducing complexity, and supporting a use case that might not be useful in all programs.

For example, generics were a highly controversial topic, and many people still believe that Go didn't need them. I'm partially in that camp. I've encountered maybe one or two situations where generics would've been convenient since Go 1.18, and in both cases not jumping straight at the opportunity to use them, but being forced to refactor and approach the solution in a different way, has produced simpler and more readable code. To this day, I struggle with Go's generics syntax, and it just looks alien to me.

Iterators are a useful feature, but are they really generally useful to deserve a change in the language spec? Like generics, I've yet to encounter a situation where they're truly required, and when I do, implementing them myself is trivial.

So, sure, language designers can cram every programming construct we've invented in the last 50 years, as Rust and Raku have done, and many programmers will find this very convenient and flexible, but a more focused and conservative approach produces a simpler language that is easy to pickup and read. This is what drew me to Go in the first place, and lately it seems that this is changing.

Iterators have been around for literally decades at this point, and have proven their usefulness in every language they've been introduced in (just like generics). It's not some kind of experimental feature.

Generics very nice as a library writer but in my own application code, I usually have no uses of them.

You have language features which are not needed for the ones using directly (for the lack of better term) the language and but really nice for the people providing building blocks for them.

I am using generics in other languages than Go but not in Go as I am normally not writing libraries in Go or they are already so specific that generics are not needed.

> Go would have been a far simpler language had it originally shipped with generics, iterators, and sum types.

You forgot tuples.

First-class tuples would make this code kind of unpleasant:

    func F() (A, B) { ... }
    func G(a A, b B) { ... }
    G(F())

Is the return value of F a tuple? Is the parameter list of G a tuple? Given H:

    func H(ab (A, B)) { ... }

How and when is H(F()) different from G(F())?

(Before you suggest a Python-like star operator, personally I consider using it for non-homogenous arguments about as hygienic as keeping your toothbrush and dirty socks in the same drawer.)

> First-class tuples would make this code kind of unpleasant:

Which assumes this code matters much, outside of errors for which intermediate processors can just take... a tuple of a value and an error.

Also assumes the magical splatting of MRVs is a good thing in the first place.

> (Before you suggest a Python-like star operator, personally I consider using it for non-homogenous arguments about as hygienic as keeping your toothbrush and dirty socks in the same drawer.)

That is literally what Go currently does for MRVs, except opt-in instead of magical.

There is nothing about such splatting which wouldn't work with proper tuples, if anything it's the opposite. It is, essentially, the value side of `uncurry`.

I don't necessarily disagree, but it depends on your goals as a language designer. I think it's OK to sacrifice a niche feature for improved readability in the common case. Anonymous structs are not as fundamentally useful as e.g. anonymous functions.

> Also assumes the magical splatting of MRVs is a good thing in the first place.

IMHO, G(F()) reads nicer than some variant of G(*F()) or G(F()...). Lua does this too, and is (not unlike Go) considered a simple and elegant language.

> Which assumes this code matters much, outside of errors for which intermediate processors can just take... a tuple of a value and an error.

Then you'll sooner or later find yourself writing a G((a, b)), which again, I think is a net loss to readability.

I think my argument would've been much stronger if Go had had generics in 1.0, and the stdlib/ecosystem wasn't already full of MustParse's and MustCompile's. But in that case we could've had a Result type as well.

I would rather have type inferred literals: https://github.com/golang/go/issues/12854

> Adding dedicated support for iterators would promote it as the way to iterate. That would actually simplify the language - one way to express a common need as opposed to the current mess of ~5 different ways [1]:

There is one. Return a channel. Just a bit iffy when you want to stop iteration in the middle and throw away the rest.

Returning a channel is not really a cheap operation. They internally have locks that slow everything down as they are intended mostly as a safe concurrency primitive

And in this case there is really no need to spawn goroutines to support iterators

because go developers are sick of being pure and like to type shorter things (and I don't mean variable names) to get their jobs done.

programming is a lazy hobby. it's why as soon as generics were released, I immediately implemented 'Keys', 'Values', 'ToSet', etc. Writing multiple lines to do the equivalent of these functions might be 'pure' but it's not easier to read. Worse: instead of standardizing these common operations, you just have everyone rolling their own implementations of the exact same functions with slightly different naming/parameters/etc.

Am I confused in that there is some merit to this approach beyond dogma? Why do we, as a community, need to accept so few batteries? If you want to only use assembly in your code (the purest!), that's always an option.

Well, Go authors decided that adding features to the language comes with cost to every user (which is correct) and decided to be careful with that.

Which is, looking at absolute hulking abomination C++ or Java has become, is the right approach in theory

They just made a massive miss with lack of generics and sum types on the start. They could've avoided nils entirely with that for example. Or have less annoying error handling.

> They could've avoided nils entirely with that for example. Or have less annoying error handling.

TBF neither generics nor sum types were necessary for either, these could have been builtins or language-level features.

To start with it's not like Go has been shy about those for things they didn't want to formalise (yet), but even in languages where they're library-ish features it's common for them to have language-level support.

It's not about purity or dogma. It's about keeping complexity out of the language as much as possible, which is much harder to do. Language designers always make a tradeoff between making programming tasks convenient, and introducing more complexity that makes the language more difficult to maintain and less approachable, and programs written in it more difficult to read and reason about.

Being productive in a language is not just about using every convenience at your disposal, and typing less. It's about producing code that is easy to read and approachable for others or yourself years down the line. The more language features one needs to be aware of when writing or reading code, the less approachable it is.

Having less features also forces the programmer to rethink their solution in a less clever way. In dynamic languages like Python and Ruby it's very convenient to use some fancy trick to write less code. In languages like Rust or Raku, you can (ab)use some obscure feature in a non-intuitive way, and feel good about the "elegance" of the solution, but a fresh pair of eyes might struggle to understand what the code actually does. Considering reading code is much more important than writing it, as we literally write code for humans, not machines, optimizing for simplicity and having less features in a language is a more productive long-term approach.

> It's not about purity or dogma. It's about keeping complexity out of the language as much as possible, which is much harder to do. Language designers always make a tradeoff between making programming tasks convenient, and introducing more complexity that makes the language more difficult to maintain and less approachable, and programs written in it more difficult to read and reason about.

Complexity is not something you can remove.

Complexity removed from language moves to the application.

The best thing you can do in the language is to make expressing complex concepts in easier, more clear way. Going too simple just produces more boilerplate and makes it harder to read.

Keeping complexity out of the language does not remove the complexity. It moves it into the programs. MIPS assembly is a very simple language with specs fitting on 10 pages. Yet you probably wouldn't want to write a web application in it. Brainfuck is even simpler as a language, but the programs are even harder to understand.

sometimes it moves the complexity up the developer's head : "are you sure you really need to do all this, and there isn't a conceptually simpler solution to your specific problem ? Think again !"

That's the real value of "simple" languages like go.

“Am I really sure I want to loop over batches instead of fetching one record at a time (or the whole set of results at once)? Ugh, it’s a complete pain in the ass though, one at a time it is.”

You literally don’t have to use iterators if you don’t need them. You can just range over a collection as you do already. But when you want something a bit more performant like looping individually over batches or generating an enormous sequence of records on the fly, they make what would either be tedious or difficult into something trivial.

Go being “simple” just makes application code unnecessarily verbose and makes dealing with even moderately complicated problem domains feel like solving novel computer science problems when it’s be four lines of boring code in better languages.

Aren't we talking about standardizing iterators? That makes the compiler slightly more complex, I guess, but for the language it makes it much easier to read and more approachable imo. That's the reason that range exists in the first place.

I don't see why the author's implimentation is so complicated. I really just want an interface that works with the range operator, and to call it a day.

    type Iterable[T any] interface {
        Next() (T, bool)
    }

so that

    for _, el := range someIterable {
    }

is essentially the same as

    for el, ok := someIterable.Next(); ok; el, ok := someIterable.Next() {
    }

I certainly think the first is more clear btw.

>because go developers are sick of being pure and like to type shorter things

"go developers" is not a homogenous group. I write Go and am pretty happy with the language design as it is. I'm also fearful of it becoming a kitchen sink language because of a vocal minority who want Go to be more like language X, instead of just going and writing in language X.

There's plenty of things could be added to make code more concise, but they come with a hidden cost.

I too hope this will be rejected, the proposal is terrible. One of the core advantages of Go was, that if newbie colleague comes to the team, he will be able to read and understand the code very quickly. Things like this will over time kill this Go's advantage.

> Adding syntax sugar muddles this, and introducing another keyword

There is no new syntactic sugar nor there is a new keyword here though. It just changes how range works.

That being said, I'm not a fan of range having two modes of functionality. Perhaps it could just use an interface, now that we have generics it can be implemented on the existing supported types. And compiler can do whatever with it if it would be too slow.

This is a textbook example of syntactic sugar, where a feature can be trivially transformed into simpler code that doesn't make use of the feature

Exactly. Go making it's way faster than I expected to be replaced by yet another language that discovers that there's a demand for feature restriction and simplicity. The very thing that made it popular in the first place. It should be rejected but I'm losing hope. I have less and less faith in rsc as time passes.

Why do you have less faith in rsc? Asking because I feel the opposite.

I think the pace has been extremely slow. So many of the changes to Go have been tool chain or library enhancements. Outside of generics related stuff, what are some added core language features that you'd prefer were not?

Bingo

I'm a go newb, but would you do this with channels and gofuncs now?

The readability this provides is considerable.

That kind of suffers if you want to be able to cheaply cancel the iteration.

Lets say the logic is:

  for item := range list {
      if condition(item) {
          break
      }
  }

That is nice and readable, but if the thing your iterating is returning items in a batch then this gets far, far uglier. Its possible to end up with one loop that fetches x items at a time, and another that iterates over those items.. etc. But if you switch that to channels you maintain the readability and everything is nice! .. Except, you need a way to cancel the iteration, and it can't be closing the channel or the writer will panic.

  items, cancelFunc := apiCall()
  for item := range items {
      if condition(items) {
          cancelFunc()
          for _ := range items { }
          break
      }
  }
  close(items)

Now, this is even worse if you need some kind of finalization of the goroutine that spawns to write to the given channel. Now you have a wait group, or a second channel.. etc. If its possible to error in the middle of iteration then things get complicated as well.

I may not understand what you're looking for, but what's wrong with this:

    topLoop:
    for _, batch := range list {
        for _, item := range batch {
            if condition(item) {
                break topLoop
            }
        }
    }

The language also supports a limited goto. Combined with select and context, I've never seen a situation I felt I couldn't adequately describe.

I would probably just do the following:

    type Iterator[T any] interface {
        HasNext() bool
        GetNext() T
    }

And then use it like:

    for i.HasNext() {
        current := i.GetNext()
        // do things here
    }

It's an extra line, sure, but it's pretty obvious what's going on.

This is the way how Java iterators work. And having worked with them for many years I must say - please don't go that route. Having separate hasNext/next methods is terrible for implementers of the iterator. The problem arises when you need to actually compute the next item to learn if it exists. So then your hasNext is actually effectful and needs to cache the result internally. There are so many ways to get it wrong, I've even seen some terrible implementations which required the caller to call hasNext or otherwise they would not advance the iterator.

Iterator design is another area where sum types would help simplify things a lot. Just one method returning Some/None would be so much better.

> Iterator design is another area where sum types would help simplify things a lot. Just one method returning Some/None would be so much better.

It would be better, but since go has MRV and zero values

    func Next() (hasNext bool, nextValue T)

works fine, this is not Java or C# where you need to split the protocol into two separate operations.

Since we're talking about implementing it in usercode, as opposed to adding it to the language, we fortunately have the option to rework it (or have a couple completing implementations to see what works in practice).

If you're looking for those particular features, I'd recommend adopting go's `, ok` idiom that appears in several places in the language. A type like:

    type Iterator[T any] interface {
        Next() (T, bool)
    }

Lets you write code like this, if that's more to your liking:

    for current, ok := i.Next(); ok; current, ok = i.Next() {
        // do things here
    }

Which, really seems like a more go-like way to do this in the language as well (ranging over a function then becomes syntactic sugar for the above).

And make a simple iterator into a multithreaded program with channels and goroutines?

Slower. More complex. Why?!?

Just call your function in a loop.

Which is to say: yes! Many do it with channels currently.

It's awful, but they do it.

> Slower

Why would that be slower?

The design of channels is constrained by the need to support concurrency. If you do not need concurrency, using something designed to support it will be suboptimal. And because it's specifically concurrency, the overhead will be substantial.

... that's the theory behind it. For more grounded explanations, see [1] & [2].

[1] https://news.ycombinator.com/item?id=29510751

[2] https://ewencp.org/blog/golang-iterators/index.html

Yeah, I kind forgot we don't have anything concurrency related here.

Thanks for the links!

Channels and the goroutine scheduler need to be thread safe. They use locks and/or synchronized instructions that run much slower because they need to synchronize between the different cores of your CPU (or between CPUs, if you have more than one, which is slower still.) Running goroutines also saves and restores registers, trash part of the cache, or move to another core with a cold cache. Threads have more overhead, and processes more still, but that still is very slow.

Thats how I would do it. You can have a goroutine send on a channel (buffered or non buffered), and then another function can consume the chan as needed which lets the goroutine generate results as needed as well.

As I recall, the performance of that approach is quite bad, even with Go's concurrency support.

>> "there's only one way to do it"

What's the one way to do iterators now? I thought this proposal came about because there wasn't a default way and everybody had to roll their own.

you can just return channel. range works over channels. Close channel once you're done with iterator. Technically there is side effect of that (iterator done like that would always produce extra value before exiting) but you could probably work around it without much boilerplate

Channels introduce a ton of overhead (https://syslog.ravelin.com/so-just-how-fast-are-channels-any...), plus concurrency issues, and termination concerns (they are essentially a resource leak). It's nice that you can iterate channels, but they're a terrible iteration protocol.

Go development is becoming more and more KPI driven.

What is KPI in this context?

They probably meant that programming language designers/developers can only advance their careers by adding/implementing features, not presiding over something complete.

Reply on news.ycombinator.com