Hacker News Reader

Hacker News

Comment by emmelaich | Hacker News Reader

Parent Full thread

emmelaich

I'm a go newb, but would you do this with channels and gofuncs now?

The readability this provides is considerable.

View on HN

That kind of suffers if you want to be able to cheaply cancel the iteration.

Lets say the logic is:

  for item := range list {
      if condition(item) {
          break
      }
  }

That is nice and readable, but if the thing your iterating is returning items in a batch then this gets far, far uglier. Its possible to end up with one loop that fetches x items at a time, and another that iterates over those items.. etc. But if you switch that to channels you maintain the readability and everything is nice! .. Except, you need a way to cancel the iteration, and it can't be closing the channel or the writer will panic.

  items, cancelFunc := apiCall()
  for item := range items {
      if condition(items) {
          cancelFunc()
          for _ := range items { }
          break
      }
  }
  close(items)

Now, this is even worse if you need some kind of finalization of the goroutine that spawns to write to the given channel. Now you have a wait group, or a second channel.. etc. If its possible to error in the middle of iteration then things get complicated as well.

I may not understand what you're looking for, but what's wrong with this:

    topLoop:
    for _, batch := range list {
        for _, item := range batch {
            if condition(item) {
                break topLoop
            }
        }
    }

The language also supports a limited goto. Combined with select and context, I've never seen a situation I felt I couldn't adequately describe.

I would probably just do the following:

    type Iterator[T any] interface {
        HasNext() bool
        GetNext() T
    }

And then use it like:

    for i.HasNext() {
        current := i.GetNext()
        // do things here
    }

It's an extra line, sure, but it's pretty obvious what's going on.

This is the way how Java iterators work. And having worked with them for many years I must say - please don't go that route. Having separate hasNext/next methods is terrible for implementers of the iterator. The problem arises when you need to actually compute the next item to learn if it exists. So then your hasNext is actually effectful and needs to cache the result internally. There are so many ways to get it wrong, I've even seen some terrible implementations which required the caller to call hasNext or otherwise they would not advance the iterator.

Iterator design is another area where sum types would help simplify things a lot. Just one method returning Some/None would be so much better.

> Iterator design is another area where sum types would help simplify things a lot. Just one method returning Some/None would be so much better.

It would be better, but since go has MRV and zero values

    func Next() (hasNext bool, nextValue T)

works fine, this is not Java or C# where you need to split the protocol into two separate operations.

Since we're talking about implementing it in usercode, as opposed to adding it to the language, we fortunately have the option to rework it (or have a couple completing implementations to see what works in practice).

If you're looking for those particular features, I'd recommend adopting go's `, ok` idiom that appears in several places in the language. A type like:

    type Iterator[T any] interface {
        Next() (T, bool)
    }

Lets you write code like this, if that's more to your liking:

    for current, ok := i.Next(); ok; current, ok = i.Next() {
        // do things here
    }

Which, really seems like a more go-like way to do this in the language as well (ranging over a function then becomes syntactic sugar for the above).

And make a simple iterator into a multithreaded program with channels and goroutines?

Slower. More complex. Why?!?

Just call your function in a loop.

Which is to say: yes! Many do it with channels currently.

It's awful, but they do it.

> Slower

Why would that be slower?

The design of channels is constrained by the need to support concurrency. If you do not need concurrency, using something designed to support it will be suboptimal. And because it's specifically concurrency, the overhead will be substantial.

... that's the theory behind it. For more grounded explanations, see [1] & [2].

[1] https://news.ycombinator.com/item?id=29510751

[2] https://ewencp.org/blog/golang-iterators/index.html

Yeah, I kind forgot we don't have anything concurrency related here.

Thanks for the links!

Channels and the goroutine scheduler need to be thread safe. They use locks and/or synchronized instructions that run much slower because they need to synchronize between the different cores of your CPU (or between CPUs, if you have more than one, which is slower still.) Running goroutines also saves and restores registers, trash part of the cache, or move to another core with a cold cache. Threads have more overhead, and processes more still, but that still is very slow.

Thats how I would do it. You can have a goroutine send on a channel (buffered or non buffered), and then another function can consume the chan as needed which lets the goroutine generate results as needed as well.

As I recall, the performance of that approach is quite bad, even with Go's concurrency support.

Reply on news.ycombinator.com