Philosophy and "for" loops — more from Go and Rust | Hacker News Reader

Philosophy and "for" loops — more from Go and Rust

lwn.net

138 pointsadestefan55 comments

Ack, what bad timing. Rust is in the middle of switching its dominant iteration idiom from internal iterators to external iterators. See also:

https://github.com/mozilla/rust/wiki/Doc-detailed-release-no...

http://journal.stuffwithstuff.com/2013/01/13/iteration-insid...

The `for` construct will be changing in the 0.8 cycle to reflect this change. Currently it looks like:

  for [1,2,3].each |i| { /* a closure */ }

Where eventually it might look something like:

  for i in [1,2,3] { /* just a regular block */ }

I'm actually surprised that the author praises the "freedom and flexibility" of our old `for` loops... the reason for making this switch at all is because we found our old semantics to be neither sufficiently composable nor sufficiently flexible! :)

Hopefully the author will revisit this space once the new implementation is finalized, perhaps with a comparison to D this time.

TL;DR: Rust is switching idioms from Ruby-style iteration to C#-style iteration because (for our purposes) it's more performant, easier to optimize, more composable, and more flexible.

Well, in 0.7 the external iterator `each` pattern is mostly gone, leaving external iterators the standard style, but because the semantics of `for` haven't changed yet, there is an `advance` method provided for iterators, such that the code is actually at present thus:

    for [1, 2, 3].iter().advance |i| { /* a closure */ }

Right, I was hoping we could avoid mentioning this brief and ugly interlude in our evolution. :)

It's been a courageous effort by strcat, pushing such ugliness on us over the 0.7 milestone, but it will certainly pay off into the future. I look forward to the new sugar!

That is basically a while loop without a "proper body", right? At each iteration advance is called with the closure as first argument and iteration continues until it returns false?

You're correct. You can see as much here in the implementation of `advance` (as you'd expect, it's very simple):

https://github.com/mozilla/rust/blob/master/src/libstd/itera...

Rust is actually in the middle of changing how iteration (and eventually for-loops) works, from "internal iterators" as described in the article to even more flexible "external iterators". I'm not fully up-to-date, but this comment (http://lwn.net/Articles/557494/) on the original article links to a mailing-list post on the topic: http://thread.gmane.org/gmane.comp.lang.rust.devel/4528

Are internal iterators still possible?

Internal iterators are just syntactic sugar for a higher-order function. As long as Rust has higher-order functions, internal iteration will be possible (just not "the standard way to do things").

Edit: In fact, this is true of any language with higher-order functions. For example, this is how you'd define a Rust-style internal iterator for lists in Racket scheme:

    (define (iter l f)
      (when (and (pair? l) (f (car l)))
        (iter (cdr l) f)))

And how you'd use it:

    ;; displays elements of the list, up to and including the first one >= 3
    (iter '(1 2 3 4) (lambda (x) (display x) (< x 3)))

but the `break`/`return` syntactic sugar inside for-loop closures will be lost.

Yes. You might be able to get something like it back with Rust's macro system, though. But it's probably better just to use the new syntax & iteration protocol, assuming the Rust folks come up with something reasonable.

To be fair, it wasn't so much "syntactic sugar" as it was "a compiler hack that was rarely used and didn't even suffice for a large class of cases involving borrowed pointers".

And, it is an incorrect compiler hack (https://github.com/mozilla/rust/issues/6675), but that's not saying much at the moment.

As kibwen explains above, Rust is just switching the dominant iteration protocol to external iterators, i.e. the protocol privileged/understood by the `for` syntactic sugar. It will still perfectly possible to write

  do tree.visit_each_leaf |leaf| {
    // ... do things with leaf ...

    if some_continuation_condition { true } else { false }
  }

(The last line is the replacement for what would be currently written `if !some_continuation_condition { break }` with the `for` sugar.)

Great article - I particularly enjoyed the discussion of the relative distributions of different types of loops in the Go standard libraries.

Having said that, I thought the Go `readLine` using channels example was strange. The goroutine that provides lines to the channel actually still needs to loop over the lines, and in fact includes a DRY version of the loop in question:

  for {
    line, ok := file.readLine()
    if !ok { break }
    // ...
  }

This is not to say that using an unguarded loop with a break is necessarily the best way to do this, but it seems to be the simplest DRY way to do this. I think he just wanted an excuse to demonstrate `range` with a channel.

Indeed, further down in the comments (on the article) one of the Go team members pointed this out and the author just called this solution "measurably worse" without any factual basis. It's a pity that so many seemingly technical articles are littered with bias and unfounded opinions these days...

I still have a soft spot for the Swiss Army Knife of the iteration world - the Common Lisp Loop macro:

http://www.gigamonkeys.com/book/loop-for-black-belts.html

My head hurt after looking at how Icon[1] does looping.

1) http://en.wikipedia.org/wiki/Icon_(programming_language)

That looks like (ab)using exceptions for normal flow of control!

It's only "abusing" if it's not idiomatic.

Yes, but I have seen production Java code that used exceptions to return values from methods as part of normal execution - which is what it reminded me of and that most certainly was abusing :-)

Then you might like its big brother: iterate.

http://common-lisp.net/project/iterate/doc/index.html

ITERATE is more powerful than LOOP and extensible.

Ah yes, in describing LOOP:

"one often has to consult the manual to recall lesser-used aspects of the strange syntax"

Guess where my copy of CLTL2 falls open at?

[Unfortunately, it's a long time since I did much CL development]

What is intressting is that the Loop macro never made it into clojure. There has been some discussion but nobody every fully impmented it.

Also even within CL, there are lots of people who dislike it. Also this is just nice suger, the article was really about using diffrent datatypes.

It's a slightly pedantic thing, but since the article is about the syntax of the loops, the author has misused the range loop in most cases.

When you only provide one variable for assignment, you are actually assigning the key (or incremental value), not the value, the syntax is:

  for key, value: := range someThing {

When you do:

  for something := range someThing {

something won't have the value. If you just want the value you need to use:

  for _, value := range someThing {

>range does not work with user-defined types at all

that's close to true but not strictly true, since you can define your own types using slices, maps and channels. The reason you would do that is that by defining your own type, you can give a slice, map, or a channel type a method set, which by extension means that you can create a slice, map, or channel that satisfies an interface. E.g.: http://play.golang.org/p/04IBK5OwNk

>While interesting new control flow is unlikely to be a headline item on a newly developed language these days

...ummm, goroutines? I know that's not "new" in the sense that CSP has been around since the 70's, but the `go` keyword is a new control flow mechanism for many programmers. Concurrency is a control flow concept.

> Concurrency is a control flow concept.

Couldn't agree more. Also a list of all flow control constructs that doesn't include destructuring pattern match ain't a complete list in my book. if and switch are just special degenerate cases of pattern-matching.

In golang I like that there is only 1 way to do a loop. Some people coming from Ruby or other lang that have a lot of options to do loops might looking a little strange or feel like a step backwards but it truly is pretty awesome. In less then 15 lines I can express 100% of Golang.

I haven't used Rust so I don't know how its done in Rust but it looks promising can't wait to play around with it as well.

It's interesting that there seems to be a lack of appreciation for having a single way of doing things. I think there are very few applications for which the performance gains of additional flexibility are significant, whereas the productivity and test simplicity can be more significant for non trivial apps.

on the internet, it seems that there is a lack of appreciation for having a single way of doing things, but in the real world... it doesn't really same to be the case. The appreciation is there, it's just not as vocal, because it's... well, not as interesting to talk about, to be honest. How much of the world's software is written in C++? How many HN articles are about C++? HN is representative of HN, not programming in general.

It is ironic that you mention having a single way of doing things and C++ in the same breath.

The reason there are multiple ways of doing things in nearly all languages is that people have found that not all common use cases fall nicely into one construct. A language can either be plauged by everyone reimplementing some common constructs or it can add some of those constructs as "another way of doing things".

I don't really get this thread at all actually. Most of the article was about how Go actually has three looping constructs, so the entire premise seems ...off.

It's not about performance gains; it's about making it harder to mess up. C-style for loops are easier to get wrong. (Try iterating backwards from 10 to 0 with an unsigned int loop index in C; most programmers get it wrong the first time due to underflow.) They are also more vulnerable to race conditions. (See "race on loop counter" at [1]; with first-class iterators you can create a separate binding for each iteration of the loop, eliminating this problem.)

[1]: http://golang.org/doc/articles/race_detector.html

I don't mind if there's a single way of doing things (although I prefer when there's an escape hatch). But plenty of languages have a single way of, say, iterating over the lines of a file that doesn't involve writing the same expression twice and feeling straitjacketed.

Using the same keyword in multiple forms is fine but I don't think that qualifies as "one way to loop". What the range-ified form can do is pretty distinct from the others.

Almost like there could be some kind of "List" monad >_>

(Or syntactic sugar for map/foreach)

Inaccurate article! `for {}` in Go acts like a `while (1) {}` loop in C, but the article claims that another form of for loop is the "final form of for loop" for Go.

Otherwise well written and enjoyable.

C also has an empty for loop that works like while (1):

for (;;) {}

My brain always thought that was a no op, but the rule says infinite loop. Why would void be true? Not sure, but there likely was a good reason.

Maybe because true is defined as anything that's not false?

This is also called an "infinite loop"?

djb uses this a lot.

I think the author is seeing things in C's for loop construct that aren't there. C's syntax is conspicuously clean of unnecessary abstraction. The for construct is just three expressions grouped together for the sake of being convenient. It's not a coroutine. The head of the loop can't be a separate context, or part of the loop's scope: it's not inside the curly braces (which aren't part of the loop, anyway; you can use them anywhere -- loops don't have a scope). The first clause is available to initialize variables, but it's perfectly good style to leave it blank and use an already-initialized variable. It doesn't want to be an anonymous function with its own scope and parameters, because then you couldn't do this:

int c; for(c=0; str[c] != '\0'; c++) {} printf("Length of string = %d\n", c);

C doesn't have a message or a worldview. It's just about saving typing and making common things convenient.

LWN seems to be very good at writing articles about Rust which are already out of date at the time of publication.

Ah well. This too shall pass.

Alternatively, Rust is very good at changing so quickly that trying to write authoritative articles about it is a futile endeavor. :)

Nice example how the same fact can be spined in diffrent directions.

Just my humble opinion but for loops seem unnecessary in high level languages (the kind with GC). Vectors make more sense to me on this level (APL, J, etc.). I consider for loops as better fit for low level languages that don't automatically manage memory for the programmer (like C).

Historical programming was done by advancing from memory location to memory location, one step at a time. The loop is a natural abstraction over this linearity, representing a jump from the end of a line of instructions back to the beginning.

The problem being, as you point out, that it imposes a fixed order, regardless of whether the problem domain requires such an order.

In database programming one of the biggest challenges for beginners is to stop thinking in terms of loops; to learn to "think in sets".

Thanks for this. I'm relieved to know I'm not thinking the "wrong" way. :)

But, honestly, I could not articulate the problem as you have. I wish I had your knowledge. The idea of a "jump" really captures it. setjmp. goto. It's navigating memory. And nothing more.

I can visualize the problem, sort of - I just intuitively feel like sets, Lisp-like lists, and vector-based approaches, are the way to go in high level programming - but I do not know the right words to describe and argue my point.

As such, I doubt I could convince many high level programmers to abandon the use of iterations in the code they write.

Again, you have beautifully articulated what I could not. Cheers.

I did a little assembler in a course at university. When you write instructions that tell a CPU where to go next, it all becomes fairly visible.

Even C blurs the model (which itself is a lie these days, but no matter). It didn't matter so much when C was created, because early C programmers were also assembler programmers. These days that's no longer true.

Go Rust Go! :)

Sorry, I could not resist.

Is that "Go forth!" or "Go away!"? :)

Maybe he meant Go Forth in disguise?

I was supporting Rust. :)

Some philosophers will argue that a loop should be a macro.)

  (loop for i from 1 to 10 collecting i)

Other would say "use recursion".)

For someone who knows LISP, your parentheses are curiously unbalanced.

But he did got the superfluous part right.

Reply on news.ycombinator.com