The Problem with Implicit Scoping in CoffeeScript | Hacker News Reader

The Problem with Implicit Scoping in CoffeeScript

lucumr.pocoo.org

133 pointsmichaelty134 comments

I'll copy below jashkenas' longer answer from the old github issue about this, https://github.com/jashkenas/coffee-script/issues/712#issuec...

"""

Sorry, folks, but I'm afraid I disagree completely with this line of reasoning -- let me explain why:

Making assignment and declaration two different "things" is a huge mistake. It leads to the unexpected global problem in JavaScript, makes your code more verbose, is a huge source of confusion for beginners who don't understand well what the difference is, and is completely unnecessary in a language. As an existence proof, Ruby gets along just fine without it.

However, if you're not used to having a language without declarations, it seems scary, for the reasons outlined above: "what if someone uses my variable at the top of the file?". In reality, it's not a problem. Only the local variables in the current file can possibly be in scope, and well-factored code has very few variables in the top-level scope -- and they're all things like namespaces and class names, nothing that risks a clash.

And if they do clash, shadowing the variable is the wrong answer. It completely prevents you from making use of the original value for the remainder of the current scope. Shadowing doesn't fit well in languages with closures-by-default ... if you've closed over that variable, then you should always be able to refer to it.

The real solution to this is to keep your top-level scopes clean, and be aware of what's in your lexical scope. If you're creating a variable that's actually a different thing, you should give it a different name.

Closing as a wontfix, but this conversation is good to have on the record.

"""

> As an existence proof, Ruby gets along just fine without it.

Missing that Ruby stops scoping variables at a method and uses separate lexical scoping rules for constants thereby avoiding this issue mostly.

Also, Ruby can shadow method names with local variable names just fine:

    class Foo
      def bar
        "bar"
      end

      def baz
        bar = "foo"
        puts bar
      end

      def bang
        puts bar
      end
    end

    foo = Foo.new
    foo.baz  # => "foo"
    foo.bang # => "bar"

Well, this is a case where the equivalent CS behaves exactly like Ruby.

  class Foo
    bar: -> 'bar'
    baz: ->
      bar = "foo"
      console.log bar
    bang: ->
      console.log @bar()

  foo = new Foo()
  foo.baz()  # => "foo"
  foo.bang() # => "bar"

Not quite; @bar is "this.bar", and is an unambiguous reference, not just "bar". In Ruby, because self can be implicit, it's literally just "bar" in both cases, and Ruby gives the local variable precedence over the method name. You can still invoke the method via self.bar, but accessing the method as just "bar" is shadowed by the variable. Additionally, even though the method may be invoked by referencing bar, you can't redefine the method by assigning a new method to bar. Neither Javascript nor Coffeescript has that behavior.

Here's a better example:

Ruby:

    def bar
      "I called bar!"
    end

    def foo
      puts bar
      bar = "I manually assigned bar"
      return bar
    end

    puts foo()
    puts bar()

    # =>

    I called bar!
    I manually assigned bar
    I called bar!

Coffeescript:

    bar = ->
      "I called bar!"

    foo = ->
      console.log bar()
      bar = "I manually assigned bar!"
      return bar

    console.log foo()
    console.log bar()

    # =>

    I called bar!
    I manually assigned bar!
    TypeError: string is not a function

When I said the two programs have the same behavior, I was simply referring to the end result--what they output. I understand how both languages work here. Ruby solves a problem that simply doesn't exist in CS. In JS/CS "bar" never implicitly refers to "this.bar", so there's no ambiguity in the first place.

Your example does demonstrate how CS works. When you assign bar a new value inside its original scope, the value of bar does indeed change.

  bar = ->
    "I called bar!"

  foo = ->
    console.log bar()
    bar = "I manually assigned bar!"
    return bar

  console.log foo()
  console.log bar()

Here is a slightly less contrived example:

  log = (data) ->
    console.log data
    
  enhance_logging = ->
    i = 0
    log = (data) ->
      i += 1
      console.log i, data

  log "no line numbers"
  enhance_logging()
  log "line one"
  log "line two"

Here is the output:

  > coffee foo.coffee 
  no line numbers
  1 'line one'
  2 'line two'

and well-factored code has very few variables in the top-level scope -- and they're all things like namespaces and class names

Unless you happen to embrace JavaScript's functional side and write lots of top level helper functions (in a closure of course after which you export)

shadowing the variable is the wrong answer. It completely prevents you from making use of the original value for the remainder of the current scope.

This smells of static enforcement - strange for a language expounding JavaScript's dynamic nature and an odd departure from "it's just JavaScript".

Shadowing doesn't fit well in languages with closures-by-default

Not sure what evidence this is based on given the heap of great languages with closures-by-default that give programmers more control over scope without introducing goofy constructs like special assignment operators or global/nonlocal keywords.

CoffeeScript breaks the one real form of encapsulation (which includes the power of naming) that JavaScript has - function locals.

I may just ask you in a few minutes when you get in ... but what exactly is the scoping scheme (and generated JavaScript) that you're proposing here, without "var", "nonlocal", ":=", and with shadowing?

In addition, to repeat myself elsewhere in this thread, the goals here are conceptual simplification and readability, not giving the programmer more control over scope. The final result is that hopefully:

    someVariable
      ... more code here ...
        someVariable
          ... more code here ...
            someVariable
          ... more code here ...
        someVariable

... in the above code, you can know that "someVariable" always refers to the same thing. With "var", the above code could allow "someVariable" to refer to three different things, each for slightly different sections of the above chunk of code.

If you really want three different values, use three different names. In all cases, it will read better than shadowing would have.

In all cases, it will read better than shadowing would have.

You are going against the consensus established by ALGOL and Scheme (and used by most languages in the functional camp since then) here. That's your prerogative of course, but personally I'd be wary of design choices that go against established wisdom.

With "var", the above code could allow "someVariable" to refer to three different things, each for slightly different sections of the above chunk of code.

Yes, but that's not an issue, as it is easy to check (by looking at the code section in isolation) which of the three bindings an identifier refers to - which is, for me, the definition of lexical, static scope.

ruby is terribly ambiguous..

If you really favor protection from errors, then the correct choice is to require 'var' for new declarations but forbid shadowing (it's an error). Shadowing a la Scheme is a source of errors, too -- you can intend the outer variable, failing to notice it's been shadowed. Of course, if you really favor early detection of errors, you'll also have static name binding.

[Shadowing] prevents you from making use of the original value for the remainder of the current scope.

That's totally false.

In languages like Scheme, O'Caml, etc you are never prevented from using the original value.

The point is that lexical, aka static scope is all about lexical pieces of code that you fully control, and all their properties are statically apparent, by looking at a single piece of code.

Scheme:

  (DEFINE FOO 1)
  (LET ((FOO 2)) ... FOO IS SHADOWED HERE ...)

Inside the LET, FOO is shadowed (FOO is "your FOO") and that's lexically, statically apparent by looking at the piece of code.

If you don't want it shadowed, and use the global FOO, you just use another variable name. You cannot be prevented from using the original global FOO, because you choose the local variable names you use in a piece of code.

You cannot be prevented from using the original global FOO, because you choose the local variable names you use in a piece of code.

Unless you're using a system in which non-hygienic macros are present and they expand into it...

It's worth pointing out that JavaScript 1.7 resolves this mess by introducing block scoping using the "let" keyword. It works just like it does in Scheme, Common Lisp, and Clojure (i.e., correctly). Not supported in anything except Firefox, unfortunately.

... and Standard ML, OCaml, Haskell, Smalltalk, etc

... and Perl except its called my instead of let

Scheme got this right in 1970. There is no excuse to design a new language this way.

I agree with this wholeheartedly--Scheme use an extremely simple scoping model that is nonetheless more expressive than Python (before 3, I guess) and CoffeeScript's. In fact, I only really completely understood JavaScript's model--and realized that, even if a little awkward, it was fundamentally elegant--after writing a Scheme interpreter.

I think Python's is deliberately unexpressive, forcing you to use local variables pretty much everywhere. This tends to decouple stuff, which is usually good. The model is "everything you do is local, unless you know better, and want to jump through a lot of hoops". Coffeescript works a similar way.

Javascript seems to have the model "everything you do is global, unless you know better".

I'm not a big Javascript hater, but this is one very sore point.

"everything you do is global, unless you know better"

That is not the JavaScript model at all.

He is talking about if you don't do anything extra (in this case, use the var keyword). Much like the nonlocal keyword in Python requires knowing about it and wanting to use it in order to actually use it and get the non-default behavior.

var is not extra in JavaScript the way nonlocal is in Python. var is a language construct that any good JavaScript program must use.

The point of the linked article is that this is not the CoffeeScript way at all (although the author wishes it was).

No, the Javascript model is "you name things with `var`, you mutate things with `=`".

Python went for a more complicated scheme of having every variable in a function be a new binding when first assigned, and then it's just mutation (i.e. 'local'), unless explicitly declared to be nonlocal, but it still makes sense with their... penchant for mutable state.

And Coffescript instead went over to PHP to have some of the glue it was eating.

No, the Javascript model is "you name things with `var`, you mutate things with `=`".

If only it were that simple. It also has "you create thing with `var` and sometimes with `=`", plus it does not have block scope:

    if( true)
    {
       var x = 1;
    }
    // x == 1 here.

Functions define scope. The end. Not hard. One rule.

'let' is a way to make things more local! When I say: let x = 0, I'm saying "hey, I don't care what 'x' is in other scopes, in this local scope it's 10, dammit!"

Yes explicit is best and in Perl its very clear what you want...

  my $x = 10;

Scheme dates to about 1975. But there were older languages with this kind of scoping, like John Reynolds' Gedanken and Landin's ISWIM. (I guess I wouldn't count Algol-60 since it was call-by-name.)

Another Issue:

Although the following examples could become unambiguous with parenthesis, these examples demonstrates how a trivially overlooked ending delimiter further complicated the language. Not only is the intent of the CoffeScript code unclear in the examples below but the slight variation in the CoffeScript, produces radically different output. The CoffeeScript differences are so small it would be easy for someone to add accidentally while editing. Anonymous function passing and function calling in Javascript require no additional wrappers or edits, while in CoffeeScript you must add special case clarity.

http://img542.imageshack.us/img542/7379/coffeescripttojavasc...

@mitsuhiko Not gonna happen ;) Forbidding shadowing altogether is a huge win, and a huge conceptual simplification.

How arrogant! You'd think he'd step back for a second and consider the suggestion, but it sounds like he's on autopilot.

It sounds like he had 140 characters to succinctly say what he wanted to say.

Everyday there seems to be a new post on HN, complete with inflammatory headline, criticizing Coffeescript in someway because it doesn't work in the exact way the author expected.

There are lots of people throwing in their $0.02 on how the language should work without having joined the mailing list or seen any discussions on the thought process behind its features.

I'm not saying the suggestion made isn't reasonable, but I can understand glib replies like this from the author that don't make too much effort to explain his stance more than 140 characters.

In fairness, Armin (the author the blog post) did engage Jeremy (the author of coffeescript) on this issue over twitter. I wish that he would allow comments on his blog. I also wish that he could engage the broader CS community in a proper forum before dissing the language and/or creating a fork of the language. He's overreacting. This whole blog post apparently started because he had a naming collision with "log" in one of his programs. CS does have a mailing list, but most of the action happens via github issues.

> I wish that he would allow comments on his blog.

Why? Hackernews and reddit exist and everybody is free to send me a mail or contact me on twitter. This way I do not have to moderate any comments or deal with spam.

> I also wish that he could engage the broader CS community in a proper forum before dissing the language and/or creating a fork of the language.

And do what? Duplicating an issue that is already there? Commenting on a dead issue? The author has expressed his unwillingness to deal with this issue so why should I reopen the issue there?

> He's overreacting.

How am I? I wrote a very short blog post about why I think the scoping is bad and how it caused me problems. Many people asked me on Twitter why I think the scoping does not work as good as it should and since I only have 140 characters to explain stuff there I wrote it to my blog. How else should I communicate that?

> CS does have a mailing list, but most of the action happens via github issues.

There is an issue about this topic from a year ago which was closed and the author does not want this to be changed. I am okay with that, a language needs leadership. That does not mean however that other people should not know about this issue when they design the next programming language.

I wish that you would allow comments on your blog, so that folks following your twitter link would be able to see both sides of the issue, but I obviously understand your spam issues, which is why I said "wish" and not "should".

I wish that you would engage the CS community on this topic without the sole agenda of getting this fixed. It's true that the issue was put to rest a while ago, but the decision wasn't made in a vacuum--there was consensus involved. I think it's a little unfair to say that Jeremy "has expressed his unwillingness to deal with this issue"; that makes it sound like he was dismissing you or dismissing further debate on this, when in fact he just told you what had already been decided.

When I say you're overreacting on this issue, it's just my opinion, so don't get too worked up about it. You had a bug. "Log" means two things. IMHO you don't need to fork coffeescript; that would be a gross overreaction, but YMMV.

You are not doing anything wrong by taking the time to write up your opinion in a blog--if I implied that in any way, it was not out of malice; it was just imprecise writing on my part.

For what it's worth, thanks for that detailed article. I've just used Coffeescript for a large-ish WebGL project and had no idea of that behaviour. I got freaked out when I got to the bottom of that first fragment and saw log vs log.

I'm going to keep using Coffeescript, I love it. Maybe this kind of scoping isn't the best decision for modularity and teamwork, but on reflection at least the rules are simple and consistent enough for a single programmer to keep in mind.

  > How else should I communicate that?

Who says you should?

Blog posts about (and against) CoffeeScript features are great, as is discussion on HN, as is discussion on the issues pages.

In addition, nothing in CoffeeScript is set in stone -- because every script compiled with every version of CoffeeScript is compatible with every other version, we're much more comfortable making changes to the language than we otherwise would be. If you can make the case that this change is a good idea, we'll definitely make it. So feel free to comment on the old issue or open a new one if you wish.

> because every script compiled with every version of CoffeeScript is compatible with every other version, we're much more comfortable making changes to the language than we otherwise would be.

This attitude concerns me. I can't just say "oh, that's CoffeeScript 1.0 stuff, just trash it, the JS still works". I still have to update the CoffeeScript to upgrade the version. There's no less risk in breaking backwards compat with CoffeeScript than any other language.

> He's overreacting. This whole blog post apparently started because he had a naming collision with "log" in one of his programs

Surprised at that reaction, he's had a problem and he's written an excellent blog post about it.

I think this is a recurring discussion from the dawn of coffeescript. See for example: https://github.com/jashkenas/coffee-script/issues/238

I really don't understand why this isn't being fixed: doesn't global by default break encapsulation? I'm probably missing something, but this is the main reason I haven't tried coffeescript yet.

Just to be completely clear, variables are only at top level scope if you declare them at top-level scope. The variables "x" below are completely encapsulated within f1 and f2. The variables at top-level scope are intentional in the code below--I really do intend f1, f2, and how_many_times_functions_have_been_called to refer to the same entity throughout the file.

  how_many_times_functions_have_been_called = 0

  f1 = ->
    how_many_times_functions_have_been_called += 1 # refers to top-level scope
    console.log x # undefined
    x = 1
    console.log x # 1

  f2 = ->
    f1() # refers to f1 at top-level scope
    how_many_times_functions_have_been_called += 1 # refers to lop-level scope
    console.log x # undefined
    x = 2
    console.log x # 2

  f1() # you can call f1, it's at top-level scope
  f2() # you can call f2, it's at top-level scope
  console.log how_many_times_functions_have_been_called # 3, refers to top-level scope
  console.log x? # false, x does not exist at top_level scope

Yes, I probably wasn't very clear: what I meant is that a programmer writing a function somewhere in a program must have a complete knowledge of the scope where the function is and will be in the future, including changes in global variables exposed by the interpreter/browser. In the end I would find me forced to add a prefix to all the variables in order to avoid collisions, just like I would be forced to do, if the language only had a single global scope.

This behaviour seems quite unreasonable to me, but I haven't been able to find explanations about it, other than it's expected behaviour.

Fortunately, what you're describing isn't how it works.

CoffeeScript will automatically declare all variables in the nearest lexical scope it can find. The top-level scope in CoffeeScript isn't global -- it's the top of the file. You don't have to know anything about what values may or may not exist in global scope at any given moment ... all you have to know is what variables are visible in your function's enclosing scopes, just within the file you're working in.

Oh, I see. And you fiddle with globals by manipulating the window/exports object, which I assume are reserved identifiers.

Thanks a lot for clarifying that, it doesn't look that bad this way.

I think Jeremy probably has one of the highest programming-chops-to-arrogance ratios among today's top programmers. It was a tweet, he said what he needed to say. He's been more than willing on HN to step in and explain Coffeescript to critics and offer helpful critiques of competing frameworks (well, in the context of Backbone).

Here's how he introduced CoffeeScript to HN almost exactly two years from today: http://news.ycombinator.com/item?id=1014080

"top programmers" don't write javascript and web apps, we're all pretty much noobs

But he writes coffeescript so unlike you he's a "top programmer". except for times when he writes javascript which probably would be when hes drunk or for whatever reasons his IQ has dropped.

99% of the popular people out there are just above average enough to make useful projects for those lesser experienced. Most truly talented people are not very well known, they're busy hacking on v8, the kernel etc. Tools for average programming just get more of a spotlight. Is Express complex? no, people just use it, is coffeescript complex? no, is boostrap complex? no..

That sounds like a really accurate statistic. So what's your metric for this? Besides the obvious law that low-level code is the only grounds for elite coding, I mean.

Easy, he hand picked a 100 people of the JS community (including him) and then decided that hes the only competent programmer amongst them.

umm no haha, I'm certainly amongst the average. Trust me writing parsers is very trivial, I've written many. Thinking you're a good programmer is perhaps the most naive thing you can do

I know its trivial once demystified specially using parser generators. But language design isn't though. Anyways i think your a pretty decent programmer.

sure it is, take a few features from ruby, a few from python, add yaml and there you have it, coffeescript.

What do you think about node? Does it fall in your category of above-average people building tools for the average programmer?

they're certainly above average but even Ryan himself is humble enough to admit he's not amazing, I dont recall what his exact words were. I think it was on google+, he was talking about how he works with such brilliant people (which are lesser known), but yeah overall node's popularity is largely because it attracts more average people to the scene, vs things like erlang or haskell which are fantastic but attract a lot less attention.

This is also why I dont do conferences, I dont want to be known for going around promoting things (like crock), sure I'll blog about features added once and a while but other than that I want the projects to speak for themselves. Eventually if I can gain enough knowledge then sure being well known is neat since you can leverage it to hopefully expose better projects, but there's no end to what you can learn in this industry.

Well said.

Parsing: the solved problem that isn't: http://news.ycombinator.com/item?id=2327313

Like anything else in life there is front facing people that are generally good at marketing and would use other people's hard work to build technically inferior stuff on and be famous for. Not necessary a bad thing. The truly talented most of the time choose to go unnoticed. And there is always trolls like you who I don't know how they find the time to ship code ;p

Perhaps you are being too hard on yourself. The majority (more than half) of the well known programmers truly are "superstars". It's all very subjective, though. Where is the standard for complexity or even cleverness? How is it different than simply being esoteric. Maybe I am personally fascinated with regular expressions and after much study, can belt out a half page of parse and extract 'magic' that would make the uninitiated shake their head and crown me a genius. But to me, it's simply assembling larger ideas from smaller ideas.

Personally, I would say that a truly talented programmer is simply someone who is very capable in mathematics and can produce a working and extendable program in a reasonable amount of time, that does something new and useful. This criteria alone leaves a ton of people out, you know!

I think it has already been proven that there is no obvious correlation between being capable in mathematics and being a talented programmer. Thats so 1950's

Not a direct coorelation, but there is certainly a dependence on being capable in mathematics. A math wizard does not make a good programmer, but one cannot be hopelessly average in math and be a talented programmer.

Average of what? I'd bet 80% of 40-year-olds wouldn't be able to pass a high school level maths exam.

Actually it really depends on what math skills you talk about. Is it really the material you are taught and can apply to software that brings a lot ? The ability to solve problems ? The culture of correctness ?

The common ground between math and programming is the requirement of a very good capacity to manipulate abstract concepts in a defined frame of known validity. But it pretty much stops here. In mathematics, you define all your abstracts concepts and frame of validity, in programming, you are given a (very shaky and detailed) frame of validity on which you build up abstract concepts.

The ones that are good in approaching the discipline through the study of details to build up stuff that will work on top of it will make the developers. The ones that needs a strong and well defined frame for their work, for it brings a much more powerful ground and enable to reach very high levels of abstraction, will feel more comfortable on mathematics.

No wonder why those that can combines both of those approaches can yield stunning results.

There are certainly truly epic people out there that are well known, but hell me and Jeremy have many more followers than the guy who wrote openssl along with people like Mike Pall who could out-code most of us any day, it's a weird thing, but you're average programmer isn't concerned about lower level things like that, so they simply don't care. There are of course exceptions to this, people like Carmack or Linus, but still, you see my point.

How is it different than simply being esoteric. Maybe I am personally fascinated with regular expressions and after much study, can belt out a half page of parse and extract 'magic' that would make the uninitiated shake their head and crown me a genius.

Sounds interesting, tell me more.

I would also say a criterion is that he really understands abstraction and how to get more from less. Ie think of how John Carmack creates abstraction which is just right for the problem (and in C at that). Or think about the metalinguistic paradigm in programming (or OOP used right, for that matter). Or how JQuery (and these days Coffeescript) makes client-side web code clean and accessible to everyone, without the previous Javascript hacks and DOM-spaghetti. An average programmer just plods along and hacks out a solution in a linear manner, a great programmer will traverse levels of abstraction to not only solve the problem but also shine a light at it from a superior perspective.

jQuery nor coffeescript really imply "clean" client-side code, in fact I would argue that most uses of jQuery are the opposite if you compare communities like mootools. As far as coffeescript goes you still need a good sense of structure like regular javascript, nothing new there really.

Mootools still disqualifies itself in my mind by patching around on builtins.

certainly, but that's besides the point. If you look at the jQuery community vs the others, jQuery's is filled with much much more spaghetti code

jQuery makes the DOM so dang accessible that people use it as their data, which is completely backwards. Fortunately stuff like backbone has brought back some sanity.

Don't jump the gun. The reason is that he's already had many discussions about this and has formed an opinion.

See: https://github.com/jashkenas/coffee-script/issues/712 https://github.com/jashkenas/coffee-script/issues/238

Whether he's right is another question. But if you don't like his decision you can use the Coco (https://github.com/satyr/coco) fork which fixes it by introducing := for nonlocal assignment.

The reasoning basically seems to be "if you misuse this feature (i.e. have top-level symbols), you'll get bugs (like the OP discovered), therefore don't misuse this feature (i.e. keep your top scopes clean), and therefore you won't get bugs, and therefore it's not a problem with CoffeeScript".

It shouldn't take a lot of thought to see why this is a somewhat user-hostile, passive-aggressive approach for a language. If something is ill-advised, the language should actively steer you away from it, not dissuade you with subtle bugs a few thousand lines of code down the road.

I think this scheme would be more workable if sigils or similar conventions of some kind were mandatory for top-level symbols. Then it would be much harder to accidentally wander into this problem. The language he seems to be drawing inspiration from, Ruby, does do this.

I'm afraid that's not quite the reasoning...

Like everything in languages (or APIs), there's a tradeoff here. By making scoping automatic, and (hopefully) forbidding shadowing, you can make the language conceptually simpler. Think of it as making variables be "referentially transparent" in terms of their lexical scope. Everywhere you see "A" within a given lexical scope -- you know that "A" always refers to the same thing. In a language with "var" and with shadowing, "A" could mean many different things within any given lexical scope, and you have to hunt for the nearest declaration to tell which one it is.

On the downside, you have what Armin describes: If you happen to try to use the same name for two different things within the same lexical scope, it won't work.

Since it's always the case that you are able to choose a more descriptive name for your variable, and gain clearer code by it, I think it's very much a tradeoff worth making.

You haven't said anything new to me here; I took all this for granted and it isn't what I'm criticizing. You haven't actually addressed my summary of the apparent reasoning for why accidental assignments to top-level symbols isn't a pathology of CoffeeScript. The bit I'm criticizing is the failure mode of "if you happen to try to use the same name for two different things within the same lexical scope, it won't work".

I see. The reason I can't address your "failure mode" with an explicit suggestion is because it isn't a failure mode in CoffeeScript: If you assign a new value to a variable in an inner scope, the variable now has a new value. If you're thinking "I want to use the same name for two different variables" ... the answer is: choose a different (better) name for one of them.

But perhaps I'm still not getting at the answer you're looking for here...

You keep stating and restating how and why CoffeeScript doesn't support shadowing of symbols via lexical scoping.

That's not what I'm criticizing. Lexical shadowing is not what I'm advocating. I think your choice is fine in so far as it goes. It has its logic.

What I am criticizing is how it can fail. The top scope is different, quantitatively and qualitatively, from almost all nested scopes. It's much larger, and spread lexically over a larger area. If you have a team of developers, it will be modified concurrently. No one developer necessarily knows the full set of symbols defined in the top scope while they are writing an individual procedure.

And thus the problem: a developer thinks they've chosen a "different (better) name" for a some variable, but in fact they've chosen one that a different developer also thought was a "different (better) name", only one of them is in a lexically enclosing scope. This problem isn't likely to occur on the level of nested procedures or nested blocks, because the definitions would be visually close. But it's much more likely to happen when one of the symbols is defined in the top scope. Here, the definition could be many hundreds or thousands of lines away. It may even be in a separate commit, waiting to be merged, such that there's no way for either developer to know without closely reviewing every change.

And this is the criticism: the failure mode for this inadvertent reuse of a variable name is subtle bugs, as what one developer thought was a global symbol turns out to be modified and acquire strange values through unexpected codeflow, almost like the VM was corrupted and memory was behaving unreliably.

The qualitative difference of the top scope in situations like this is the reason why I suggested sigils or somesuch to disambiguate those scenarios. Perhaps top-level symbols can't be reassigned from nested scopes unless you use '$' as a prefix to their name; a visual shorthand that you are definitely not creating a new local symbol.

The reason I summarized your argument in the way I did is because your argument against this failure mode seems to be "don't create top scopes with lots of symbols". That's a fine argument (or rather, exhortation), but it isn't a realistic one. If the language is problematic with lots of symbols in the top scope, it should be unpleasant to use with lots of symbols in the top scope. And the unpleasantness shouldn't come from subtle bugs (the passive aggressiveness I mentioned); it should come from awkward and ugly sigils, or some other intrinsic way of discouraging those styles.

barrkel, I think you should create a new programming language with more sensible scoping than CoffeeScript, so that developers can create bug-free code while working in parallel on thousand-line codebases without close reviews. I would start by using "$" as a sigil to prevent top-level variables from being reassigned in nested scopes. It would be a visual shorthand that indicates that you are definitely not creating a new local symbol. This would prevent developers from needing to know the full set of symbols in the top scope. In fact, by eliminating the inadvertent reuse of variable names, you would eliminate the failure mode whereby what one developer thought to be a global symbol actually turned out to be modified. Eliminating this unexpected codeflow would be tantamount to eliminating memory corruptions in VM.

You should definitely go down this track. It would be a massive achievement.

I think you think you're being sarcastic. But I do actually work for a developer tools company, and I implement language features on a continuous basis. I implemented anonymous methods - closures - for Delphi, a task that involves no small amount of scope wrangling; and FWIW, Delphi almost certainly has a larger userbase than CoffeeScript, albeit one with greyer hairs. While my specialism is in statically typed, compiled languages, issues around lexically nested scope are pretty much the same as in dynamic languages.

If you want to have a constructive conversation, you could try dialing back the snark, and addressing my arguments directly.

Ok, I'll dial back the snark and just be blunt. I think all your concerns about CoffeeScript are hypothetical exaggerations. I don't think you've written much CoffeeScript at all, and I suspect that if you did, you'd quickly see that your concerns are overblown. In theory, you could have thousands of lines in a single file, and two commits from a separate branch could cause an undetectable accidental naming collision. In practice, this rarely happens. Files tend to be smaller, merge changes do to tend to get scrutiny, and name collisions often have fairly obvious symptoms once you run your tests on the code.

I think if you have good practices - small files, scrutiny of changes, unit tests - you can make any language work. The test of a language comes in how it bites you when you stray. So to be blunt, I think your defense is irrelevant.

The problem with mandatory conventions is that they represent a different kind of user hostility--you're not trusting the developer to make his own decisions about variable names.

To give an example, many CoffeeScript programmers do follow simple naming conventions to call out top-level variables, such as CamelCase for classes or ALL_CAPS for constants. When you follow these conventions, it's pretty easy to avoid naming collisions.

In certain cases, though, you want a top-level variable to be lowercase, perhaps for stylistic reasons. If your files are relatively small, it's pretty easy to check for naming collisions when you introduce top-level variables after the fact, so a developer might decide that the risk is acceptable, especially if there is good test coverage.

Another kind of user hostility is to optimize for safety at all costs. I don't think any scoping mechanism totally eliminates the possibility of bugs, but some schemes do err on the side of safety over convenience. There's nothing wrong with trading off convenience for safety, but, on the other hand, you can make judgment calls that convenience and/or simplicity of the scoping model outweigh the risk of naming collisions.

Obviously, I like CoffeeScript, so I think Jeremy's made the correct tradeoffs. All languages work a little different--JavaScript, CoffeeScript, Python2, Python3, and Ruby all have different rules--and none of them are perfect in all situations. In all of the languages, though, it's reasonably straightforward to write correct code once you adopt general good practices--be careful with your names, and understand the language's approach to scoping.

@mitsuhiko Not gonna happen ;) Forbidding shadowing altogether is a huge win, and a huge conceptual simplification.

Poppycock. It doesn't get simpler than

  a. newly bound variables are new
  b. any variable which wasn't bound in the present scope must be bound in a enclosing scope.

If your language makes it much more complicated than First Order Logic (http://cnx.org/content/m12081/latest/), you're doing it wrong.

Of course, it's kind of socially acceptable to get wrong because a lot of language designers didn't think it through and used the same operator for both binding and reassigning a variable (i.e. '=').

Sorry -- that was supposed to be a friendly tweet-sized response. The discussion about this "feature" is always good to have, especially because it's one of the most rightfully controversial changes that CoffeeScript makes to JavaScript semantics.

There are dozens of lengthy conversations about this in the CoffeeScript issues, if you'd like to take a deeper look.

Why not allow CoffeeScript to use the 'var' keyword, explicitly telling CS that this variable should be scoped locally even though it may be shadowing another variable? This seems consistent with their approach of allowing (although optional) native javascript syntax such as {}, and []. This still allows CS to stick to it's paradigm of forbidding shadowing unless we explicitly state that we know what we're doing.

Because we're aiming for a conceptual simplification.

Pretend like you're a beginner, learning this stuff for the first time. If everywhere you see a variable "A", within a certain lexical scope, it means the same thing ... that's much simpler to understand than if "A" means three different things at three different places, because you happened to shadow it twice.

Yea, I completely get what you're saying and for the most part agree. To be frank, I think the author of the article made an error in deconstructing on the 'Math' object (at least at a global scope). 'Math' provides a namespace for all of its methods and a similar approach should be taken to other libraries/pieces of code. I also agree that keeping things simple and straight forward makes sense, but you're doing it by forcing one to abide by those standards although someone might have completely legitimate reasons for explicitly scoping their variables.

Either way, I think it is what it is and the benefits of CS very much outweigh the cons. Thanks for the feedback

I like the look of coffeescript's assignment better, but I cant help but think "let" is much less ambiguous, once you see it you look no further. This reminds me a bit of Ruby, where "foo" could be a function or variable potentially from anywhere so it's a little unclear although better looking.

Look at how MoonScript handles this: http://moonscript.org/reference/#the_using_clause_controllin...

I didn't want to change the default semantics, but I wanted to have a way for the programmer to be safe if they wanted to, so I created the `using` keyword for function declarations.

You explicitly declare what you intend to overwrite in the lexical scope, including overwriting nothing at all with `using nil`.

I found one of the referenced links in Github issue #712 quite interesting:

http://www.rubyist.net/~matz/slides/rc2003/mgp00010.html

Source: https://github.com/jashkenas/coffee-script/issues/712#issuec...

These links explain the thought process behind CS's current behavior:

https://github.com/jashkenas/coffee-script/issues/712#issuec...

Considering we won't see this changed since the author has already closed the issue and expressed his satisfaction with the current rules this article should at least serve as a reminder for errors not to repeat with the next language someone designs.

It's open source. Why not fork it and get some like minded coders to change it with you?

There already is a fork that changes this (Coco), already mentioned in another comment: http://news.ycombinator.org/item?id=3380423

With "var" and shadowing you can still shoot yourself in the foot, it's just the other foot.

If you need global variables, it's sensible to just adopt a simple naming convention, like prepending g_ (or whatever pleases you) to all your variables. I already did that with plain JS and it's well worth the "effort".

Kind of a side note to the posting, but I just have to say: Please make your usage of parens consistent. If you aren't going to use them, don't use them everywhere.

Here is an example of what I'm talking about:

if isAir cx, cy, cz + 1 then addPlane('near', block)

Should be:

if isAir cx, cy, cz + 1 then addPlane 'near', block

Personally, I use them everywhere because I like having the stronger visual clue that this is a method I'm calling. I think making them optional in CS was a bad idea.

if isAir(cx, cy, cz + 1) then addPlane('near', block)

imho, so much more readable.

Not allowing parens would mess up the usage of anonymous functions.

Not sure I understand you or maybe you don't understand me. I was talking about using parens for method/functional calls, not getting rid of them from CS.

I understand. If you don't have parens on method calls, how could you use an anonymous function that is not the last parameter? For example:

  doSomething(->
    # stuff happens
  ), onerror

I totally agree with you.

First, your code snippet is wrong: you need a space before your first paren.

In coffeescript you can do any of the following, or a few other variants:

  doSomething ->
      stuff
    , onerror

  doSomething(->
      stuff
    , onerror
  )

  doSomething (->
      stuff
    ), onerror

  doSomething(
    ->
      stuff
    onerror
  )

  doSomething (-> stuff), onerror

I had this question a while back as well. Take a look at the discussion here, including Jeremy's thoughts: https://gist.github.com/1215863

CoffeeScript's approach toward top-level variables is quite elegant and simple. When you declare a variable at top-level scope, it is equally available to all code within that file for both reading and writing, with no strange "nonlocal" or ":=" syntax to complicate manners.

Once you understand the reach of CoffeeScript's top-level variables, it is easy to write bug-free code. Since you know that top-level variables have wide scope, you simply need to be judicious about putting variables at top-level scope. If a variable is not needed at top level scope, don't put it there.

Way to miss the issue. I was not suggesting ":=" for global variables at all. I was suggesting ":=" as a replacement for the nonlocal keyword that Python 3 has to solve the issue I was demonstrating.

Nope, I get it, you were suggesting ":=" as a replacement for the "nonlocal" keyword in Python. I'm not sure how my comments demonstrate any lack of reading comprehension. All I said about ":=" and "nonlocal" is that they are overly complicated once you accept that top-level variables can have wide scope.

Obviously, plenty of folks managed to write bug-free code in Python before "nonlocal" was invented. I'm not saying it's a bad idea, but you can avoid bugs without it.

> Obviously, plenty of folks managed to write bug-free code in Python before "nonlocal" was invented.

Python has the inverse behavior of CoffeeScript. So that was never an issue.

Well, fine, but Python2 had the inverse problem--you couldn't mutate top-level variables without awkward "global" statements.

That's a whole different matter, which can't make you shoot yourself in the foot without knowing it for any non-trivial code.

CoffeeScript's scoping forces you to always keep track of whatever is enclosing the current scope ALL THE WAY TO THE TOP. This is way too much when your function doesn't need to access outer variables (which should be the minority of the cases).

So, problem is, you either make all your functions have non-free variables (but Jashkenas seems to dislike functions shadowing outer variables too, which is just... overtly weird), or you keep track of all variables above the current scope.

The former is not too unreasonable, until you remember it makes no sense with closures :3

You are correct that variable scoping goes all the way to the top, but nobody is forcing you to create top-level variables with overloaded names like "log". Seriously, if you have a file that uses log files and logarithms, just take some care to distinguish the concepts. Use "log_file" for log files; use "logarithm" or "Math.log" for inverse exponentation.

> Use "log_file" for log files; use "logarithm" or "Math.log" for inverse exponentation.

Just clarifying; are you actually suggesting this, or was that sarcasm?

It's a serious suggestion. Instead of introducing four top-level variables (log, sin, cos, and tan), just introduce one (Math) that wouldn't possibly conflict with a local.

The CoffeeScript sourcecode itself does not advocate it. It uses functions named like 'last', 'starts', 'ends', 'extend' etc.

You are correct about "last", "start's, etc., but there are also good examples in the CS sourcecode of easily avoiding naming conflicts with a sensible naming convention:

  browser.coffee:CoffeeScript = require './coffee-script'
  coffee-script.coffee:{Lexer,RESERVED} = require './lexer'
  coffee-script.coffee:    {Module} = require 'module'
  command.coffee:{EventEmitter} = require 'events'
  grammar.coffee:{Parser} = require 'jison'
  lexer.coffee:{Rewriter, INVERSES} = require './rewriter'
  nodes.coffee:{Scope} = require './scope'
  nodes.coffee:{RESERVED} = require './lexer'
  repl.coffee:{Script}     = require 'vm'

The problem is actually worse. You also have to keep track of all the variables BELOW the current scope as well so you don't accidentally (as in the case of the article) turn a declaration into a reassignment.

yeah that's a sketch thing to have to worry about :s

I actually agree with his stance against shadowing variables, just on philosophical grounds. It encourages good, descriptive naming. On the other hand, I acknowledge that sometimes it's desirable to shadow outer variables. I think it should be discouraged, but not prevented.

I think the big problem with Coffeescript's behavior is that it can introduce some damn subtle bugs that can be really hard to track down if you don't know what you're looking for, because you're not able to explicitly specify scope semantics. It's even worse if you're polluting higher-scope variables of the same type, because it becomes even less obvious where the error comes from.

Coffeescript more or less shares Ruby's scoping rules, but there's a cultural difference between the Ruby and Javascript communities that makes it a little less workable in Javascript. Specifically, Ruby's "everything is an object", aggressive use of namespacing, and the general idiom that only constants go into the global namespace tends to limit scope issues that could arise from mix-ins.

Coffeescript does attempt to mimic this by providing class semantics and wrapping everything in anonymous functions to limit scope leak, but there's still a lot of temptation to just create a bunch of top-level functions, and that leads to situations like the one described in the blog post.

Yes, and it's very uncommon that you need to do that. I learned Scheme years ago before I learned Python (in 1993, I think). At first I was dismayed by Python's lack of non-local assignment. However, in years of Python programming I can count the times I needed the ability on one hand. I haven't yet found a need for 'nonlocal'.

Obviously it depends on programming style. In Coffeescript you don't have something akin to 'self' and so assigning to a non-local happens a lot.

> However, in years of Python programming I can count the times I needed the ability on one hand. I haven't yet found a need for 'nonlocal'.

Might have to do with Python itself as well: because it's function-scoped and it tends to avoid higher-order function (in part due to the limitations of its anonymous functions), there are far less occasions write to lexical closures than in Scheme, Smalltalk or Ruby.

This is my biggest problem with CoffeeScript. And the author stubbornly refuses to fix it. Apparently it's some sort of Ruby religion.

Here is a ghetto "let" form in coffee

((a = 5, b = 6, log = x -> console.log x) -> log a + b)()

> The simple solution is to either add a nonlocal keyword like Python has or to introduce a := parameter that works like = but explicitly overrides a higher level variable.

I disagree. The simple solution to this is to write tests.

> I disagree. The simple solution to this is to write tests.

That's a step backwards. Code error checking should be done as early as possible. In order of earliness:

* Typing in the code. (Ideal: it is clear from the syntax that the code performs X instead of Y.)

* Compiling. (Strong type checking ensures you cannot return 5.3e7 or null from GetHostName.)

* Running the code at all. (Code contracts and assertions trigger if GetHostName returns "".)

* Automated unit tests. (Check that DB.GetHostName() returns the same string given to DB.Connect().)

* Automated integration tests. (Check that the DB module can connect to and retrieve useful data from a dummy database.)

* QA ("Hey Joe, the system hangs when I give "¤;\@" as my username and press the connect button rapidly for a few seconds.")

* Customer ("Hi the system has a problem, please fix.")

The further up, the faster, more accurately and with less "noise" the error can be discovered.

FWIW this is how CS works:

  top_level_variable = null

  f = ->
    top_level_variable = "hello"

  f() 
  console.log top_level_variable # prints hello

The concern is that because Coffeescript automatically scopes variables to the scope of their first reference, it can introduce maintenance issues. Consider the following:

    foo = ->
      bar = "woot!"
      console.log bar

This compiles to:

    var foo;
    foo = function() {
      var bar;
      bar = "woot!";
      return console.log(bar);
    };

bar is locally scoped to foo(). Now, 2 weeks later and 200 lines earlier, you come along and define:

    bar = ->
      alert "Holy crap cheese is awesome!"

Which compiles to:

    var bar, foo;
    bar = function() {
      return alert("Holy crap cheese is awesome!");
    };
    foo = function() {
      bar = "woot!";
      return console.log(bar);
    };

Now, all of a sudden, the "bar" reference in foo isn't scoped to foo() anymore, it's scoped globally, and once you invoke foo(), it'll replace the function bar with a string, potentially breaking your app. It's an ease-of-maintenance issue.

This isn't consistent behavior, though. If you define your top-level bar() function after foo, like so:

    foo = ->
      bar = "woot!"
      console.log bar

    bar = ->
      alert "Holy crap cheese is awesome!"

Then you get "correct" scoping (and the outer bar is shadowed):

    var bar, foo;
    foo = function() {
      var bar;
      bar = "woot!";
      return console.log(bar);
    };
    bar = function() {
      return alert("Holy crap cheese is awesome!");
    };

On one hand, it could be argued that this is a "name things better" problem, but on the other, I have to agree that it'd be nice to be able to explicitly scope things when needed. Given that the behaviors are divergent based on what order the variables appear in, I'd say it's confusing enough that a way to explicitly say "hey, I know what I'm doing, I want to shadow any outer variables and declare local scope here" would be useful.

This is a better explanation of the problem than the blog post. Thanks!

The inconsistency doesn't stop there; `for` variables are specialcased:

    bar = ->
      alert "Holy crap cheese is awesome!"

    foo = ->
      for bar of bars
        console.log bar
      return

↓

    var bar, foo;

    bar = function() {
      return alert("Holy crap cheese is awesome!");
    };

    foo = function() {
      var bar;
      for (bar in bars) {
        console.log(bar);
      }
    };

That makes plenty of sense to me; when do you ever want to leak a for loop counter to an outer scope?

And when do you ever want to leak local variables to an outer scope? Never.

Not to mention that loop counters are no different than other `var`iables in JS.

You do want to sometimes reference outer scope variables from inner scopes, though. The only difference between a leaked local and a properly referenced global is usage semantics.

Thanks for this! Way better explanation than the blog post :)

Is there anyway to make the scope explicit in coffeescript?

No. That's the crux of the complaint. In plain Javascript, you can use "var x" to scope x to the current scope (and in 1.7, you can use 'let' to scope to the current block, not just to a function), but there's no corollary in Coffeescript, since Coffeescript manages scoping for you automagically. It's a convenience 99% of the time, but the 1% of the time that it's not, it's a giant pain in the ass. If there were something like var/let/local, you could solve the problem manually, but as the Twitter exchange indicated, there is no plan to add it.

The solution/workaround is "use descriptive names and avoid polluting scopes", but the reality of software development is that you're eventually going to cross those wires, and it's going to make you crazy until you figure out what happened.

Yes, create the variable as a function argument.

While that does allow you to use a locally scoped variable (even one that shadows an outer variable), it's still not explicit scoping.

There is some discussion here on why a "bar" variable at top level only affects the scoping of other "bar" variables that are lexically below "bar" in the file, not above it.

https://github.com/jashkenas/coffee-script/issues/1121

Reply on news.ycombinator.com