Design Note: Implicit Semicolons (2017) | Hacker News Reader

Design Note: Implicit Semicolons (2017)

craftinginterpreters.com

40 pointstosh40 comments

In my opinion, a little redundancy is good to avoid rockets falling out of the sky, so I vote for a layout rule plus redundant semicolons. The semicolon is hardwired in many programmers hands anyway. I don't think it should be a design goal to get rid of it. Instead, the compiler's/interpreter's job is to help the programmer to avoid bugs, and with a little syntactic redundancy, the parser can already help out with a few syntax errors. I love gcc's -Werror=misleading-indentation.

I unhardwired myself using Swift.

It is painful when I need to use them in another language.

After coding for decades, I’m glad to have one less thing to type.

If we want verifiable code, let’s move up a level of abstraction and build safer languages.

Counterpoint: I find a lack of semicolons entirely natural in ruby and python, and find it equally natural to insist on them in JS.

I have no idea why this is.

Y'know I was just musing over this exact feeling myself. I think some languages are more suited to it than others just by their "look". It's just a matter of trade dress — semicolons just go with curly braces. I find their absence in go jarring too.

What really keeps me up at night is wondering about the semicolon after a class definition:

  class Foo {
    ...
  };

This doesn't get one. It's confusing, but explicit in the ecmascript spec. That form of class definition is called a ClassDeclaration, and they don't take semicolons. I haven't checked, but I think what you've typed here is valid because the parser sees a zero-length expression after the '}'.

The reason it's confusing is because if you modify it very slightly, then the semicolon becomes required:

    let Bar = class Foo {
      ...
    };

In that context, precisely the same sequence of characters as you had before is now called a ClassExpression, and the semicolon isn't attached to the class, but instead to the preceding let.

I do find semicolons useful as an indicator at the end of the class expression that there's a let or a const somewhere above, but it's far simpler to just not do that and pretend that language facility doesn't exist unless you really, really need it.

I suspect it is exactly that simple. It's just what we're used to. However I do have a lot of sympathy with the argument (as here) that one shouldn't use an error recovery mechanism for stylistic effect. Automatic semicolon insertion is for fixing mistakes.

I would even go so far as to say that more characters/typing is objectively better than fewer any time fewer can introduce ambiguity, or confuse downstream (even future self) readers. Longer names are good except where shorter names well known and widely agreed upon idioms (example: I always type `acc` in a `reduce` callback, because that’s the convention; but I always type `index` instead of `i`, because single-letter names are frequently ambiguous). Explicit types at interface/module boundaries are better than implicit/inferred types because they prevent regressions, surprises, breaking changes and sweeping contract changes that wouldn’t otherwise be necessary. Stricter rules are easier to comply with than more flexible rules; typing a semicolon at the end of every statement accepting one is significantly less cognitive load than remembering which ones are special and no more burdensome than correcting surprising mistakes. And those benefits are significantly more valuable than fewer punctuation marks and keystrokes.

"implicit" or "automatic" semicolons should not exist.

If there are conceptual semicolons, they should be visible.

A language that is semicolon free should not have a rule for invisible/phantom/imagined semicolons, that is not the way to free the programmer from thinking about it.

Given how capable our editors are today, it seems reasonable to both require semicolons at the end of statements and expect users to configure their editor/formatter to insert these semicolons for them wherever possible. The inability of the editor/formatter to automatically infer the locations of semicolons would be tantamount to a syntax error.

If your editor can do that, so could the compiler.

So, why require them? The only reason I can think of would be readability, but I haven’t seen a language where that is a serious concern (scala gets close with its “there must be a blank line here” at times)

I think “no semicolons” nowadays is the option to go for.

> If your editor can do that, so could the compiler. So, why require them?

So that there's a step in between where the human can check the result

Editors can get away with things compilers can't- they are inherently interactive. If your editor gets it wrong, you can just fix it. If your compiler gets it wrong, you might get an error message, or weird behavior.

This is, for example, a major philosophy behind Rust's well-regarded error messages. The compiler rarely makes this kind of choice behind your back- instead, in reports an error message with a suggested fix-up for what it thinks you mean (and editor integration can apply those automatically).

Then the program source remains unambiguous and clear, there are fewer ways to confuse readers, but you still get all the benefits of "if your editor can do that, so could the compiler."

Certainly in enjoy Prettier (in spite of its foibles) for JS because of this sort of stuff. It removes whole classes of lint errors and adds semi-colons and doesn't care how I format the code as I wrote it.

Far too much thought and angst goes into things that should not need to be a concern. I used to feel more dubious about such tools, but after gofmt I tried Prettier and always use it now.

It's just one more bit of cognitive load that is gone. Indeed, if it doesn't reformat my code as expected I know I've made an error, so it's even quicker for me at noticing some bugs than ESLint messages which are already pretty excellent these days.

I agree. I have autoformatting turned on for everything and I just don't care what my code looks like because it’s consistent and I can get used to it.

I used to spend time configuring formatting settings to be exactly right, but nowadays I’m moving towards defaults.

My only problem with gofmt is that since Go standardized on tabs, I’ve encountered problems with noisy commits because people rewrap comments, and the comments will wrap differently depending on the individual’s tab width setting.

It's always seemed odd to me that a language with significant whitespace like Python became so popular, because human languages naturally have separators like semicolons and periods between sentences; otherwise, it all looks like one big run-on sentence. I admittedly work a lot in C and C-derived languages, and while reading I mostly disregard whitespace (as the compiler also does), so the latter is what code in other languages without statement separators/terminators looks like.

I see the point about dots, commas and colons being helpful. I just used them in the previous sentence for readability!

On the other hand, I worked exclusively in C-like languages until about five years ago (I almost exclusively use Python now) and I made the opposite observation. In English articles, indented lists and paragraph breaks really pop, helping me understand the structure of what I'm reading at a first glance. When I realized Python's whitespace gives me the same thing, I find it hard to go back to curly braces defining blocks unless there's an additional convention of whitespace-indentation on top.

My pet theory is that a lack of non-whitespace separators heavily discourages clever one-liners or compact sequences of operations and forces the programmer to write more readable code.

Python actually has ';', and I think you can write everything in one line if you really want to, e.g. in an inline script

  python3 -c 'import math; tau=math.pi*2; r=1; u=r*tau; s=r*r*tau/2; v=r*r*r*tau*2/3; print(u,s,v)'

In this simple script you can, but eventually you need an loop or something and you realize you're kind of stuck:

before_loop; while something: step1; step2; after_loop;

Note that after_loop is still inside the loop. Also you can't have multiple colons on the same line, so this doesn't work either:

if test: do_this; else: do_that

That gives a syntax error

I think of the different approaches available you can make a language workable & convenient with (1) explicit semicolons, like C/Java (2) implicit semicolons, like JS/Go/Haskell and (3) no semicolons like Python/Lua. JS’s specific implementation is definitely the odd one out, like the article says.

One of the funny things about Go’s automatic semicolons is that you can easily trip up gofmt if you type sloppy code and expect gofmt to clean it up for you. For example, let’s say you type this:

    func foo() {
        if x < 3 {
            return 1
        } return 2
    }

You think, “Oh, I missed the enter key when I was typing it but gofmt will fix it for me.” No, it will not. Syntax error.

no semicolons like Python

That's not true, Python has semicolons and they're invaluable for composing `-c` command line scripts in a whitespace sensitive language

S-expressions gracefully disambiguate all three examples in the note, with nary a semicolon nor a comma in sight.

  first
  -second

will either be

  (- first second)

or (though this is a bit silly)

  (do
   first
   (- second))

and there is never any danger of mistaking one for the other.

I just solved this problem in Oil. In our two main influences, shell and Python, newlines are significant.

However Oil uses {} for blocks, and it also has dicts.

Originally I thought we should have something like %{} for dicts to give the lexer and easier job when deciding when newlines are significant. Oil uses Python's rules of newlines being ignored between () and [], and it would make sense to ignore them inside {} but not %{}.

But I found I could just makes newlines significant within {} and then put them in the grammar for dicts, as well as for blocks. It seems to work fine.

http://www.oilshell.org/blog/2020/10/big-changes.html#dicts-...

I am open to feedback if anyone wants to kick the tires :)

I find Mathematica very neat in the semicolons department (possibly because it's the only Lisp I know, and because all Lisps are neat in this way). In Mathematica, a semicolon is nothing more nor less than an infix syntactic sugar for `CompoundExpression`, and `CompoundExpression` has the attribute `HoldAll` to prevent eager evaluation of the syntax tree. You could roll your own very easily, complete with its own infix operator.

You can also override the default evaluation strategy of `CompoundExpression` if you `Unprotect` it first, or you can define upvalues so that your own things behave differently when they appear inside `CompoundExpression`s. It's all very lovely.

I've never thought about adding upvalues to CompoundExpression -- what kinds of applications have you used this for?

Literally none, ever. But you have the option!

In fact, here's a silly idea, executed with two kernels:

  SetAttributes[Parallel, HoldAll];
  CompoundExpression[x__Parallel] ^:= ParallelCombine[Identity @@@ # &, {x}]

  In[28]:= (Parallel[Pause[3]; 3]; Parallel[Pause[3]; 4]; Parallel[Pause[3]; 5]) // AbsoluteTiming
  Out[28]= {6.00864, {3, 4, 5}}

I'd never seen UpSetDelayed used before (I've always used the longer /: notation), that's interesting.

A silly idea I was thinking about is supporting

  With[{x = 2}];
  With[{y = 3}];
  x + y

for

  With[{x = 2},
    With[{y = 3},
      x + y]]

to reduce nesting.

I'm sure there are subtle bugs, but it seems to work in a simple case:

  SetAttributes[Have, HoldAll];
  Have /: HoldPattern[CompoundExpression[x___, Have[s_Set], y__]] :=
    CompoundExpression[x, With[{s}, CompoundExpression[y]]];


  In[698]:= (Have[x = 2];
             Print[x];
             Have[y = 3];
             Print[y];
             x + y)

  During evaluation of In[698]:= 2

  During evaluation of In[698]:= 3

  Out[698]= 5

Note that you can nest within a single With:

  With[
    {x = 2},
    {y = 3},
    x + y
  ]

Though this form is not (yet) documented.

Crucially, y could refer to x here.

It can really help with the indentation.

I use notebooks primarily -- it would be nice if syntax highlighting didn't put x+y in scary red :-)

Something I've wondered: would it be possible to support With[x = 2, x + x] when there's just a single variable? It's certainly very low-priority, but it saves a few keystrokes when doing quick calculations.

Yep, the proper syntax highlighting will follow when documented.

There was actually a massive internal discussion in 2015 re: With[x = 2, x + x]. And With[x = 2, x + x] actually did work for a time.

Unfortunately, several other ideas like With[x, x+x] and Module[x, x+x] (which was unintuitively NOT implemented as Module[{x}, x+x]) also got thrown into the conversation and things became confused, these uses were seen to be problematic, and soon the baby was thrown out with the bath water.

So technically yes, With[x = 2, x + x] could work, but it is unlikely that it will ever be implemented in product.

Mathematica is surely not a Lisp, as exemplified by the above. You may be able to find old commentary from Richard Fateman about that, e.g. from comp.lang.lisp, and from Kent Pitman on Wolfram's attitude to the language.

you don't semicolons, but they can be handy. One way of looking at them is as an infix operator (a monad if you want).

Another remark is that Python parsing can be hard as a single newline can actually end multiple blocks. Lexers are not designed to have a single char denote multiple tokens.

Can't semicolons essentially be either infix or postfix (terminators), depending on the language?

Parsing the offside rule -- I know that's not actually Python -- can be done straightforwardly with a parsing combinator transformer (originally due to Hutton?) if tokens are associated with layout information.

BASIC had the 'End' keyword instead of a delimiter.

Semantics were different. End ended the program. https://www.c64-wiki.com/wiki/END

Pascal however used BEGIN and END to denote the beginning and the end of blocks. Maybe your memory got garbled.

VB6 used "End X", such as "End Function" and "End If"

For what it's worth, Fortran goes further with (optional) "end function X".

Reply on news.ycombinator.com