The Hardest Program I've Ever Written – How a code formatter works (2015)

Any more general comments on applying algebras to solving programming problems would also be appreciated.

My rough understanding of the benefits/rationale atm:

1) the algebra is a nice/minimal factoring of the problem

2) being able to formally classify it means we know a bunch of its properties, which tells us certain approaches are definitely (im)possible (and these properties are drawn from a fairly small set: e.g. associativity, distributivity, existence of inverses)

3) If the algebra is explicitly represented in code (maybe in a library), we can use that general structure to verify (and generate?) certain aspects of our particular problem solution.

Edit: The paper mentions "Over the years, Richard Bird and others have developed the algebra of programming to a fine art" —and I found Bird's book "Algebra of Programming". Seems like what I'm looking for maybe, but would be grateful to hear of any alternative resources as well.

This shows that writing a formatter is a task that is probably never finished :)

Obviously a formatter can't change the language, but just want to point out that the issue there is really that the arguments are linked logically but not in the code.

I think a language designed around using an auto-formatter will work better in that scenario. See, for example, Elm—where the use of auto-formatting is essentially universal.

> We do not yet use dartfmt. Flutter code tends to use patterns that the standard Dart formatter does not handle well. We are working with Dart team to make dartfmt aware of these patterns.

So it looks like they want to use an auto-formatter

> In directories where most contributors have already adopted clang-format, and code is already consistent with what clang-format would produce, some teams intend to experiment with standardizing on clang-format. When these local standards apply, it will be enforced by a PRESUBMIT.py check.

From the looks of it it's a lot of the tree [1].

Where manual formatting gives you some freedom in meeting the style guide, it also brings along with it errors (style guides are massive and it's not possible to remember everything) & sometimes you just make a typo. This also means that your reviewers are spending time validating against the style guide which is not only the most wasteful use of their time, it's still an imperfect process because reviewers are just like the author - they don't have the entire style guide in their head at all times and they may miss violations.

[1] https://github.com/search?q=CheckPatchFormatted+repo%3Achrom...

    // convert from screen coords to clip space
    // formatting-preserve-start
    clipX = mouseX / screenWidth  *  2 - 1
    clipY = mouseY / screenHeight * -2 + 1  
    // formatting-preserve-end

Sadly, ignore comments of the Prettier formatter (https://prettier.io/docs/en/ignore.html) are not this powerful – they only support ignoring the next line of code.

Maybe using some natural language processing, machine learning, and crowdsourcing, a formatter could understand intent sooner or later?

Your style should require brackets, and probably not be a single line

Here’s my obligatory (and excellent) blog post by John Carmack where he adopts the functional style for a month: https://www.gamasutra.com/view/news/169296/Indepth_Functiona...

Oh, how I wish. Dart is no C/C++ with preprocessor insanity, but it has quite a rich grammar.

> The challenge is when the program is _not_ well-formed, i.e. has errors

Being able to format erroneous code has come up as a feature request a number of times. It would be useful because then you can format even in the middle entering code into your IDE. We do have a very robust parser that can handle incorrect, incomplete code and give you a partial AST.

One of the reasons I've hesitated so far is that testing that in any sort of rigorous seems really difficult to me. Not to mention even defining what "correct" formatting is with code like that.

> refers to to code elsewhere, so you can't even decide if it's well-formed or not.

Thankfully most languages are context free, so that's rarely a problem outside of C/C++. :)

Dart does have a couple of semi-contextual syntax corners that could potentially affect formatting. An expression like:

    foo.bar();

* Call the method "bar" on the local variable "foo".

* Call the top-level function "bar" imported with prefix "foo".

* Call the named constructor "bar" on the class "foo".

Because dartfmt does not do symbol or import resolution (for many reasons), it can't distinguish those cases. In practice, all that means is that we don't have th eability to have different formatting rules for those different cases and they are all formatted uniformly. So far, that seems to be an acceptable limitation (though it would be nice if we could treat import prefixes as stickier).

It's really nice knowing as soon as you hit ctrl+s whether you've just written in a syntax error or not.

Common Lisp has big, meaty, possibly-undecidable macros which compilers need to expand, but Common Lisp formatters are piss-easy because the basic grammar doesn't change unless you go so wild you invent Dylan or something.

So, yes: In the general case of Really Advanced Programming Languages, the amount of information you need to format is the same as the amount of information you need to compile, but that's fine, because it means you can flip it around and see that if you can't format cleanly you're never going to compile anyway, so what's the point of formatting code you can't even use?

I don't like auto code formatting in my workflow and I don't like "easy"-syntax languages like Go. I see it is of value for big treadmill-code-machines like google. They want every codemonkey to be easily replaceable.

But for me code is also art and fun. I align my assignments and statements and I use different code-formattings according to the situation.

If I do all right I only write the code once but read it again and again for years. So I like to spend some time to format my code nicely, whatever that means :-)

Writing a code formatter is a fun challenge and sometimes its useful to have a good formatter but please not in my day-to-day job.

Because of a handful of syntax quirks particular to Dart, it's really difficult. Most other languages that have optional semicolons were designed from the start to make them optional. Dart was not. On top of that it:

* Uses C-style variable declarations which means there's no keyword to indicate the start of a variable declaration.

* Likewise uses C syntax for function and even local function declarations. Again no keyword marking the beginning of a function.

* Has complex syntax for things like function types which means a type annotation can be arbitarily long and may split over multiple lines.

* Has lots of "contextual keywords" that are not true reserved words but behave like them only in certain contexts.

* Has a rich syntax that overloads a lot of tokens to mean different things in different contexts. "{}" can be an empty map, empty set, empty block, empty function body, or empty lambda body.

All of that makes optional semicolons just really difficult. I haven't given up entirely. We have a thing now called "language versioning" that lets different Dart libraries target different specific versions of the language. That lets us evolve the language in nominally "breaking" ways without actually breaking existing code. (For example, this is how we will migrate the world to non-nullable types.)

That may give us a way to make other grammar simplifications that would then also let us make semicolons optional. But syntax changes like this are always difficult. People have very strong opinions about you changing their existing syntax under them, and the value proposition in return isn't super compelling.

There's an old saying, the best time to plant a tree is twenty years ago. The second best time is today.

Language syntax isn't always like that because the migration cost once you have a lot of extant code (and users) can just be too painful. The best time to make semicolons optional was twenty years ago. The next best time may be today, but it may just be never.

JavaScript gets this wrong in both directions, sometimes unexpectedly ending a statement (e.g. in `return \n 0`) and sometimes unexpectedly not ending one (e.g. when a new line begins with an open-parenthesis).

Python's method (newline always ends a statement unless it's inside (), [] or {}) is straightforward, but makes the language syntax strictly line-based. This matches Python's significant indentation, but can it work in a language without it?

Another option I've seen is for all newlines to end statements, unless they follow a token that cannot end a statement. Unfortunately that means that the following two assignments have different behavior:

    foo = bar +
          baz

    foo = bar
        + baz

Block comments are a little tricky and there are a handful of heuristics to handle them. Generally they stick to the preceding token. Block comments are rarely used in Dart and when they are, it's often just a small marker to describe the previous element like:

    foo(null /* missing */, blah);

> If so, what to do if there are already indents in the comment?

It doesn't touch the body of the comment itself. There's too many ways for that to go wrong given all of the various things a human might choose to stuff in there: ASCII art, tables, prose, etc.