Comment by BoppreH | Hacker News Reader

Comment by BoppreH | Hacker News Reader

Parent Full thread

I've been working on a Structured Code Editor for a couple of years:

https://i.imgur.com/wvcduDk.png

(in the picture above "string" is selected, and "find" is slightly highlighted because its on the same level. this helps visualize the tree and plan your movements)

It actually started as my final project during undergrad. Here's my 78-page thesis on it (unfortunately Portuguese, but has English pictures):

https://projetos.inf.ufsc.br/arquivos_projetos/projeto_1398/...

Structured editing is much, much, MUCH better suited for programming than plain text. Syntax preservation and source-display separation are game changers, and they are not the only benefits.

Unfortunately it has a Vim-level difficulty curve and people don't realize how much time they currently waste on syntax mucking. This makes marketing kinda hard, even with a fully operational and well polished implementation.

Fun fact: structured editing also works with non-programming structured text, like JSON, HTML and CSS!

> Structured editing is much, much, MUCH better suited for programming than plain text. Syntax preservation and source-display separation are game changers, and they are not the only benefits.

You can do a lot of things with a parser and a rich code editor...e.g.

http://research.microsoft.com/en-us/projects/liveprogramming...

Parsing is just a detail, if you can get it right, you don't necessarily need a structured or projected text editor. And its not even the "big" detail, which would definitely be type checking, and structural isn't going to help you much there (that is, if you want any kind of fluidity).

Interesting research, specially because it has a lower learning curve. I see it as a halfway between fully structured and fully textual.

The large screenshot[0] is a textbook example of structured editor, but the cursor moves around as if it was text. It looks like the document is modeled as a combination of complete structures plus a few incomplete pieces of text. That's interesting because you get the benefits of structured display without paying for structured input.

However, I'm not sure I would use this. It's not clear how to call the Δ function, for example. You can consult the manual or ctrl+c the character, but that's not desirable. Also, does erasing the dot after ≱ yield ≥ or ≯?

I do like structured input. You type an awful lot less and there's no meddling with commas and parenthesis. The learning curve is tough, but Vim users are here to prove people are willing. If structured editing lives up to the hype, of course.

Anyway, I'm always glad to see research in this area.

[0] http://research.microsoft.com/en-us/projects/liveprogramming...

> Also, does erasing the dot after ≱ yield ≥ or ≯?

Yes. Actually, in my newest prototype, the dots disappear and we treat ≱ as having 3 characters (the first is !, the second is >, the third =). Deleting the first character is deleting the ! (you get ≥), or deleting the last character is deleting the = (you get ≯).

Very neat!

I think the issue with high learning curve tools is not that the learning curve is high, it's that the benefit seems (rightly) dubious at the beginning.

You can find videos of people whizzing along with emacs or vim, and it's not difficult to see the productivity gain. So it'd be cool if you had a video of what it looks like to program with this thing.

Thank you, and you're absolutely correct. I think a side-by-side video comparison would work even better, because it's not immediately clear how much time is spent moving commas around, playing with whitespace or hunting parenthesis.

That said, I'm not trying to convert people just yet. It still needs polishing and support for a few more languages (right now it only does Lua, Python and basic Lisp, which is not very exciting).

Is your editor open source? It looks really interesting.

I haven't opened it yet for two reasons:

1) Premature open-sourcing complicates commercial projects, and a man gotta eat.

2) Because it would be a first, any flaws in the implementation will be assigned to the concept of structured editing itself, hurting the development of alternative implementations. Even if my vision of it fails, I want other people to have a shot without fighting incorrectly formed opinions.

In the end, it's the ideas and not the source that are important. It's like developing for VR now, or mobile a few years ago. The space of possible interfaces is so big that most of the effort goes into combing it.

Something I notice about the UI you use is that it tries very hard to emphasize the structured-ness of the underlying engine. Perhaps the middle ground goes back towards Emacs(or rather, a new iteration of the Emacs concept) - plain old text editing is the default, but the underlying editing engine is always working in structured form.

Ultimately, I have to concur with the people calling out the lack of need; if my sustained average is 80 lines of code per day, I'm not being bottlenecked by my type time, but I am bottlenecked by typing errors that cascade to runtime. I would welcome seeing more modes that simplify specific, necessary-but-error-prone tasks like copy-paste-modify.

I hear what you're saying. But I've been programming for 35 years. I've found that getting syntax right is at worst a tiny percentage of my time/energy/brain cost when doing programming. Once out of the newb phase. Once I've fully grokked the language and I've had enough ramp-up that I'm in "the zone" it's nearly effortless to type syntax-perfect code on my 1st attempt. Even getting built-in library calls and macros right, on 1st attempt, becomes nearly effortless, once in that frame of mind.

It's the think-design-code-test loop that is the biggest driver on my time/energy/brain cost. Not syntax conformance. Conformance becomes like a musician having played a certain instrument for enough years, you acquire muscle/eye memory for what note requires what finger/body/mouth positions/behaviors, nearly a one-to-one map, autonomous.

Do agree that anything that enforces syntax correctness, upfront, is helpful for newbies, especially "forever newbie" use cases, the non-expert users.

Try this exercise: change the first line into the second, with the cursor starting at the caret. Do it slowly and pay attention to each key you touch.

  lineTo((10, 35), (20, 15))
         ^
  lineTo((20, 15), (10, 35))

Maybe you had to hold Ctrl+Shift and mash the right arrow to select the first tuple, and then Ctrl+X. Or you typed "d2t,", paying attention to the inner comma. Probably between 10 and 20 movements, all over the keyboard. Three or four seconds, and to me feels like using a blunt knife.

And yet you are just moving one element down a list. You do it all day with parameters in a function, statements in a block, items in a literal list. Which is why it has a dedicated command in my editor: "Move down" (https://i.imgur.com/wvcduDk.png). It's a single key. Works in all cases mentioned, for all supported formats.

I'm not saying a structured editor is for people who forget semicolons. It's for people who have to work with semicolons.

That's a lovely example. I was going to post that I don't like structure editors, but your example is two keystrokes in Emacs: c-m-F and c-m-T -- which I now (after 35+ years) realize is a structure editor.

(one of the first things I worked on when I started at PARC was adapting an Emacs clone since I didn't like the Interlisp structure editor. Little did I know!).

again... I hear you. been there, done that, have the T-shirt, over 35 years of coding in multiple domains, including GUIs, graphics, games, code-driven visual layouts, etc.

and I'm saying with a low-overhead deterministic-optimized editor like vim, and the right person at the keyboard, this can be done very quickly and accurately. and again, to continue your example, the hard part is not getting the syntax correct it's ensuring the resulting visual image -- your example suggests a vectory visual artifact ala OpenGL or SVG, etc. -- has the right shape and position. Syntax is something my brain/eye system just tells me, instantly, RIGHT or WRONG. our brains are great at this.

I'm not saying you're wrong. I'm saying for non-newb, non-lame programmers it's a use case that optimizes for a cost that's one of the smallest costs imposed on the programmer. not unlike "premature optimization".

I'm not saying syntax-enforcing keystrokes are a bad thing. I do think there are benefits to having a set of syntax-generic consistent keystrokes, like vim, across all the various syntaxes one has to deal with, day in and day out. If the only thing I ever had to edit was C files or JSON, that's it, nothing else, then yes having a C or JSON-semantic keystroke-restricted inescapable mode (with prompts, wizards, etc.) would be a help. (Which arguably is how all the big fat modern IDEs have evolved towards anyway.) But I'm very aware of the phenomenon where one can gain in the small but lose in the large. Local maxima, etc.

Also benefits to having screen match print, etc. Being grep-friendly, diff-friendly, textual VCS-optimized friendly, etc.

Local maxima. The sneakiest wrongs are right in the small.

I don't disagree with you, but:

> non-lame programmers

How many programmers are out there that are "lame"? Honestly, I still to this day work with code written by developers with years of experience who still get syntax wrong. Often I forget that most skills follow a bell-curve, and most developers are not one-with-their-machine-and-language -- hell, even I'm probably not, although some days I might feel like it. I think having a system that removes that barrier would allow that vast sea of average programmers to move closer to that sublime moment where your thoughts are transcribed in code, perfectly, in one go.

> How many programmers are out there that are "lame"?

from a few decades of observation in the wild, I'm sad to say that it is a surprisingly large percentage. the market demand for programmers seems to exceed the supply of those of us who truly can.

I don't agree with the premise that structural editing is for developers that are (necessarily) subpar to begin with (see my other comment).

I do think the "killer app" use case for the modalities you're talking about are data entry kinds of use cases. Where a non-expert user, perhaps one who has to deal with a variety of formats, very randomly, is required to type things in that strictly confirm to a perfectly-defined syntax. And he/she has a gigantic amount of data to enter manually, by hand, in a short time. Higher throughput is better, but also highest correctness is the other dimension. Tiny percentage time overhead spent on thinking, design, test. Mostly on data entry by hand. Then yes your approach starts to yield disproportionately higher benefits.

Arguably one reason why XML/XSD/XSL took off. It was not just yet-another-structured-text-format like CSV and JSON. It also had an "official" way to express and constrain an application data format, and generic query and mapping languages. Great for low brain, high volume, high repetition use cases. However... programming itself is high brain, lower volume, low repetition.

Nice example, but...I work with graphics code a lot (I'm a game developer), as well as in tons of other domains. That operation wouldn't be worth the mental storage (and certainly not a key!) to memorize. I can't think of the last time I would have needed it. Maybe when getting a pair of function parameters in the wrong order?

The example you gave wouldn't ever be a program I'd write. Magic numbers? Hard-coded rendering? Rendering something that can't be edited by an artist? All bad. I understand that it's just an example, but can you come up with one that would be valid in production code?

I know you were looking for a simple example, but in addition to the comments others have made about syntax not being a big deal for an experienced developer, this looks like a helper operation for developers who are doing it wrong to begin with.

Aside from that, years of user-experience research shows that modal editors are worse than the alternative (if you can hit one key, that means you can't type text in that mode; sorry vim-lovers, but it's true).

Yes I tried your experiment, and I could get it down to 7 keystrokes in my editor. I type quickly, so that's a couple of seconds at most; less than a second if I'm in the zone. With how rarely I do that operation, I'd not bother trying to optimize it more (another comment mentioned premature optimization).

But if I did need to do it a lot, creating a new macro that did the right thing wouldn't be hard; in a couple minutes I could bind a simple version of the command to a key permanently. Also, if I were modifying many similar lines to change them, I have options: Refactoring tools (if we're talking a function signature change), simple keyboard macros, or smart multi-file search-and-replace regular expression conversions.

And all of this in a "text" editor with a low barrier to entry (unlike vim OR emacs) -- though one that understands some structure at least.

Isn't d% a more likely vim command?

Although, you'd think vim would have a general "transpose text object" operator. If it were bound to some ŧ, then you could just do %ŧ% (here emulating Emacs' C-M-f C-M-t)

Short of having a transpose verb, here is what I cape up with (using the arg text objext):

    dia      # 1. delete argument under caret
    df       # 2. delete separator (comma and space)
    %        # 3. move after argument (i.e to end of tuple)
    p        # 4. paste separator
    "2p      # 5. paste argument

I'm not doing this out of the desire of golfing but to analyse and point out that what vim lacks in this specific case is how to semantically do step 3 (ga/gA ?) and possibly step 2 (because what if there's no space? a "separator", like "surround", might be useful), upon which one could easily and semantically build a "ŧa" (transpose argument) sequence (which could also become ŧ2a, ŧ3a to swap with the second or third next argument, and capitalize the a to swap with the previous).

That said vim is still "only" a text editor, but it goes a long way at being a general purpose one with semantic operations.

Thinking structurally, I would use `da(`. Of course, `d%` is less keystrokes.

Using vim with the sideways (https://github.com/AndrewRadev/sideways.vim) plugin and the following mappings in .vimrc:

    nnoremap <silent> <leader>h :SidewaysLeft<CR>
    nnoremap <silent> <leader>l :SidewaysRight<CR>

I pressed two keys, job done: ,l

Double click, type "20", double click, type "15", double click, type "10", double click, type "35".

(All without typing the actual quotemarks.)

If I had realised earlier that it's (x,y) to (y,x), I could've done it without the keyboard at all.

An advantage of using something that isn't text is that our tooling can also be better. I think we've all had the experience of git diffs being complete messes due to some variables moving around but git matching the wrong parentheses

I agree that with a language like python (or most all languages) syntax doesn't become an issue, but if we can reduce complexity on one part of the editor, maybe we can introduce stronger _general_ refactoring tools (not just language-specific ones).

Anyways I think there's some good research happening in the domain of program editing (light table being one thing, albeit derived from some other tools), because the objective isn't to write hello world quickly, but to be able to quickly and confidently iterate on larger codebases (much like what vim lets you do quickly for simpler formats)

>Once out of the newb phase. Once I've fully grokked the language and I've had enough ramp-up that I'm in "the zone" it's nearly effortless to type syntax-perfect code on my 1st attempt. Even getting built-in library calls and macros right, on 1st attempt, becomes nearly effortless, once in that frame of mind.

And yet, it's this "nearly" that creates all the huge C buffer overflow and memory corruption bugs...

Are you suggesting that syntax errors create memory corruption?

Quoting: "even getting built-in library calls and macros right, on 1st attempt, becomes nearly effortless"...

Getting the syntax to parse != getting the correct syntax (meant for the job).

Even an off by one error in a library call can create memory corruption. Preprocessor macros? They can fuck everything if done wrongly.

Those are semantic errors, and a language aware editor will not fix them.

That sounds like a language problem, not an editor problem.

> I hear what you're saying. But I've been programming for 35 years. I've found that getting syntax right is at worst a tiny percentage of my time/energy/brain cost when doing programming.

I don't want structural editing because I can't remember the syntax of a language. I want it because I want to edit the code more efficiently. I don't know if I'll really be more efficient, but that's nonetheless what I want.

People who use more plain editors like Gedit or Notepad++ might scoff at Emacs or Vim users. But if they think that power editor users use them is because they can't write or navigate text, they're really missing the point. The point is to navigate and edit text more efficiently.

Reply on news.ycombinator.com