Hacker News Reader

Comment by YmiYugy | Hacker News Reader

What does it mean though for a GUI to be keyboard-driven? The obvious way is that every action simply gets a shortcut assigned. My counter would be that that is not really keyboard-driven, but merely keyboard-compatible.There is the issue of discoverability. The best practice right now seems to display the shortcuts of buttons in tooltips, menu items, or when pressing a different shortcut. I’d content that buttons are a fundamental mismatch with keyboards. A keyboard driven UI shouldn’t have buttons. The issue is that genuinely keyboard driven UIs like CLI or TUI suffer terrible discoverability that being the reason that mouse driven UI exists in the first place. So can we have a keyboard-driven UI that is as intuitive as clicking with a mouse?

View on HN

Yes, they should be discoverable. There are platform conventions for this: on Windows you'll see underlined letters; menus show you the shortcuts. Tab is a pretty much universal "move to next field" shortcut and I don't think that's particularly discoverable if you don't already know it.

There's nothing wrong with a keyboard driven UI having buttons as long as they advertise their key, though. That's one way discoverability is supposed to work. It means you can fall back to the mouse when it makes sense.

Since you mentioned Windows, Microsoft made Ribbon which uses Vimperator-style navigation to make toolbars and buttons discoverable as long as you know one hotkey (Alt). It can also be hidden until triggered with a hotkey. This is miles ahead of most discoverability tricks in power user interfaces, including fuzzy command search which is not guaranteed to remain stable as new commands are added.

One downside to the ribbons that I see is that they somehow inverted the hierarchy between menus and toolbar buttons: Previously, menus were the slow but predictable way to find a command, and both toolbar buttons and keyboard shortcuts were just that - shortcuts. Menus were a sort of "ground truth" which commands existed. They also had a standard way to be navigable through the keyboard: You can press alt and then simply use the cursor keys.

With ribbons, the menus were gone and the toolbar buttons became the "ground truth", but in a much more haphazard way.

Fuzzy command search goes a bit back to menus, but as you say in a way that's much less guaranteed to be stable.

> A keyboard driven UI shouldn’t have buttons.

You mean because they're basically just in the way, right?

I know it's asking a bit much, but IMO advanced software ideally has

- buttons, pretty colors and icons where they make sense

- keyboard shortcuts for everything

- complete documentation that ships with it (sure "it's online anyway".. so find a way to automate putting what you have on the website into the release in a form that works offline)

- preferences to hide some or all buttons/toolbars, turn off the icons, and change some or all keyboard shortcuts. GUI is cool for that, text files are also cool. In other words, why not have everything? Maybe you could even compile a version that strips out all the stuff that can be turned off, or at least hardcodes some preferences, so the compiler can skip some things that are no longer called etc.

Not a must-have but nice-to-have: tooltips for all things and maybe advanced help, so you don't need the documentation as much, or not at all for simple use cases. But if the docs are complete, I think it's fine to keep the program and development process simpler by not duplicating that in tooltips. Don't go out of your way, unless you want to.

But as I said, that's for "advanced software", by which I mean software you spend a lot of time in. Browser, text editor, email client, file manager, graphics software, and so on. With simpler tools I'm usually just happy they exist.

My logic for determining if a GUI is keyboard driven is pretty simple: if I can fire up a screen reader and completely drive your UI via the keyboard, then by definition your UI is keyboard-driven. If your app ever requires me to emulate mouse clicks to drive it, then it isn't entirely keyboard-driven and that should probably be fixed. This is even something that can be measured/tested for on platforms like Windows where you can both read the accessibility tree (which in itself is amazingly powerful particularly for testing) but you can also simulate arbitrary input.

I think a first step would be a sane tab order, so you can easily navigate to a control using tab or cursor buttons. Second step are keyboard shortcuts.

I don't see why removing the buttons would be desirable. All that would do is to make the GUI unusable with the mouse and also make the discovery of the keyboard shortcuts much harder.

One of the reasons I loved learning Helix is that it shows you the list of possible next keypresses every time you press the first key in a combo. Amazing for discoverability. It even works for user defined shortcuts!

We know what it means for a GUI to be keyboard driven now:

Edit with Copilot[1]

Press the Copilot key on your keyboard[2] and type the thing you want to happen on your keyboard in English without leaving the home row. Forget the keyboard shortcuts, the path through the menus, the button location in the toolbar, the command name in the omnibar, the marketing name of the feature. Forget queueing up keyboard keys in an input buffer based on memorising which dialog will popup next, or the tab ordering on this or that screen. DWIM is back, and this time it still isn't determinstic, but it's the way we are going to manage ever-growing software features and task complexity and finally turn computing declarative instead of imperative.

"Move rows with dates from 2020 to the bottom of the table"

"Change the date format in the table from mm/dd/yyyy to yyyy-MM like 2026-Aug"

"Style the table with alternating dark/pale contrasting colour backgrounds and a different header row colour"

[1] https://support.microsoft.com/en-us/word/edit-with-copilot-i...

[2] https://blogs.windows.com/windowsexperience/2024/01/04/intro...

LLMs do solve the discoverability problem which underpins every power user feature ever. It is fantastic to be able to describe what you want without the domain knowledge that was previously a prerequisite to achieving that goal. Hopefully as more systems create declarative ways to interact with them (MCP, clis), power users will be able to profit from this as a side effect as well.

They sidestep it (partially), but I don't think they solve it. Sure, you'll get your task done if the agent does it for you, but the agent won't show you how to do it without their help. So the next time you have to do the task, you have to ask them again.

I also think this is misunderstanding "discoverability" a bit. For me "discoverability" also means you get an understanding of what options are available at all in the system, which can be helpful to get a better idea of what you want to do in the first place.

Seach and chat-based UIs always assume you already have a perfectly thought-out plan what to do (conveniently in terms if the software you're working with) and just have to type it in.

Look how many programs and features exist on Earth.

Project this forward to the worlds of 2050, 2076, 2126.

Your "understanding of what options are available at all" becomes moot when every program has so many options that you couldn't read them all in a lifetime even if that was all you did. Or when the answer is effectively "everything". Future programs will vibe single-use features like you make a disposable regex for every search. You don't cement xyz\d{3} into a feature, and you don't ask "show me all patterns I could search for".

At what point is "how to do it without their help" a strange thing to say? Select text and press Ctrl+B for bold, you reply "no I want to do it without the software's help". What? Typing [b][/b] markers, coding a function call, pointing to a bolded font - there's no world where you "do the bolding" and the computer "doesn't help". There is a submerged iceberg of bolding with a bit poking above the surface where you press Ctrl+B and feel you are doing meaningful work. Like the story of instant cake mix, it was a market flop until they removed powdered egg from it and turned it into "just add an egg" cake mix, which lets us feel like we are still cooking.

Why does Future Word need an "insert picture" feature? It can press enter a few times and display that picture of your dog in the gap, easy. If you had a personal assistant they would not have a please-call-my-partner-and-say-I-will-be-late-home "feature" and you wouldn't want to micromanage which cellular network and which audio codec was used, and you wouldn't say "I want to call my wife without the telephone's help". "Insert picture" doesn't need to be "a feature" any more than the regex "xyz\d{3}" needs to be "a feature". Future turbo-LLM has seen you write every document you've ever written, future Microsoft Word has been trained on every document ever uploaded to Office 365/SharePoint/OneDrive. Future computer tracks your typing when you pause for emphasis, or hears when you speak with emphasis, or the cameras observe as you scowl for emphasis. When you indicate the bit that you want emphasised, it puts bold markers on that bit, and you won't give a damn how - whether it put [b][/b] markers or whether it printed the document through a virtual fax machine, simulated a hand drawing more ink on the simulated print, virtually faxed that back to itself, ran OCR on the incoming fax, identified the added ink as the bolded font, and put <p class=></p> and a CSS stylesheet indicating the bold font.

> "Seach and chat-based UIs always assume you already have a perfectly thought-out plan what to do"

What? CLIs assume you have a perfectly thought-out plan, and understand the internals of the tool, all its options, and how they combine. You either type `fffmpeg --foo-transform --start-frame=234 --bounding-box=0,0-100,100 --option=reticulate-splines --reticulation-formula-bellard-optimal` or you get an error.

A GUI assumes you have a plan and you can find your way through the menus to find the foo-transform, then the popup dialog box will prompt you with textboxes, comboboxes, radio buttons, for the options, which you can set using your plan. You still need a plan but you don't need as much software internals.

Chat based UIs are even more of that; you type "I want it to look like an old photo" and it says "here I setup the foo transform for you with options that will make it sepia toned, here's preview, accept?".

Yes, there should be a difference between keyboard accessible and driven.

Screen readers allow users to use their keyboards and other assistive devices to navigate websites and apps by tabbing through elements with extra controls and shortcuts to skip to regions, interact with elements and such.

To that degree I agree that all GUIs should be keyboard accessible.

This is obviously not what the original author had in mind, if he had to tab through every item in a web app he would probably get frustrated rather quickly. What he is imagining is like TUIs where all the interactions are through keyboard shortcuts and macros. These are not popular with the majority of users of modern day software, especially if the software is something that is used infrequently. I think we take for granted how component we, people who create software, are.

What has become more popular are interfaces that expose extra controls with a single key press, where the user can type to get the result they want. So instead of memorizing the keyboard shortcuts to dozens of actions, you can press something like Crl+K or / to summon a palette where you can search for the action.

To me spacemacs nails the discoverability aspect through helm-Mx-fuzzy-matching and which-key

No clutter, but hints come as you stutter on what to press next

Oh, finally somebody who gets it! That was incidentally my major disappointment with vim: it is keyboard driven, but the UX wasn't designed to be a perfect keyboard experience (which I expected, given how old it is).

Some examples of what you are asking for can be found in videogames due to controllers

Even as a fairly confident nvim user, I still use and recommend <https://github.com/folke/which-key.nvim>

Even as a devoutember of the church of emacs I couldn't live without the emacs version of which-key [1]

[1] I was going to link to a repo, but I just learned that which-key is included in Emacs out of the box.

It's perfectly possible to design a GUI without button controls, where literally all the interaction is done with keyboard inputs. It's still not a TUI: it has graphical visual layout, and the UI signaling is out of band with respect to the data it's presenting.

"What does it mean though for a GUI to be keyboard-driven?"

For literally decades after they were state of the art, people who used old-school mainframe TUIs would still swear by them.

I think the key difference is that they would buffer inputs. You could be on the "Receipt" screen, and know that if you needed to enter the code for some rebate receipt or something, you could hit F6 - F2 - down - down - down - {start typing code}, and even if the UI took multiple seconds to actually get to the point that the code was being entered, the input would be kept.

Normal GUIs have discarded this for fairly good reasons. Especially when it comes to queuing mouse clicks. If you get down to the nitty-gritty of what it means to queue a "mouse click" it get very conceptually complicated. But you could create a GUI toolkit that still had the same idea integrated into it for a keyboard interface, if you have well-defined points where you are and are not accepting keyboard input.

I would submit that "keyboard-driven UI that is as intuitive as clicking with a mouse" is actually the wrong direction to cut on. You need to beat the conventional mouse GUI completely in some other dimension, one where it can't even compete. This is the only one I know of that has historical precedent.

The downside is that on a desktop I'm pretty sure you're looking at a completely new widget toolkit. Trying to force this on the current ones is going to be hard and could even be impossible.

If I were going to create a new web frontend framework, this is what I'd focus on to have an actual new feature that isn't just a respelling of the current set of features. And I can't guarantee that it won't also be missing something crucial, though, I think everything you need is actually there. You could even have modern nicities like, store the entire GUI field state transition diagram statically at the beginning and as the user types ahead of the GUI's current state, show them a rendered view of where they are going, e.g., the key sequence I showed above could be "[Rebates] -> [Rebate Receipt] -> [Name Field] -> [Address 1 Field] -> [Address 2 Field] -> [Rebate Code Field] -> [Entering {'A8F2-132A-9283'}]", live updating as it goes so if the user can just bang that out even while the first screen hasn't loaded they can even be sensibly editing the text input they're going to put in.

Combined with something as simple as making sure everything always has a keyboard shortcut in a defined format, which is conspicuously displayed every time you change something, e.g., every time the user clicks the Rebate Code Field you can also display something like "Ctrl-F R" or something so they also learn how to drive the screen with a keyboard and you would, if nothing else, have something different than what's out there today.

Anyone who wants to, steal this idea. I'm not going to do anything with it. I don't see this as something you want in every app necessarily, but people who spend hours a day in the same app for whatever reason would be confused for a week, then I bet you couldn't pry it from them with a crowbar.

The fundamental difference is that mouse input requires unpredictable motor movements. We can't queue mouse clicks because the user doesn't even know where a button is going to appear. Even if they did, the actual physical mouse movement needed to get there is different every time. There is a necessary mouse -> screen -> eyes -> hand -> mouse feedback loop for every action.

Keyboard input can, when designed well, be fully predictable and require no feedback loop with the computer. The motor movement involves a fixed set of physical buttons, which we can fairly easily train to press very reliably, and that training is then universally applicable across all software.

That is why you can buffer keyboard input but usually not mouse input. With keyboard input, I know what the software is going to do and I know exactly what hand movement will get it to do the next thing, so I can perform that movement even before it's ready. With mouse input, it's not really possible.

Making UI interactions bufferable has appeal to me as well.

I had a similar thought but just for the more conventional 'keyboard shortcuts are not discoverable' problem.

Using the status bar or an overlay showing on-screen-keyboard shortcuts, the app could give visual feedback every time you click a button or slide a widget revealing what keyboard shortcut would have triggered that same action.

I click the "GO" button: OSK overlay or status bar shows [GO] (Ctrl+G)

I slide a widget to turn volume up: status bar shows [Volume 22->37] (Alt+V,22)

I love this idea. I also just checked out your blog (jerf.org) which also has some gems.

Demoscene music tracker interfaces come to mind.

https://news.ycombinator.com/item?id=49093434

Keyboard specific focus + navigation, shortcuts, poupable command entry/console (eg ~ in mpv, or quake :D) with commands that interact/integrate with the GUI side.

start with CUA standard.

Its only been around for 50 years, but few people ever consider it.

I'll try with an example first, and then will try to extrapolate to something more general.

I'm a longtime Emacs user and I detest tools like VSCode (VSCode in particular) because they aren't fully keyboard driven. "Fully" is the important part here. You can do a lot of things with keyboard in VSCode, but every now and then there isn't a way to do what you want without the mouse. And it screams "bad (no) design". For example, jumping between different panes in the editor is not always possible without a mouse, or the path to doing it is too long.

Before my attempt at generalization, I also want to address these:

> UIs like CLI or TUI suffer terrible discoverability

They typically do, but they don't have to. The reason for making a CLI and, to a lesser extent, TUI is the development speed. A full graphical interface will take longer to develop. So, whoever ends up making a CLI or a TUI tends not to invest time into accessibility either, because their goal was to save time on the UI in the first place.

Another aspect of this problem is the baseline knowledge. Users of graphical interfaces know how to discover functionality, they recognize familiar controls and patterns, they know that a cogwheel icon is likely to be a gateway to a program's settings, that if they want to make their changes persist, they need to look for something that looks like an open folder, or maybe a diskette icon...

Emacs has absolutely amazing discoverability. No program with graphical interface ever made doesn't hold a candle to what Emacs can do. But the user needs to have the baseline knowledge to... ironically, discover the discoverability. And I don't think there's a way to bootstrap from this problem. The minimal knowledge necessary to discover Emacs' features must come from reading the manual or asking a friend etc. Just to give you a few examples: Emacs can easily display all, without exception, keys and associated functionality available in the current editor's layout. It can also display them per individual component (if you, for example, knew that a key was defined, but was missing from the aforementioned list). You could also use the function name, if known, to find what key it binds to in the current layout. You can easily discover what any key (combination) does in the current layout without actually trying it (rather reading its documentation).

> can we have a keyboard-driven UI that is as intuitive as clicking with a mouse?

Absolutely! If we commit to a general set of rules that allow bootstrapping. This would require a degree of uniformity between such tools and, perhaps, some visual or even physical aids in the computer periphery to make this work. Computer interfaces, both physical and on-screen, evolved to support mouse-first UIs. For example, the navigation arrows on the keyboard are virtually never drawn on jklm etc. The keyboard is presented to its users as a tool solely designed for entering text, not a tool to engage UI controls. Compare this to gamers' keyboards that often emphasize wasd keys (because they are often used for moving your character in games) or even more extreme designs like Nostromo that are geared towards typical game interface.

* * *

Being keyboard-driven isn't about, as you said, having a shortcut assigned to an action. This is not how good keyboard-driven programs work. This is the way of thinking if your baseline experience is a GUI, which must be somehow retrofitted into the world with the keyboard as a primary driver.

Keyboard-driven, to me, means that the program has a systematic, hierarchical and consistent approach to deal with the keyboard input. Assigning keys to actions at random will quickly overload the user's ability to memorize such bindings. A keyboard-driven program is designed to have a system that the user can internalize and use to predict the right key for the desired action. Such a program should make no exceptions when binding actions to keys: any action should be accessible in a reasonable small number of keystrokes.

Reply on news.ycombinator.com