Mouseless – fast mouse control with the keyboard | Hacker News Reader

Mouseless – fast mouse control with the keyboard

mouseless.click

271 pointslvturner120 comments

This is a really creative idea!

I wish more keyboards would integrate that little red Lenovo mouse nub.

Some thoughts:

- allow me to maximize the youtube embed on your site, I can't really see the video clearly enough to gauge the product so had to open it on youtube

- little green cursor in your vid is very hard to see

- took me a long time to understand that your cells were rectangles, so "2 characters of any cell" didn't make sense for a while, maybe you could emphasize it (if only for the video)

Agree about the rarity of trackpoints. One manufacture of those is TEX (Shinobi, Yoda), always wanted to try one out. Still waiting for someone to make a custom keyboard in an Alice (or similar) style with a trackpoint; that way you could have a trackpoint but also use standard MX keycap sets. The TEX ones would involve carving into the G, H, and B keys.

There used to be more brands with trackpoints but IMHO most were pretty bad, except for Lenovo.

That said, their newer ones also seem a bit more slippery / harder / less precise than older ones, in my (subjective) impression.

TEX Shinobi - I have many of these and they are excellent: https://tex.com.tw/products/shinobi?srsltid=AfmBOoqVuwqI9Ot2...

Looks very interesting, not sure I love the layout of the Home/End/Del but I do love how narrow this would be on your desk

Okay, wow. I was looking for something like this for a long time. I'm going to pick one up immediately.

Is there an ergonomic keyboard you recommend? I used to buy Msft Sculpt but want one that is compatible with macOS.

There is also the Lenovo Trackpoint Keyboard II if you want something smaller.

I am intrigued by this product, but the video is, frankly, terrible marketing.

People want to know two things: 1. Is it quick? 2. Is it easy.

Demonstrating the first is easy, but your product will live or die by whether you can demonstrate the 2nd, and its already working against you: First impression is an overwhelming matrix of letters. The video's demonstration ignores this, and explains "type the letters in the cell", and then the video moves on within a second, when he should slow down and explain.

My recommendations: Don't say cell, say box. Its less technical. Don't assume people realize that each box has a unique combination of letters, tell them. Say: "Each box has two letters, and no two boxes have the same two letters. Type those two letters and your cursor moves to the box."

trackpoint - its called a trackpoint

Other options for mouseless navigation on MacOS:

- [1] warpd - uses grid

- [2] Scoot - uses grid

- [3] Shortcat - uses accessibility ui

- [4] Superkey - uses text ocr

[1] https://github.com/rvaiya/warpd

[2] https://github.com/mjrusso/scoot

[3] https://shortcat.app

[4] https://superkey.app

Wasn’t there a builtin solution in macOS directly?

I would love to know if there is. macOS has, IME, terrible keyboard support. Even navigating Finder is a nightmare. The new 'tiling' is sort-of keyboard accessible - some apps work, some don't.

To my knowledge, "Full Keyboard Access" in the accessibility settings is about as close as you can get; however, it falls rather significantly short compared to 3rd party applications. Try to enabling the setting, and you will see what I mean.

https://support.apple.com/guide/mac-help/navigate-your-mac-u...

Highly recommend shortcat.

I've had shortcat installed for a year or more, but it has never "stuck" with me. What are some of your "Can't live without"s?

I’m not sure I understand the question. The short cuts are dynamic, I don’t know what any of them are.

Talon Speech recognition also includes mouseless options and eye tracking: https://talon.wiki/Quickstart/Hardware/tobii_5

Works on Mac, Windows, and Linux

Great list. Small correction: the latest version of Scoot (admittedly, I haven't updated it in quite some time) offers both grid and accessibility-based navigation modes.

Thank you so much for creating Scoot, by the way. I wanted to create my own ShortCat like app for macOS, and Scoot has been a great source of inspiration (on top of being useful, of course!).

Also [Homerow](https://www.homerow.app/) is really good

That website crashes my browser on iOS (Orion fwiw). I recall this being an issue the last time I checked it out 6-12 months ago.

Hard pressed to think of any other sites that I’ve seen this issue with, super curious about what is actually happening there.

A very good list. You should add https://github.com/nchudleigh/vimac to the list. It works similar to shortcat but I found Vimac to be more performant and reliable.

I've found mousemaster (windows) to be excellent and the dev incredibly nice and responsive

https://github.com/petoncle/mousemaster

This looks really cool but the Youtube video was very confusing. You were talking about a cell but I couldn't see WHERE the cell was. There needs to be a big arrow or obvious flashing circle or something. I still don't quite understand how this works.

Agree. Apparently the grid is not made up of one letter squares and you have to figure out the row and column, it's made up of two letter rectangles.

What it looks like when you select that rectangle to give you more precision, I could not figure out from the video.

Two or three static images would be most helpful.

It looks to me that each cell is two letters [ A K ], so if one wishes to go to that cell, then CMD + A + K would put the mouse in the center of that cell. Letting go of CMD would close the overlay and the mouse is just there in that space, while pressing Space before letting go of CMD would cause a mouse click event. If one wishes for greater mouse precision, after typing CMD + A + K and holding CMD there, a smaller grid of letter squares are displayed within the original cell; typing the matching character in the sub-cell moves the mouse to that even more specific point.

Or it might not be holding CMD, I could be wrong. Here's a better and shorter video explaining it: https://www.youtube.com/watch?v=A2rmvlOz5l0

Looks like the screen is divided into a 26x26 H-V grid represented by |[A-Z] [A-Z]|, and typing e.g. `[hotkey], d, e` will click (4, 5) from top left? The cursor seem to click the center of the grid. Presumably that happens to be precise enough on macOS.

It's also unclear to me how you handle movements at a scale smaller than an individual cell, or if it's possible at all.

You can press any key on the keyboard afterwards for further precision. I think.

Yes. Once you hit "JM" for example, you see a subgrid with more letters (these are standardized for all the character pairs, so you only have to remember them once). You can just press space or hit one of the additional letters for additional precision.

It's difficult to see in the video, but I've downloaded it and tried it, and on a real screen, it's very obvious that the grid is made up of cells that contain two letters, like [MK].

I'm not sure if I had a use for this, but it was extremely easy to use. I'm already looking where I want to click anyway, so when I hit the Command key, two letters appear where I'm looking, I'm typing them and space, and I've clicked the area.

In your video you mention not being able to use tab, enter etc for system dialogs on Mac, if you enable 'Use keyboard navigation to move focus between controls' in the Keyboard settings (under Shortcuts), you can cycle the focus with tab (and shift+tab), and select them with space. It's one of the first things I enable on a new install :-)

It's quite weird that this isn't the default imo.

If MacOS didn't have insane defaults, it wouldn't have any defaults at all.

Obviously, I exaggerate, but damn I hate getting a new MacBook for work and have to go disable mouse acceleration, disable autohiding scroll bars, invert the scroll wheel so it goes the RIGHT way, and who knows what else.

Showing proxy icons of course. That's a big one.

The characterization of mouse keys as taking "five...more like ten seconds to get to where you want to go" is incorrect with a decent implementation/practice.

I exclusively use mouse keys with my QMK keyboard [1] and I'm just as fast with it as with a mouse. I have four different cursor speeds that I toggle between as I move the mouse: I use the SUPA FAST LIGHTNING SPEED mode to move across the sceen and then reduce the speed as I zero in on my target. It's totally seamless and easy.

I also happen to use Tridactyl which uses a very similar mechanism as mouseless, but sometimes when I'm lazy I just use mouse keys. I don't think navigating by key anchors is always better: there's a greater cognitive load to reading off characters and typing them vs. using WASD to move your mouse around.

Alternatively, we could all simply relent and accept the fact that this problem was solved long ago in hardware and ship TrackPoints on all keyboards.

[1] https://docs.qmk.fm/features/mouse_keys

A trackpoint with more articulation (probably like a mini joystick), so I don't get pain after two days, would be awesome. Perhaps also with another button you can hold for 3x speed?

I tried a joystick on my keyboard, but for me it didn't really work! To be fair I also don't really like the track-point. I wasn't able to find a joystick "nub" that was comfortable to use, most joystick on the market are not meant for the kind of use that i would need on a keyboard and are really uncomfortable, maybe you could 3d print something more ergonomic but I'm not sure.

One thing I would love to try is a couple of really sensitive encoders for the two axis of a mouse. My split keyboard already has two encoders but they are to "stiff" to be used as mouse.

I also started using mouse key due to RSI and i was surprised how well it works, I use a single speed and I can still be extremely fast (not yet fast as a "real" mouse but very close).

One feature i would love to see is some sort of absolute pointer mode that work in a similar way of what op is doing as an alternative to do quick large movement (especially in multi-monitor scenario).

Interesting. Is your config.h on GH at all with this config to toggle between speeds?

It's not, but it's a straight-forward use of Constant mode, documented here: https://docs.qmk.fm/features/mouse_keys. You change speeds by pressing MS_ACL0, MS_ACL1, and MS_ACL2 (or none, i.e., the default speed).

I used a similar system when I had pretty bad RSI over a decade ago but it was a bit different - basically you would hit a keyboard shortcut and it would draw divide the screen into 4-shaded quadrants, you would type 1,2,3,4, and then it would zoom on that quadrant, and divide it up again into four quadrants. Rinse and repeat until the cursor position was at the position you wanted it.

At any point in time, you could hit escape if your mouse was at the precision level you wanted. It actually worked pretty well.

Binary space partitioning, kind of.

yep. it sounds like keynav [1], which I used for a couple of years and blogged about here [2]

[1]: https://github.com/jordansissel/keynav

[2]: https://felipecortez.net/blog/mouseless.html

I've been using Vimac [1], which puts the targets directly on UI components.

[1] https://github.com/nchudleigh/vimac

Huh this looks cool, like vimium but for the entire UI vs. just the browser.

For Linux users there is https://github.com/moverest/wl-kbptr/

Also: https://github.com/rvaiya/warpd/

Giving this a whirl as it's both MacOS and Linux compat. Thanks.

Update: It's picky about the Wayland compositor and I didn't notice that. Couldn't get it to work under Kwin (didn't try very hard but life's a bit too short for that).

I wish I had seen this a few months sooner! I have solved the problem with hardware via a Keyball: https://github.com/Yowkees/keyball/blob/main/keyball44/doc/r...

It looks great! How do you like it? I think I would prefer a small touchpad because it seems like it might put less of a strain on my thumb, but I am very curious about your experience.

(I'm not OP - I'm on the Charybdis.)

I prefer a trackball because I can momentum-flick my cursor from one side of my 4k screen (at 100% zoom) to another side monitor pretty easily... as long as I keep the ball bearings clean. I've been thinking about using a metal trackball for more momentum, but haven't gotten around to it yet.

I have a "sniper" key I use to decrease sensitivity for more precision cursoring... but rarely use it (maybe once a month?). I do not have thumb fatigue.

Thanks!

Did you see: https://naya.tech/

Thanks for the link, the keyboard looks amazing! Sadly, it's waaaay too pricey for me.

I’m loving it! It was quite the jump from a standard layout to split + ortholinear + column stagger but within a week I was comfortable using it in the office.

No strain from using the ball, it is (or can be) quite sensitive so you only need very slight movements and as another poster mentioned there are a variety of ways to adjust the sensitivity of the ball on the fly via layers for speed vs precision.

It’s entirely programmable, and easy to do so, so if something isn’t working or doesn’t feel right it’s you can adjust every key and ball function to behave as you’d like.

Was also very fun to build - but you can buy them prebuilt too

Glad to hear it and thanks for responding!

The keyball and trackball manuform variants[0] all seem great to me. My one issue with them is that the thumb clusters are not symmetrical.

I don't have money for a svalboard[1] so I opted for an interesting variant found on AliExpress[2]. Unfortunately the delivery person threw the package on the ground[3], so my integrated pointing device project is in limbo until this curious planck-like keyboard arrives[4].

[0] https://bastardkb.com/charybdis/

[1] https://svalboard.com/

[2] https://a.aliexpress.com/_m0Tn0vv

[3] https://ring.com/share/ff5c6ccf-d168-4975-a726-c58faff51204?...

[4] https://a.aliexpress.com/_mqFzwnl

Happy Charybdis user here. If you ask, I'm sure Quentin would be willing to build a Charybdis with a trackball on both sides - he was very accommodating of a change I requested (I wanted ball transfer units). He already supports having the trackball on either side.

Are there any eye tracked solutions? I definitely see the use case for something like this on large monitor setups.

A colleague of mine had an eye-tracking mouse attached to his monitor; this was something like 5 years ago. It seemed pretty effective. So "yes" I guess.

https://gaming.tobii.com/products/

I have tested out a tobii eye tracker in a microsoft store and it's just very nice. There's a circle that hovers over what you are looking at and some configurable ways to "click". Tobii is probably the consumer leader in this space.

Their older trackers are perfectly functional and already did everything with perfect accuracy so them releasing newer ones that are twice the price with next to no added functionality (hand tracking? At my desk? Why?) is annoying.

I'd absolutely try this out.

It's similar to the grid that's provided by voice control on MacOS: https://support.apple.com/guide/mac-help/use-voice-control-c...

There seems to be a bug at the moment with multi-monitor on different resolutions, but reading thru the dev's github issues it also seems it will be fixed soon.

Aside from that, I'm really enjoying this, taking a little bit to wrap my head around it, but I think it'll very quickly become second nature -- can already see how in many cases it's going to be much faster and less distracting than reaching for the mouse.

When is there voice control where I can configure the use of vocal sound effects (clicks, pops, whatever) as meta ?

it's configurable for both language and command phrases, so with a bit of effort (does apple already support any click-using natural language? if not, can you reverse engineer phoneme descriptions?) it ought to be possible?

EDIT: replied and edited via voice command; HN looks very amenable to this style of navigation (though I personally will go back to silent clicking as soon as the novelty wears off)

His letter method makes so much more sense than numbers though. Higher resolution and faster to type.

I'm not sure about the naming; similar software already exists for linux with the same name: https://github.com/jbensmann/xmouseless https://github.com/jbensmann/mouseless

When I was young there was a thing called "keyboard navigation". The user would press tab to move the keyboard cursor to the next control element on the screen, and spacebar to activate it, Enter to activate the default button (OK), or Esc to press "Cancel". Menus and most important controls had ALT+key shortcuts. Windows provided all elements with keyboard navigation by default and devs took great care for the controls to activate in the most logical order. Even browsers and websites had keyboard navigation.

Such wonderful times... Gone now. Everything is touchscreen-only.

Not sure what alternate universe you got stuck in, but I'm using a Macbook Pro with all those features, plus some. OS accessibility has gotten better, not worse. Web design has gone the opposite direction unfortunately, which is ironic because the web community is the one berating everyone over a11y constantly

> Not sure what alternate universe you got stuck in, but I'm using a Macbook Pro

"Everything else" universe.

This reminds me of Vimium, the Chrome extension, that allows you to follow links just by a couple of keystrokes.

Recently started using Viumium, much prefer it to using a mouse

Isn't it called Vimium, or is Viumium a new thing?

Probably just a typo

Vimac[1], is Vimium for the entire screen and also works for web pages like GMail that don't work on Vimium.

[1] https://github.com/nchudleigh/vimac

Vimium: https://chromewebstore.google.com/detail/vimium/dbepggeogbai...

Don't know if the developer is here, but feedback - a bunch of people complained that it was hard understanding that cells had two letters in it in the video. That also translates into the product. I have a hard time figuring if the cell is LO or OL. I think having the first and second letters either in different colors, or different font weight (eg first is bold, second is not) would really help.

I developed a similar solution for Windows over a decade ago which I've been using every day. It works but it needs to be made easy for people other than me to use (Readme, requirements file, etc.) If anyone volunteers to spend a few hours polishing it, email me and I'll open-source it.

Why hasn't this type of program been integrated into i3 windows manager? It seems the natural fit for a system that wants to keep your hands on the keyboard.

Does anyone know of something like warpd, but is in the ubuntu repository, instead of someone's github like warpd?

So about this:

https://github.com/croian/mouseless-issues/blob/main/mousele...

You at no point explain why your application needs to interact with SSL in any fashion.

What's up with that?

The combination of giving elevated permission to control the host computer, with network interaction? That's a bad combination. Why are these things combined in your application?

Edit: this comes across more accusatory than I meant it. I do think that the section should have a complete explanation of why the program needs an SSL bypass on certain networks.

A similar FOSS software is keynav https://github.com/jordansissel/keynav

it uses dichotomic splitting though

Keynav is good but warpd [1] offers all of keynav's features and more (and on Windows, an equivalent of warpd is mousemaster [2]). They not only implement dichotomic splitting but the hint grid too. And they're both FOSS software as well.

[1] https://github.com/rvaiya/warpd

[2] https://github.com/petoncle/mousemaster

Great app! I'm already considering buying it. It reminds me of another app called Wooshy (https://wooshy.app/), which takes a different approach by targeting the mouse through search words from the entire screen. While Wooshy is more intuitive, Mouseless is more versatile and can perform drag-and-drop very easily.

This is great, and it gives me an idea for an alternative:

A tool that uses a visual representation of your physical keyboard to move the mouse around the screen. So pressing Esc would move the mouse to the top left of the screen, and pressing H would move it closer to the center. Then repeatedly highlight smaller and smaller rectangles around the mouse cursor, repeating the process of tapping a letter, until you are satisfied with the cursor position.

I've wanted this for years

I don't get the last part where you repeatedly have smaller and smaller rectangles, can you expand on that? How are the rectangles defined? I pressed H, I'm at the center of the screen. Now what?

I assume something like this https://github.com/rvaiya/warpd/?tab=readme-ov-file#grid-mod...

Ok so just a warpd/keynav style grid mode then? I thought OP was describing a novel concept based on the idea that initially the whole keyboard would represent the entire screen.

Yeah, "Grid Mode" sounds similar. But instead of only 4 zones it would just roughly map the keyboard to dozens of invisible zones based on the position of the keys on the keyboard. So it would mostly be by feel. I have no idea if this would be better or worse than "Grid Mode" but it would be interesting to try it.

On mobile the embedded YouTube video is extremely tiny.

On desktop, the embedded video is also fairly small and annoyingly, they disallow full-screen. why?

Its as effective as Shortcat, which needs time to collect clickable elements and build overlays, and of course can be used in more than browser.

I changed keys to ASDFGHJKL;RVTBUM (18 characters) which is home row and index fingers and ok enough for clicking a close button a 27 inch screen. Dont recall any other tool offering this configuration.

This uses Python and Pydantic to validate configuration.

"moving your hand back and forth between the mouse and the keyboard gets very tiring, pull you out of the zone and just takes up precious time "

:DDD

keyboard is not for fine grained movement, period. it might be useful for handicapped people, but I don't see the relevance in everyday usage.

For MacOS users of Hammerspoon, I made a similar plugin based on devoting the screen into successively smaller 3x3 grids: https://github.com/msolomon/griddle

It’s free and fully customizable too.

I used to use vimac (https://github.com/nchudleigh/vimac) when I had macbook. I was pretty happy with it. I liked its interface. Its interface was inspired by Vimium extension on browser.

I had an idea for something like this, but for Linux instead and utilizing numbers

Woot makes keyboards that can detect the position of every key. It'd be neat to see that used to control a cursor, with variable speed depending on how pressed down the button is.

After watching the video, I think this is an exciting experiment to see if there exist better ways to control PCs without a mouse and just a keyboard. The standard of today being keyboard shortcuts.

Lenovo still sells $100 ThinkPad keyboards with TrackPoint, it will never be as fast, but no need to move hands from the keyboard

Years ago my Mac got confused and seemed to think my keyboard was a mouse. Pressing any key would move the cursor up a few points.

Ahead of its time, I guess.

These overlays should follow your keyboard physical layout to its advantage, so top-left should be starting with Q, not A

I've been hunting for a mouse panning solution for a multi-monitor setup on macos. This might be it!

Just downloaded and installed, I already love this idea :) nice work!

How clever!

Clever? Yes. New and novel? No [0]. Shame there's nothing like it for Windows? Absolutely.

[0]https://github.com/rvaiya/warpd/

On Windows, there https://github.com/petoncle/mousemaster which took inspiration from warpd.

How have I never run into this?!?!

Thank you!

Thank you for making this a one time purchase, not a subscription.

I feel like this could be implemented in autohotkey for Windows.

This looks like it would be really useful for multimodal LLMs

Is there anything like this for Windows?

There are a few:

https://github.com/GavinPen/AhkCoordGrid

https://github.com/EsportToys/TPMouse

https://github.com/lesderid/keynavish

None of those are quite like this, which is a shame. Unfortunately, I don't have time to get nerd sniped right now.

mousemaster is a warpd-like, opensource, for Windows: https://github.com/petoncle/mousemaster

interesting idea

Reply on news.ycombinator.com