FUTO Swipe – A new swipe typing model
swipe.futo.tech
swipe.futo.tech
So for the longest time, I've wanted a new keyboard layout specifically designed for swiping. In the same way that Dvorak was optimized for ergonomically typing English words, I want a keyboard layout designed to minimize word overlap/ambiguity when swiping.
It doesn't even necessarily have to have 26 keys, e.g. maybe there could be one key overloaded for v/w/x/z (and you long-press it if you ever want to type a single letter). On the other hand, maybe there need to be separate keys for 'e' and 'ee', or a special key for "double the previous letter".
Because I love swiping, but all my problems with it come from the fact that the QWERTY layout is far from ideal for it. I am 100% willing to learn a new layout if anyone will develop an optimal one for English so that swiping has a 99.9% accuracy rate instead of what currently feels more like 90% or 95%.
https://github.com/futo-org/futo-keyboard-layouts/issues/163
I'm happy with the switch. Like any keyboard switch (I've gone from Qwerty to Dvorak and now a Colemak-dh derivative with about ten years on each) it takes some time to learn the layout. Overall I'm happy with it though and there are less frustrating misinterpretations and corrections needed.
This post was swiped on it with only two corrections and the second one was my fault as i misremembered a key location.
[1]: https://clearflowkeyboard.github.io/#section_activate [2]: https://youtu.be/rSfbvE9cEKE?si=NbJC93sTiOHqw4lX
I'm writing down a few impressions: - the layout is unusual, but I get the motivation. Distances are minimised and letters are arranged so that ambiguity is removed. - although I'm very slow, I haven't made a single mistake so far. Clearflow allows me to swipe much more accurately than stock gboard. - the square keyboard layout unfortunately means that half the letters are constantly hidden behind my thumb. As I'm unfamiliar with the layout, this means that before swiping a word, I have to look at the layout, memorise letter locations and plan the movement - since I write in multiple languages and Clearflow is available in only one of them, I would have to memorise a completely new layout for a language I write in only half the time.
For learning ClearFlow, I used the Games app available from the "Clearflow Games" section on their website: https://clearflowkeyboard.github.io/
I also have the issue of the thumb getting in the way so I spent a couple of days playing the games to get my layout memory up and then it became usable without frustration and I'm not looking back now although I occasionally still forget the odd letter location.
It's one of those things I've been wanting for a decade now, sounds like it's been around for about a year and a half.
Man, this is the kind of stuff I come to HN for.
EDIT: Ugh! I have an iPhone, and FUTO is Android-only, and ClearFlow is only available in the Android Gboard, not the iOS Gboard. :(
It could be interesting for applying it to different languages (or modified word corpus).
Maybe I'm mistaken though. Are there any physical clearflow keyboards? Are they any good, or does clearflow really only work well with swipe?
1. Switching to Dvorak on my computer AND 2. Switching to Colemak on my phone
I already had a feeling that my Dvorak experience on phone wasn't gonna help typing in a physical keyboard and indeed that was the case. But also, the physical keyboard experience with Colemak didn't really help with thumb typing/swiping on the phone. On a physical keyboard, I can only type QWERTY at around 30WPM and Colemak at around 130WPM. However, on the phone QWERTY is my strongest layout (even when I can only type at around 50WPM with it).
Then there's the design philosophy. Colemak aggressively keeps the most used letters on the homerow. Paste a random Wikipedia article into this page and you'll see (https://www.patrick-wied.at/projects/heatmap-keyboard/ alternatively, here's the heatmap for the introduction to today's featured article about Augustus: https://postimg.cc/9RT8P4N6). On a phone this means that a lot of the times you'll be swiping back and forth horizontally. Resulting in identical swipe paths for a lot of common words.
And if Clearflow was ported to a physical keyboard, most of your fingers won't be doing any work and the active fingers will be doing weird tap dance. The position of spacebar (undoubtedly the most used key on the keyboard) to the sides where you'd be using your pinky would make it an RSI machine.
My point is that if you're comfortable with QWERTY on the phone, that's not because you're comfortable with QWERTY on the desktop. That's because you got used to QWERTY on the phone.
90-95% is a very good estimate! That's about what we measure on our test set. I have good news for you, and we will have a blog post about it soon. Because of how our models are built, we are able to optimize for detection accuracy directly by constructing synthetic swipes on each layout for ~50k words, and then testing them through the model. We tested around 800,000 layouts this way.
The biggest issue with QWERTY is that there are far too many words that swipe colinear or obtuse angle letter trigrams. These are both hard to detect and frustrating for swipe users, because you can't clearly indicate the letters you're gesturing. Neural swipe models (at least ours) look for indicators in the gesture pattern that suggests a user was targeting a specific letter, rather than trying to match a gesture shape like algorithmic detection does.
The shape of the keyboard can significantly improve the way the gestures are formed so that there is better indication of letters. The model can still respond to dwell times because unlike shape matching it uses the temporal information. But dwell interrupts flow, and in my opinion should be minimized in swipe layouts.
The ContextLM model is a very small language model that is trained for a single language. It's used to improve the quality of predictions by eliminating nonsensical words given the preceding words in the sentence. It only requires text data for training.
You mean like the two E’s in “feel” or the two L’s in “fell”? I just tried and it handles them well. Are you aware of the circling technique? When you want to double up on a letter, you briefly circle it slightly. I believe some keyboards let you hover momentarily without circling.
Try it, swipe F-E-L, it should complete to “fell”, then do the same thing again but form a small, tight circle over the E, it should then complete to “feel”. Works for me every time.
So for feel, you start at F, go to E, loop once, then L. For fell, start at F, go to E, then loop on L. Very easy to pick up as a physical habit.
I just tested those two on futo, and it easily picked them both out.
I looked in the "Swipe Typing" options and there doesn't seem to be a config for it.
Is it possible that your keyboard’s particular dictionary knows the words you’re more likely to use and adjusts for it?
Edit - Also got ‘grill’. Notice how the -t in felt and -I- in grill are not near path to L.
A workaround is to use the Notes app and use the return key to make a new line after each try, rather than deleting. That should give you more consistent results.
Eg enter Bürger Dienste and have it autocorrect to Bürgerdienste. Or even Führung Kraft and turn it into Führungskraft (inserting an s).
Edit: apparently there's a modern successor? https://play.google.com/store/apps/details?id=inc.flide.vi8
* https://github.com/dessalines/thumb-key
It has a similar sort of 'It doesn't have to have 26 keys on something the size and shape of a mobile 'phone.' thinking as 8vim has, whilst raising a good 'You know 'phones worked fine with a 3 by 4 grid for 60 years, ne?' point, but adding a modern twist of 'We can swipe, in the 21st century.' to the old notion of multiple letters on a button.
There are still these people thinking outside of the typewriter-keyboard-on-a-'phone box. (-:
But it just can't touch swiping for speed. Frankly, the keyboard I miss most is the T9 predictive text from my old school pre smart phone era.
Nothing has come close to the same expressiveness and speed while being usable completely blind, only by feel.
I do feel like mobile keyboards have stagnated in a bad spot, though.
In fact (with Gboard) the suggestions don't change with cursor position. Surely, when I place the cursor you the right of a letter I'm planning on removing that letter, or adding a letter, but the suggestions don't change according to cursor placement.
There's also no apparent frecency - I had to correct cursor every time.
I swiped "frecency" as "decency" - corrections offered were "d doubt difference". The swipes for those are wildly different. d->e is NNW (350°), d->i, d->o is ENE (~70°).
It's really so basic. Surely they can do better than this.
I hope FUTO does start caring about language support, because for example their AI powered text prediction is only available for English. I'd happily train a model for them in my native language if they provided instructions on how to do so. And I'd help with swipe typing too.
http://www.fitaly.com/fitaly/fitaly.htm
However, there’s no Fitaly keyboard available for modern phone OSes.
http://networkimprov.net/alphatap/light.html
It was set up for just this.
c.f., the opensource research project Dasher
There are a few issues, like it randomly capitalizes words in the middle of sentences. Also, it doesn't seem to take context into account when suggesting words, so words that clearly wouldn't follow the last word will often show up.
It's not as good as gboard yet, but close enough that I'm going to stick with it.
Note that if you have a more powerful device, you can get larger models for voice and larger dictionaries from their site. They make a noticeable difference.
The only fundamental issue I have with it, they seem to be ideologically opposed to adding a GIF search, which I miss occasionally. https://github.com/futo-org/android-keyboard/issues/293#issu...
Nice to see the hour of swiping I did adding to their dataset actually helped. I'm using it now and it feels as good as the Google keyboard.
Edit: It is sending me a little that it keeps swiping "whats" instead of "what's" though, hopefully they fix that later.
- https://gitlab.futo.org/keyboard/swipe-library/-/blob/master...
- https://github.com/futo-org/android-keyboard/blob/master/LIC...
https://huggingface.co/futo-org/futo-swipe/blob/main/LICENSE...
Is it this part?
you may not remove or obscure any functionality in the software related to payment to the Licensor in any copy you distribute to others.
As an aside, Eron Wolf, the billionaire behind FUTO, has some rather... out of touch views[0] on the meaning of open source, and seems very committed to diluting the term to mean something closer source-available by removing the most of the rights granted (as defined by FSF, OSI, DFSG and others).
[0]: https://gitlab.futo.org/eron/public/-/wikis/Thoughts-on-Open... - please keep in mind that the RMS quote at the top is taken out of context; he is arguing for more freedom, not less
This is because 99.9% open source projects not targeted enterprise never ever see more than $100 of donations and being maintainer of such software is literally thankless job that will never pay you anything.
Blaming organizations for giving money or maintainers for taking money is worse you can do no matter who sponsor is: FSF, FUTO, Cloudflare, Microsoft, Facebook, Oracle, DARPA or MAGA INC.
Dont like FUTO or its owner? Make a better fund, give your money to SFC, FSF or whatever open source sponsor organization is acceptible to you.
There is so little money in end-user open source software and making pie even smaller or antoganizing thankless people maintaining it is awful. Everyone doing this is either dumb, malicious or both.
this is not an out of touch view. on the contrary. what is out of touch is to believe that Open Source is perfect as it is, and nothing should be changed about it.
the problem that FUTO is trying to address is the commercial exploitation by large companies (like AWS) at the expense of the original creators. a problem that several companies (redis, mongodb, elasticsearch, zerotier, terraform, vagrant, and many more...) in recent years had to face, causing them to change their license.
it's a problem that bruce perens is also trying to address: https://news.ycombinator.com/item?id=38783500 "What comes after open source? Bruce Perens is working on it"
btw: i recently asked bruce about this and he said that AI now changes everything and he wants to wait and see how that will affect Open Source.
it is not obvious that FUTO's approach is the right one. it is an attempt at addressing the problem, and i expect that it will take more such experiments to shake out what the best approach to this problem really is.
i wrote about this before:
It's just a commercial license with very mild terms.
The source code is fully available, none of the features are paywalled. They only prohibit you from taking their code and reselling it.
If you take a look at the Play Store, there are thousands of instances where open source projects are lazily renamed and sold for $5 or $10. It's the definition of scummy, pathetic, worthless behavior, and I'm glad the license prevents those kinds of leeches from succeeding.
I know this isn't the only case, but it's the majority of cases. So I have no problem with their license at all.
So no, the license doesn't matter.
It might be not a foolproof solution, but I think the license is better than nothing. Then you have a legal precedent that you can cite when you file a lawsuit against these rats.
Most of these people doing this probably aren't in the same country. But whatever. It's better than nothing.
They prohibit you from removing the constant nags about buying a licence.
1. Paying and clicking “I paid”
2. Not paying and still clicking “I paid”
So it’s an honour system right now.
That said, if they ever implement e.g. license keys or some other mean of actually checking that you’ve paid, seems that you would be able to remove it and recompile, you just can’t help others do that:
> Notwithstanding the above, you may not remove or obscure any functionality in the software related to payment to the Licensor in any copy you distribute to others.
(IANAL)
Or you know, also buy the license and click it honestly.
Clearly not a cuck license (https://lukesmith.xyz/articles/why-i-use-the-gpl-and-not-cuc...) so looks good to me.
Is that really true? My memory of the original iPhone's touch screen is that it was pretty much pixel-accurate.
The article mentions that the keyboard wasn't accurate enough: "But by early 2006, the iPhone keyboard still didn’t have the accuracy Apple needed to ship the phone." I don't think that means the screen wasn't accurate; all it means is that the original iPhone had a small screen, so the buttons on the keyboard were tiny, and hitting them precisely was difficult. That's why the hit boxes of more likely keys were enlarged.
The base reason is the size of the keyboard compared with the size of thumbs and the imprecision of thumb typing. Adjusting the hit boxes results in a better error rate. It isn’t because of the resolution of the screen or touch detection.
See https://www.grammarly.com/blog/engineering/deep-learning-swi... for more details - it's very similar to the architecture described by the FUTO folks.
One key difference is that the learned model does not decode in a context sensitive manner but does it a word at a time. The main reason is because we wanted to release this soon and wanted the user's personal dictionary (i.e. contact names, etc... to show up correctly when swiped). It would have been nice if we could have followed through with the context sensitive decoding as described by the FUTO folks. It would really help with accuracy when dealing with words like:
1. (food, good, hood) 2. (you, toy, rot) 3. (our, or, it) etc...
(Disclaimer: I am one of the authors of the Grammarly swipe system as described in the linked blog post).
> The library also supports recognizing two-finger simultaneous swipe input through the SwipeEngine::recognize_multi method.
Aieeee!
I don't think I could ever learn to use that. I am from the 1990s fast-T9 entry era, and my thumb dexterity is poor. (And I'm left-handed with a badly-damaged right hand.)
I think this is a text entry method solely or primarily for the 2-thumb console-gamepad generation. (I find gamepads really hard to use, too. Give me a joystick, or mouse+WASD.)
Integrated speak to text, good autocorrect typing, good autocorrect swiping.
This is such a massive deal. This is, as far as I can tell, the first useful free and open Swipe model. This paves the way for things like swipe typing on platforms other than iOS and Android, a major pain point to newcomer OSes.
It’s unfortunate that the Bing team at Microsoft has so much power. They destroy products for the glory of Bing, and some money. Perhaps it’s about the money. But I feel like Microsoft doesn’t have to make a lot of money on everything they do.
I wasn getting a constant panel from them regarding using the backup feature, that will just keep re appearing.
It was so bad I even moved to GBoard. Not the same, but I'm getting used to.
With the risk of sounding like a broken record - what are people using on iOS? I've been using `SwiftKey` for a while as it has dictionaries for languages that the native keyboard doesn't have. But I would love to switch to something else, as M$ has been shoving AI features on it that I am definitely not interested in.
So far in this thread I've seen Grammarly and Nintype. The former seems to suffer from the same things as SwiftKey, while the latter doesn't look it's maintained (last update 7y ago). I don't mind a paid app, as long as it doesn't invade my privacy.
I'd also be very interested in SwiftKey alternatives, since MS already almost killed it once.
I am interested in something that has support for multiple languages + swipe support.
https://www.apple.com/uk/ios/feature-availability/#quicktype...
Maybe this is supported only for specific language combinations.
Unfortunately some of those Swype nicities are patented [1], so other keyboards can't use them.
100% this. THE best text-entry method for touchscreens. Nuance enshittified it and then Microsoft killed it.
https://www.theregister.com/on-prem/2022/06/06/microsofts-sw...
Every year, I try the stock iOS keyboard, but then I always go back.
Boy did I hate when a "secure" input came up and I was forced to use the default keyboard.
Wrt the “occasional bug”, what happens is that sometimes (enough to make it bad) the keyboard doesn’t render at all!
But in general, there is no giant improvement, none of the major ones are even competent enough to figure out a grid of numbers is better than a row, but Apple is even worse - it doesn't even let you type a number on long hold, for some reason thinking that a letter ų you'll never type in your English life is a better alternative
To me, not being able to type in numbers easily is a permanent suffering, though ok, not the end of the world
And unfortunately don't think there is a single swipe keyboard that's properly customizable to fix those glaring issues...
my biggest issue is that i make up a lot of words as i type and the google dictionary for icelandic is .. well it can never be fully complete because of the way the language works, so dictionary words are always a mess.
What's the problem of Icelandic? Is it because you stick words together like in Danish or German?
Their local voice transcription is top notch and proper swipe gestures would be icing on the cake.
Now to fix that issue there tapping "Tuesd" sometimes suggests "Thursday"
The keyboard on the other hand I never really liked for some reason which I can't even explain clearly. So for the keyboard I've been using https://github.com/HeliBorg/HeliBoard for ever, but it does not have sviping.
I might try FUTO Swipe just to test it.
Apparently this new swipe function is tied to FUTO keyboard. I don't use the FUTO keyboard as there is no support for the language I use intensively. Nevertheless, this line of work deserves more recognition.
In particular, if you end up using the voice input mode of it and have trouble with accuracy, I would giving a try to the biggest model that it supports. It's slower (although really not bad at all on my Galaxy Fold), but it's so nice to have it actually be as accurate as it is.
Or, SwiftKey will allow you to change a typed word's case by pressing shift, but FUTO just does nothing in that case.
I really hope these get fixed, or I can just issue PRs myself for them.
1. super responsive
2. arrow keys!! why this is impossible is beyond me
3. easy access to text editor and undo redo via the long pressing of the enter button
4. easy resizing and im using a floating mode so it's all near my thumb and no awkward bends, just my phone being too heavy for my pinky because phones are massive these days
5. offline voice is comparable at a glance to the offline voice input app I was using with heliboard. that's with their biggest ugliest English model and my phone is from 2022 at the newest.
6. have I mentioned how performance is great on this app yet? pleasantly surprised.
y'all do you but I am keeping this FUTO keyboard.
It's notoriously not on f-droid. The way FUTO licenses and/or builds its thing is made deliberately incompatible with f-droid's main repository.
You can add the futo repository to the f-droid client, but when people talk about f-droid they really mean the main repository, not the extra hoops to add less trusted third parties.
Futo just has all the languages I need, and does a better job with EN swipe than Heliboard. 100% convert. Plus, learning about Clearflow is a game changer.
Gboard has obvious imperfections like data mining. Though on my phone I can just block its network access altogether which mitigates it a lot.
e.g.,
Broadside Surfside integration financially illiterate calibration fantastical proliferation haphazardly horrifically striations proliferation (all typed first or second try except antidisestablishmentarianism which admittedly I got carried away)
FUTO improved a lot (I had tried it a year earlier also) but SwiftKey's suggestions are still a lot better in my opinion. With SwiftKey I can just type roughly in the right spot without looking and the correct words will come out most of the time. FUTO still suggests a lot of nonsensical next words that just do not follow after the previous in English.
I hope it improves further so I can switch.
The voice models are great though, and they can be used as part of the keyboard or standalone.
SwiftKey got annoying with pushing Ai image Gen and sometimes pop-ups.
I've had this installed for a while but found the swiping too inaccurate. I'm trying it again now and I reckon it's as good as Gboard now. I've written this comment using it. Nice!
Edit: I've noticed that, like Gboard, it's still hard to type words like "fuck" or even "tits" etc. Are these words just missing from the dictionary? It's always felt like I'm not allowed to type them because I'm not a big boy. I don't type those particular words much, as it happens, but other missing words is annoying.
Amen. Glad to see that companies like that still exist.
Previously, if you type "fac" or "face" or "facep" or "facepa" or "palm" in the search box, you'd get the facepalm emoji among the results. Inexplicably, now the lying emoji, bacon, fondue are shown when you type "fac" but not the classic facepalm. It is only after you type "facepal" that the facepalm emoji appears in the results, which is a major... facepalm for the Gboard team.
As a parent, I need to _constantly_ send the facepalm emoji to my partner, so I noticed immediately when it recently changed. I think they're now trying to account for multiple languages, misspellings, a larger list of synonyms that allows partial matches, sentiment/meaning/synonyms, etc. In any case, I've always felt like emojis should be weighted by overall usage patterns (I have never used the fondue emoji and don't know anyone who does) and now they need a better search trie, where the facepalm emoji is in the top results of "fac".
Still going strong with Fleksy even though you can't even download it anymore.
One notable advantage of swiping is that you can be quite loose with each single swipe gesture whereas you have to hit the right key many times for each word. That swipe is also usually much quicker than finding multiple keys for most people.
You preferences may reflect youth, eyesight, finger size, co-ordination, phone case usage or other advantages most people don't have.
That said, it also depends on screen size. Back when Swype first became popular, Android screen sizes ranged from 3-5". That was another factor driving it's popularity back in the day.
Definitely not for everyone or in every situation though
I run GOS and have tried it, you spend more time deleting works than actually using it.
I was forced to install Google GBoard instead and revoke its network access. Gboard swipe typing works flawlessly, FUTO still an Alpha project.
I agree. I find that swipe typing on Gboard feels much better than FUTO's because I think it adapts to my typing style. I tried FUTO about six months ago, and it was a bit frustrating so I switched back to Gboard. Hopefully, it has improved since then, but I haven't checked.
Good news then because you're currently in a thread about how FUTO has improved!
It's not perfect, but it's practically on par with GBoard aside from a few blips here and there. It's enough for me to get rid of GBoard for good, and not consider reinstalling it.
I really liked using Fleksy which let you swipe up and down on the keyboard to change autocorrect results, including adding words to your dictionary. I'm still not sure if FUTO even has that option.
The donation prompt can permanently disabled with two taps, so I'm not sure why that'd be a reason to abandon it. It's a complete non-issue.
"You've been using FUTO keyboard for X days. If you find this app useful, please consider paying to support future development.
FUTO is dedicated to making good software that doesn't abuse you. This app will never serve ads or collect your personal data."
I don't mean to be rude here, but this is an extremely bland, mild message. There's nothing passive aggressive about it in any way. What do you find aggressive about this?
HeliBoard currently uses a proprietary library extracted from GBoard, which you need to add manually (since the app has no Internet connectivity out of principle), but they're asking for swipe data contributions, to be then distributed under CC-BY-SA 4.0, which'll be then utilized by their own open swipe library.
Check our more details here [3], [4] if you'd like to contribute.
[1] https://nlnet.nl/project/GestureTyping/
[2] https://github.com/HeliBorg/HeliBoard
[3] https://github.com/HeliBorg/HeliBoard/wiki/Tutorial:-How-to-...