Fleksy - Awesome new predictive text input method | Hacker News Reader

Fleksy - Awesome new predictive text input method

fleksy.com

150 pointsnreece101 comments

I had this on my PalmPilot in 1998 but from a different company. It was written by a friend who worked at a company with all kinds of patents for it, but they never released the Palm version publicly. Only a few dozen people had it in the world. It was amazing and it changed the utility of the PalmPilot significantly.

You had to put a sticker with the qwerty layout over the silkscreen area and then you just tapped away madly with the stylus. It was so good.

This is very, very close. Most of the mistakes it makes are due to the fact that it seemingly doesn't look at your address book for completion options, so when you try to type in people's names you know it picks a different word and doesn't even have proper names as an option. Fails on stuff like my name (Pinstein) or well-known things like "MacBook".

But it's still awesome! I've been waiting 10 years to have this on my handheld again!

I’ve been using Fleksy for about five days now. As a totally blind iPhone user this app has some promise and some major flaws. The first issue is the fact that the app takes at least 30 seconds to load on my iPhone 4 if you haven’t used it in a while. I’ll deal with the crappy typing experience using the iPhone onscreen keyboard instead of waiting 30 seconds if I’m just sending a quick text. The second issue is the inefficiency of entering custom words. If a word isn’t in Fleksy’s dictionary it’s much faster to type the word using the normal keyboard instead of with Fleksy. This makes writing technical emails or messages with lots of nonstandard words quite difficult. Since you can add words to a custom dictionary in Fleksy though I would expect this to get better with time.

Actually, touch typing on Fleksy is faster than on the standard keyboard. We reduced the lag time required to input letters from .5 seconds to .35. Plus, you only need to input those terms once.

I used incorrect terminology. With the on screen keyboard I am able to drag a finger around until I hear the letter I want announced, then with another finger tap the top half of the screen to enter that letter. It is much more efficient then touch typing with either Fleksy or touch typing with the standard keyboard.

The thing he mentioned was startup time, though, not the actual input lag.

Was responding to the second issue: "The second issue is the inefficiency of entering custom words". On that front Fleksy is the same or faster than the standard iOS keyboard.

I think he's suggesting that for "custom words" flesky autocorrects to the wrong words, whereas the standard keyboard doesn't - thus leading to longer input times under flesky ... at least until it learns those words.

The standard keyboard also autocorrects non-standard words.

Can anyone explain how this works? Seems like total black magic to me... the dude is typing random gibberish and it still figures out the word? Then he's just tapping randomly on the screen, gibberish comes out, then turns into the word he wanted?

Don't mean this to sound unintelligent, the video was just crazy. What kind of key proximity algorithms are behind this?

Patent pending kind? I doubt they are sharing it quite yet but the one beauty of patents is we'll get the entire answer once the patent isn't pending anymore. If I had to imagine though, the algorithm is just accounting for the most likely word to follow the previous wording, that could have been formed by hitting keys that were next to the ones you really hit wrong. I'd like to see how it handles my wanton throwing of swear words in messages to friends. And, the use of l33t speak and text talk.

Example, at 0:45 he swipes to the right and kryuwt turns into letter.

k -> l (k is next to l)

r -> e (r is next to e)

y -> t (y is next to t)

u -> t (u is next to y, which is next to t)

w -> e (w is next to e)

t -> r (t is next to r)

The options are letter, proper, merger and keeper (plus something else further) Given the previous three words "you miss ever" the algorithm uses some prior dictionaries and knows that when talking and saying "you miss every" we get probabilities of of likelihood for each probably word. Letter ends up being 35% as opposed to the 7 percent chance you see any of those other words. ( that last bit of percentage math is an estimation given for the sake of example, in a true text prediction dictionary the percentages are usually small, given the incredibly large amount of words that can and do follow "you miss every" (chance, shot, etc are probably more likely, but not near those letters we kept missing earlier.)

It would seem they've made improvements on that algorithms. I wonder how novel it really is though.

This is pretty much how the stock Android keyboard works. It's been able to spot missed keys since at least Ice Cream Sandwich - e.g. I just typed "kettrt" and it's suggesting "letter" as a correction. Since Jelly Bean it's also had some bigram support too (guesses the next word based on the previous without you typing anything).

Aside from the "typing without a keyboard" part, it's not obvious from the video how Fleksy differs from this, but the focus on blind users probably makes a difference.

But does Android use context clues? iOS can't quite handle the two-letter distance, but it accurately corrects "kryywt" (kryuwt is too far away) without any context.

ah, i forgot about bigrams - that's a really great idea. I wonder if it's packing a huge machine learning library though... wouldn't that take a ton of space? I saw in another comment that it takes a while to start up - maybe that's why

>that could have been formed by hitting keys that were next to the ones you really hit wrong.

Google does this already and I would assume they aren't the only ones.

I once accidentally typed "cinnabd oatterb" into Google, yet it knew I wanted "Command Pattern". Still works today.

The Levenshtein distance, the standard metric for determining how different words are from each other, can easily be modified to change the penalty for different letter replacements. For example, you can say that the penalty for substituting an 'a' for 's' is 1, and 'a' for 'p' is 10 (or whatever).

Boom, there we go - that's the term I was looking for. Awesome!

It might be an application of forward error correction, using the redundancy inherent in written English. Instead of interpreting pressing the 'F' key as the letter 'F' with 100% probability, one could interpret pressing 'F' as meaning a 60% probability of 'F' and 5% probability for each of 'E', 'D', 'C', 'V', 'B', 'G', 'T', 'R' (the surrounding letters on a QWERTY keyboard). Concatenate successive key pressed to form codewords, then use FEC to find the dictionary word which has the maximum likelihood.

Getting fancy, one could use the results to train the system, fine tuning the probabilities as it goes. In this way, it doesn't matter where you type (or even the spacial relationship of the "keys"), as long as the location you use for each letter averages out to be reasonably consistent. In time the system will train itself to so English words come out, meaning the key mappings are probably correct.

For example, one could start typing "QWERTY" style, change to typing "Dvorak" and the system would put out gibberish for a while, then start putting out English again as the probabilities converged.

>we'll get the entire answer once the patent isn't pending anymore //

As an aside A-publication is substantially prior to the grant of the patent, exceptional conditions aside.

This works in a similar way to BlindType (acquired by Google) from a couple years back: http://www.youtube.com/watch?v=M9b8NlMd79w

edit: it's actually the same! http://phandroid.com/2012/08/16/makers-of-blindtype-bring-us...

I prefer to type with the 8pen.com

This is so thinking-outside-the-box! I'm amazed with that technology. Thanks for sharing it here. If this takes of (it probably should, but I don't know if it will), imagine people in a few years being able to write and read directly with the kind of the symbols that appear at 4:45 in the "Idea"[1] video!

[1] http://www.8pen.com/watch?v=1

I'm willing to bet that in a couple years all the kids will be writing with the 8pen...

Just a guess but if I were to write it, I would create a table of points that map to a qwerty keyboard then map to a word. Then I would prerender as many words as I can to seed this table. The algorithm would be about probability of shape entered. For example, if I meant to write the word 'hacker' but instead I wrote 'jsvlrt' the shape I've made would map to a similar shape in my database for 'hacker'. Add some creative elements for managing tolerances on shapes entered and you would probably have an impressively close product to this.

As a side note: awesome app.

My guess is that they use a probabilistic model that, for every tap, incorporates the probability distributions P('a'| X), ..., P('z' | X), X=(x,y) probably a normal distribution. Then assuming they have data on how likely a specific word is, you'd estimate the posterior probability distribution over all the possible words with that many characters. Pick the top most probable words as auto correct options.

I'm sure there's more to it than that, but I assume that's the basic idea.

I think there's a bit of a show here. I'm guessing he went through this demo quite a few times before they filmed this. The prediction algorithm probably starts "learning" which words follow other words based on use (on top of the proximity algorithms). So if you go through this a few times its getting "smarter" and more predictive.

I'm gonna guess that it does a couple things, but the main one is the shape you create with each touch, since you can type without typing on the keyboard itself.

I've realized that for my mobile phone the typing speed will just never be great regardless of the input method. Tools like this (and SwiftKey) improve that a bit but not to a life-changing degree.

For the desktop this might be quite amazing however since, unless you're a perfect touch typist, you will always make a small number of mistakes.

These small mistakes can be caught fairly easily (especially given the much larger training set you'll gather on a desktop) and since you're looking at the screen anyhow it's fairly easy to detect damn-you-autocorrect mistakes before they slip through.

TL;DR; Don't use this to make inherently slow data entry a bit faster; use it to make fast data entry even faster.

Apple in Mac OS X has brought the same error correcting features that exist in iOS for typing to every single text box that is rendered and has a cursor placed in it. This has allowed me to type faster and even if I make mistakes the OS will correct them for me most of the time. It still needs a little bit more time and tuning, but so far it has proven to be absolutely fantastic when typing.

See Preferences -> Language and Text -> Correct Spelling Automatically.

> for my mobile phone the typing speed will just never be great regardless of the input method.

Actually, way back when PalmOS and PocketPC were all the rage, there was a company that released something called the Fitaly keyboard. I don't know what you qualify as "great" typing speed, but with this keyboard they advertise 50wpm with a stylus, I was able to achieve 60wpm after some practice.

The point is, it is definitely possible to achieve fast input speeds on mobile devices.

Admittedly it did take some practice and probably isn't great for the average consumer.

http://www.fitaly.com/product/fitalyppc.htm

I agree that a stylus changes everything; I should have been more specific.

I never got up to 60wpm with Fitaly but I do remember not really feeling hampered by my typing speed.

In general you don't get a stylus with your phone anymore these days though...

Agreed.

Also it seems like it's finally coming full circle around back to styli again. It seems like they're now being advertised as a new feature, which I think is kind of funny. Granted, they aren't using resistive screens anymore, so it is sort of new.

I'm waiting for a good stylus-enabled device before I purchase anything new. The fact that you can't draw on most of these tablets is silly.

Yeah, I wrote about this a while back, the stylus will always be useful for pretty fundamental reasons, though there will be phases in which it's not easy to implement (we're coming out of one right now): http://techcrunch.com/2011/11/11/in-defense-of-the-stylus/

The Samsung Galaxy Note is well-liked by the subset that does prefer a stylus.

Swype boasts rates of 50+ WPM on a smartphone: http://en.wikipedia.org/wiki/Swype

I don't see this improving my keyboard speed on a desktop since I notice an error after typing two other words (I type really really fast).

These systems usually use predictions (see Markov chains) based on your previous input to automatically catch and correct those small typos and prevent you from needing to skip two words back to fix the typo.

In the case of a false positive (correction where none was needed) you can just delete the entire word and type again.

As this is usually done using Markov chains which learn from your previous input this could even be used for programming.

in the case of a false positive you have to go back two words and correct it (in the idea the word was corrected automaticaly). I'm afraid this happens more than I make mistakes typing.

Hmmm, this is probably the best argument for QWERTY I've seen. It's probably based on relative key taps identifying the keys, so having a keyboard layout that scatters everything makes it easier than a keyboard layout where aoeui are all right next to each other, which are very frequently interchangable between two legal words. Quite a few of the Dvorak consonant clusters are also often interchangable in the same way (crl being the one I have the most, err, "fun" with).

This… this looks and acts almost EXACTLY like the stock Android keyboard. The swipes to accept a correction and to delete a word are new, and the graphic design of the keyboard is more minimalist - but the core functionality of "spellchecks based on words you could spell with keys near the ones you actually hit" and "shows autocorrect candidates in a horizontal row above the keys" is absolutely identical.

It's a shame iOS is so monolithic, if you could wedge in a new keyboard like you can on Android, I would totally be willing to shell out some bucks for this because I really loathe the iOS autocorrect experience compared to the Android.

I don't think the similarity to the stock Android keyboard is coincidental. The authors of this previously made Blindtype which was purchased by Google.

Hope they hired a good lawyer to review the IP transfer agreements they signed in the previous sale.

Windows phone pretty much does exactly this already. It's horrid.

I genuinely think that a better solution is to buy a phone with T9 if you have input problems or like to enter stuff when not looking at it. You simply NEED a tactile interface to do that effectively.

I carry a Nokia 3330 and Lumia 710 around. The 3330 is still a thousand times easier to enter text messages into. I can use it with my eyes shut and make absolutely no mistakes. The 710 however, is painful to enter text into.

T9? Seriously?

I think you're an outlier, my friend.

I'm definitely not. I think your perception is warped by your social circles which is probably smart-phone oriented.

The overwhelming majority of the handsets getting shifted out are feature phones and below and they are mainly shipping with T9. Have a Google round for the stats - they are not hard to find.

These stats also only cover traceable sales. There are a lot of second hand sales.

T9 with some sort of tactile keyboard and some practice results in considerably less errors and corrections being made compared to a touch screen with a standard QWERTY keyboard.

Yes, the world is a big and poor place. But in the United States where (like other developed nations of course) people have the financial means to choose, the majority of phones sold are smartphones. And T9 is generally looked at like rotary dial.

Rubbish. Going back to September 2011 as I can't find any newer stats:

http://blog.nielsen.com/nielsenwire/online_mobile/40-percent...

http://money.cnn.com/2012/05/16/technology/smartphones/index...

By your own stats it was 40% over a year ago, and you don't think they've creeped past 50% yet?

Nonsense. For stats, look at that Pew poll linked in the CNN article alongside this comment. For anecdotal evidence, look at the free/sub-$50 phones offered by all major carriers with a new contract. They are almost all smartphones. It's cheaper up front to get a smartphone now than the alternative.

Plenty of feature phones have qwerty keyboards, including the one I used up until a few months ago. They're not all T9. Anecdotally, many of my friends have feature phones with qwerty keyboards...

haha! This has got to be trolling. It obviously doesn't make sense to carry both a 3330 and a 710, so I am guessing this is fabricated to make some English textile artisan's point.

Burn them looms!

Actually no. The 710 is my company provided WORK phone and the 3330 is MY phone. You should keep work and play separate.

My loom is fine :)

Maybe I missed it, but I don't think I saw any demonstration of correcting words that were typed with one fewer or one more letter than the intended word. The absence of tactile feedback when typing makes this a fairly common error -- anyone know if their algorithm is capable of handling it?

My iPad app needs very few keystrokes to spell most words - only the first letter has to be correct. You can see how few keystrokes need to be pressed to type full sentences in this demo: http://ktype.net/demo (app is now free: http://itunes.apple.com/us/app/ktype-free/id550288060)

I wrote my own algorithm that parses through a dictionary of one-million words and phrases in real-time (even one iPad 1) and suggests the most probable word based on the letters you've typed and in context of the last 2-3 words you typed. This means, typing 'P' after 'HARRY' will suggest 'POTTER' instead of 'PUT', 'PICK' or other common words that start with 'P'. I haven't made my algorithm public yet but when I have some time, I intend to, especially if there is a demand for it.

But you still have to type! I'll take Swype every time.

While I do love my Swype (especially as, similarly to the claims made by this keyboard, I am getting close to being able to "touch type" on it), I really, really like how this keyboard looks. Just letters, no fake key edges or anything like that, I think it's a huge improvement in the usage of the space available.

There are several keyboards available for Android that just plain suck on this front, with more space taken up by their early 00s pseudo-3d button styling than by the letters. I love what Fleksy is doing here and I hope more people follow in their footsteps.

I can't get Swype to stop autocorrecting normal words to gibberish, no matter how many times I remove the gibberish word from the dictionary. For example, every time I type "do" it becomes "siop" and nothing I can do impacts this.

The latest version has a much improved dictionary management, and I've found this issue has basically disappeared now, whereas it was very common before. Whether this is solely due to the new dictionary management, or if there have been other less visible changes under the hood, I don't know.

Sounds like you should delete local data and reinstall.

I use SwiftKey for Android for better prediction, I also use several languages and it predicts my words well. I wish there where a combination of Swype and SwiftKey however, as I agree that Swyping is a very fast way of inputting data.

Yep. I easily Swype without looking. There are occasional mistakes, but it requires very little concentration. Anyway, I think there's too much magic in Fleksy and requires a lot of faith/buy-in on the part of the user to trust it.

https://play.google.com/store/apps/details?id=com.exideas.me... this is better

That looks fascinating, thank you. It also looks like it will take a few days to get the hang of, but it may well be worthwhile!

Fleksy team guy here.

Thanks for the feedback guys, feel free to send anything else you might have for us at feedback@fleksy.com We want to make Fleksy as good as it can possibly be.

@fleksy on twitter.

Have you considered Android? You could actually integrate your app as a replacement for a real keyboard, rather than the second rate experience that is possible on IOS.

Your product deserves better.

Software ("app") pretty much depends on dictionary. What about words that aren't in the dictionary?

There is a solid way to add and remove custom words to the dictionary. You can also import a friend's(or a community's) dictionary.

Err, SwiftKey anyone? It's been around since 2008 and even has an xkcd comic featuring it: http://xkcd.com/1068/

I love SwiftKey but don't think anything equivalent was previously available for iPhone. My wife missed Swype terribly when she switched to iOS.

Haven't used SwiftKey, but from what is described in the comic, it seems that builds a Markov chain from your text.

I downloaded this and thought it was a little better, but not revolutionary. Then I decided to try copying a paragraph of text without looking at my keyboard. That's when I said "Wow!"

The combination of very smart prediction, speaking the words so I knew the couple of times I missed, and one-stroke delete made typing on glass, dare I say it, magical.

Ok, maybe I am extremely daft but how do you turn off the voice reading out each word you type? I turned my device on vibrate/silent mode - no dice, then I tried turning the system volume down to nothing - no dice. What am I missing here?

There is no way to turn off the voice. This app is primarily designed for visually impaired people who need that feedback.

The option to turn voice feedback off will be coming in the next update.

The main problem I have with input auto-completion is that the majority of times I'm using my phone to type, I'm using non-dictionary-but-meaningful-words (e.g. street names, foreign language terms) and the autocompletion usually just gets in the way. I've yet to see a system that addresses these kinds of usability issues.

Getting the exact text as you typed it is just an upward flick away with Fleksy.

Initial dictionary not trained with profanity. I hate how much time I have to spend stopping things from being autocorrected to "duck." Fleksy didn't improve on the situation.

This kind of technology has been around forever, but my guess is that the quality increase here is due to one or both of these two factors:

1) Computing a two-dimensional vector for each word typed, and calculating MSE against the ideal vectors for dictionary words, instead of just comparing the distance of the keys you hit to the locations of keys for dictionary words.

2) Using LSM (latent semantic mapping) to discover context and increase accuracy of #1.

Certainly looks faster than typing on a standard touch keypad, but this makes me really not look forward to my next phone. For the last couple of years I've had a Blackberry and I've become used to being able to send out paragraph-long text very quickly and efficiently. But of course a Blackberry Curve leaves a lot to be desired in terms of the functionality of a modern smartphone. So I want a better phone, but I'm just regretful of the fact that it's going to end up being some kind of iOS or Andriod fully-touch, because I know my efficiency will go straight down.

Well, I'm afraid I wouldn't like it. It seems it continues the only thing that is annoying me in iOS input method - not writing what I actualy type. Sorry, too often I use words it doesn't know (maybe in english it's better, but in polish/russian/ukrainian it's quite a big problem). It would mean disaster when using this app, isn't it? Does it support other languages at all? How about cyryllic script? Does it allow to switch between languages easly? Does it replace standard input method everywhere in iOS?

Interesting approach, and it makes a lot of sense. It would be interesting to see the algorithm behind it.

That said... every time I try out a new virtual keyboard, even for a few days, the more I yearn for a physical keyboard on a phone I actually want. The current options in the market for phone's with physical keyboards isn't great.

My passwords all contain plenty of special characters, and it requires many more keystrokes to enter in my password on a virtual keyboard vs a physical one. At least give me a shift key.

I was skeptical but just tried it out and it's amazing. I even tried to type a sentence blindly just by remembering the approx. position of the keys and it worked perfectly

How would adaptive keyboard layouts fair? I wrote a simple prototype that tries to minimize (mouse) movement for highly functional people who cannot speak and are fully paralyzed:

http://davidjarvis.ca/keyboard/

The centre key is the next most likely letter that will be pressed, based on digraph lexical analysis.

Pretty cool. I've been thinking of something along these lines for a while now. The fact is, the relative position of each touch very likely determines the word you're trying to type, regardless of what keys were actually hit. It's surprising it took someone this long to release prediction at this level. Looking forward to trying it out.

I tried this out just now. While the tech is very impressive and I could see why this is a great keyboard for the blind, I've honed my iOS tapping to a point where the stock keyboard is a bit faster. What slows me down is the swiping for space. Hitting the spacebar is much quicker in my experience. Still, very cool app.

To me this just seems like lipstick on a pig. I can type really fast on my Motorola Pro+ (it's like an e63 running android!) and not relying on autocorrect means I can do things like type unix commands and control vim.

For the blind - what is the advantage of a phone with a giant battery sucking screen on it in the first place?

SwiftKey already does this, and more. You can miss all the letters in a word, and type multiple words without pressing the spacebar and it can still accurately detect what you intended to type. Nothing revolutionary here.

I assume we can now look forward to DamnYouFlesky.com to complement DamnYouAutoCorrect.com. I hope this works a little better than the iOS auto correct that I have had to shut off, but my expectations are not high.

Assuming this achieves some mainstream success, Apple offers "the bullet or the bribe" (get bought out or we just rebuild your product in house and destroy your business) in 3... 2...

too cynical?

Getting bought out by Apple is basically their only option for mainstream success on iOS because of the way iOS's keyboards are locked down. A keyboard like this loses a lot of its usefulness if every app has to implement it via SDK separately, because it is far better to have a consistent keyboard system across all apps.

So I'm cynical too but from a different angle -- Apple has already fixed the game when it comes to alternate keyboards (unlike Android where you can install 3rd party keyboards that work system-wide).

I remember this product back when it was called BlindType. If you combined no look input with swype, I think would be a pretty robust product.

BlindType was purchased by Google and as far as I can tell implemented into the stock keyboard as you can touch type without looking and it will successfully correct the words.

What's the business model on this?

In order to get the actual text out of the app into an email, sms, or onto the clipboard you need to buy a $4.99 in app purchase.

They sell software - it's £6.99 for the full version.

Being acquired by Apple, Google or Microsoft.

selling?

i don't get it. how is this different from Swype? or is Swype not available on iOS?

It's different in that Swype uses one continuous motion instead of discrete taps. (There are undoubtedly other differences as well.) And no, Swype isn't available on iOS. (N.B. I use Swype on Android and love it.)

This is more magic than Googling for s,sxpm and still getting what I want.

Alright guys, get ready to bought out by Apple.

Awesome. Again. Another keyboard implemented for an OS where you can't even use the damn thing.

Reply on news.ycombinator.com