I am also quite frustrated with the current state of full text search in the javascript world. All libs I've tried miss the most basic examples and their community seems to ignore it. Will give yours a try but it already looks much better from the comparison page.
Edit: Nope, your lib doesn't seem to handle substitution well (THE most common type of typo), so yep, we are back to square one ...
the intro does caveat that it would make for a poor spellcheck :)
FlexSearch actually does pretty well and can work for you, though can get quite memory hungry depending on your tokenization settings. try other libs in my compare demo, too. there are a lot of options!
i'm an OSS dev; how thin do you think my skin is? :D
But thats a good general sentiment yeah
I settled on MiniSearch. [0] It is fast & small enough and fairly feature complete.
Afterwards I made a few contributions to improve performance and implement a better scoring algorithm. So I'm probably a bit biased now. Take my recommendation with a grain of salt.
Personally I think that OP's library does not perform searches, fuzzy or otherwise. It's much more similar to 'grep'. Try searching for "mario adventures". It won't actually find the most obvious results, because the order of the keywords in the search string must match the order of the keywords in the indexed text.
[0]: https://github.com/lucaong/minisearch
[1]: https://leeoniya.github.io/uFuzzy/demos/compare.html?libs=uF...
i'm really confused why people dont bother actually reading anything i spent so much time distilling (both in the readme and the short instructions in this submission itself), which explicitly spell out how to adjust the necessary options precisely for what you're asking.
simply toggling outOfOrder returns perfectly good results:
https://leeoniya.github.io/uFuzzy/demos/compare.html?libs=uF...
Sorry for missing that feature. I stand by my overall point though. There is more to search than finding matches. Handling real world typing errors is one (try searching for "mario avdentures" or "mario adventutes").
Ordering results by relevance is not trivial either. And it does not appear this library intends to tackle that problem completely. Relevance scoring should take into account the term frequency and the document/field length, as well as average length.
I don't mean to discount your project though. There are many distinct use cases related to searching and filtering, and power to you for solving it in a way that works for you (and I'm sure for others as well). I just wanted to share my experience in exploring the space of full text search libraries that handle long form text and typos gracefully.
s/everything/anything
this is by no means meant to replace fulltext search, with term omission tolerance, typos, stemming, etc.
fwiw, i wrote this to replace a frontend search strategy that was originally based on MiniSearch and [apparently] gave underwhelming results and/or performance.
since you seem to be familiar with MiniSearch, perhaps you can help improve the result ordering for this:
https://leeoniya.github.io/uFuzzy/demos/compare.html?libs=uF...
this is my fundamental issue with fulltext searches. even if the result order could be made sane, how do you find the "junk cutoff" (there clearly is one). at least it does seem to place the best results on top, though ordered haphazardly, which is also true of FlexSearch.
boiling things down to a single relevance score, and making the user tweak various boosting knobs to nudge things mostly [but not perfectly] into place seems like a dogma that most of these engines suffer from.
In the case of MiniSearch you can change this default and configure it to only return hits if all keywords match [0].
Searching for "super ma" seems like an autocomplete query, for which most users have subtly different expectations than regular search. MiniSearch has a separate method for that, essentially baking in different default settings. [1]
[0] https://lucaong.github.io/minisearch/classes/_minisearch_.mi...
[1] https://lucaong.github.io/minisearch/classes/_minisearch_.mi...
Edit:
> s/everything/anything
No need for the sneer.
e.g.
/^cat$/ will match cat
/^ca[tr]$/ will match cat and car
/^ca\w$/ will match any 3 letter word with 'ca' prefix. (any substitution for third letter)
/^\w\w\w$/ will allow alterations for any letter, making it useless.
you can make a more complex regex that allows a single substitution in any char position
/^(?:ca\w|c\wt|\wat)$/
but this gets out of hand very quickly, especially with possible insertions, deletions, transpositions, etc.
Anyway, not being a hater (as that other commenter suggested), if you could add support for this (even if it's limited, one or two typos at most) you will blow all other libs out of the way since what you have now is already quite good :D
This would help a lot I think.
When I'm on mobile, the most common error is "keyboard offset error", where I hit a "key" next to the one I intended. So it's not completely arbitrary, and it only happens once or twice in 99% of the cases.
On a physical keyboard this also happens, but the more likely error is synchronization error, where I hit a left-hand key before a right-hand key or vice versa, the classic teh vs the. Again usually only a single such error per word and not arbitrary.
Finally there's also the common case of simply missing a letter. Again, limited and not arbitrary.
So at least for my sake, anything that can handle the above errors would go a long way.
of course if you have specific known/common mistakes, it could be useful to only consider those. for example, spelling errors are less common at the start of words. and keyboard letter proximity limits which mistakes are likely.
At least on mobile, I find I make just as many first letter mistakes by hitting the wrong "key" on the on-screen keyboard, as I do in any other position of the word. Very annoyingly the predictive text engine assumes like you mention and it takes a lot for it to consider the first letter being wrong.
The regex library builds the NFA, so it probably bakes the fuzzy stuff into the NFA itself rather than changing the pattern.
Another option is something like word2vec where you cluster words of similar meaning together, as a bonus this usually handles typos as well. Not really in scope for your library, but I find it cool!