Hacker News Reader

Does remove words like “the” and “and” improve outcomes on this?

I’d imagine speed would

Those are typically called “stop words” and the article removed them for the example. Many NlP algorithms do remove stop words as a first step.

Although worth noting that more recent techniques (eg, Transformers) need stop words for context.

Does it actually help though? I would think embeddings of cat and the cat would be functionally similar