stephantul

Do you have benchmarks? Naively I would compare this to Numba? But maybe I am way off the mark here

This is it. Getting something on the table for stakeholders to look at trumps anything else.

I’ve never been so conflicted about an article. It has clearly been generated by an llm, but still has useful content. It’s a good article, but…

Same. Admitting to it is one thing, but still it takes a certain kind of attitude to outright forbid people to write tests.

This is such a funny interaction

Are you an llm? That would be so ironic

Tokens saved should not be your north star metric. You should be able to show that tool call performance is maintained while consuming fewer tokens. I have no idea whether that is the case here.

As an aside: this is a cool idea but the prose in the readme and the above post seem to be fully generated, so who knows whether it is actually true.

This is a thinly veiled commercial, not really useful.

Qwant is somewhat focused on France, so coverage really depends on whether that is the market you’re looking for.

Genuinely curious: what would you like to pay for? Not seeing ads? Unlocking new features? Or just supporting us?

Ecosia is not just Bing, we offer a bunch of indexes, including bing and Google.

We’re moving to our own index, which we are building in collaboration with Qwant, under the name European Search Perspective.

I do see the point of the article however.

Georgi is such a legend. Glad to see this happening

Yes. This is known as a knn classifier. Knn classifiers are usually worse than other simple classifiers, but trivial to update and use.

See e.g., https://scikit-learn.org/stable/auto_examples/neighbors/plot...

Yeah sorry, reading it back I was a bit too harsh haha. It was my pre-coffee comment. Nice post!

The speed comparison is weird.

The author sets the solver to saga, doesn’t standardize the features, and uses a very high max_iter.

Logistic Regression takes longer to converge when features are not standardized.

Also, the zstd classifier time complexity scales linearly with the number of classes, logistic regression doesn’t. You have 20 (it’s in the name of the dataset), so why only use 4.

It’s a cool exploration of zstd. But please give the baseline some love. Not everything has to be better than something to be interesting.

What an interesting long-form portrait.

Looking at your phone while driving is extremely dangerous, please don’t do it.

IMO: trust-based systems only work if they carry risk. Your own score should be linked to the people you "vouch for" or "denounce".

This is similar to real life: if you vouch for someone (in business for example), and they scam them, your own reputation suffers. So vouching carries risk. Similarly, if you going around someone is unreliable, but people find out they actually aren't, your reputation also suffers. If vouching or denouncing become free, it will become too easy to weaponize.

Then again, if this is the case, why would you risk your own reputation to vouch for anyone anyway.

What a good post! I loved the takeaways at the end of each section.

I think it would maybe get more traction if the code was in pytorch or JAX. It’s been a long while since I’ve seen people use Keras.

Yes! I distinctly remember the time magazine issue and article about this. This is exactly what I mean: we normalize and celebrate technologies without realizing what the repercussions are when we give the same tools and power to others.

You are right, I do acknowledge their efforts but did not do so here, which I should have.

I don't necessarily mean to berate the public, but rather the politicians, who saw that they could use social media/big tech for their own personal gain, and the media, who went along with the narrative that putting all our public communication into privately owned platforms was good for democracy. And maybe our own governments and institutions (speaking from a EU perspective) for dropping the ball in protecting us.

I think Evgeny Morozov's 2010-ish writing was prophetic in this regard.

Well I guess I mean the pubic in general. I also don’t necessarily mean willfully creating technology that can be abused.

For example, we all stood by when we let Twitter and other US-based social media become the main way politicians communicate with the public. This has, in my opinion, had disastrous consequences on how they communicate and actively blocks politicians from achieving consensus.

This is to say that you don’t need to have actively worked on something.

This has bean a long time coming. This is a stark reminder that you should consider who the future stewards of whatever you are building might be.

We built a vast surveillance network under the guise of servings ads and making money, and lost track of how this power could be abused by an entity not aligned with our own values.

The compression algorithm is very similar to a greedy subword tokenizer, which is used in BERT and other older language models, but has become less popular in favor of BPE.

The header line of the website (“this is our response”) makes it appear as if the experiment was already successful.

Your point about rereading without ai makes so much sense.

The economics of it have changed, human nature hasn’t. Before 2023 (?) people also submitted garbage PRs just to be able to add “contributed to X” to their CV. It’s just become a lot cheaper.

yes, you are right. I realized my mistake afterwards but it was after the edit window.

There’s many examples of noisily encoding a large embedding vocabulary. This sounds a bit like T-free or H-net? Or BLT?

One of the main issues with lines of work around this are that you end up trading embedding parameters for active parameters. This is rarely a good trade-off for the sake of compute.

Amazing post, I didn’t think this through a lot, but since you are normalizing the vectors and calculating the euclidean distance, you will get the same results using a simple matmul, because euclidean distance over normalized vectors is a linear transform of the cosine distance.

Since you are just interested in the ranking, not the actual distance, you could also consider skipping the sqrt. This gives the same ranking, but will be a little faster.

Using the phrase "without the benefit of hindsight" is interesting. The hardest thing with any technology is knowing when to spend the effort/money on applying it. The real question is: do you want to spend your innovation tokens on things like this? If so, how many? And where?

Not knocking this, just saying that it is easy to claim improvements if you know there are improvements to be had.

Stop the slop!