Hacker News Reader

Hacker News

Comment by gbalduzzi | Hacker News Reader

Parent Full thread

gbalduzzi

Neat idea, but why should I use AI for a find and replace?

It feels like shooting a fly with a bazooka

View on HN

it's like hiring someone to come pick up your trash from your house and put it on the curb.

it's fine if you're disabled

Bazooka guarantees the hit

I like LLMs, but guarantees in LLMs are... you know... not guaranteed ;)

I think that was the point

If all you have is a hammer.. ;)

do you know all the spells you're looking for from memory?

You could just, you know, Google the list.

and then the first thing you see will be at least one of ITS AI responses, whether you liked it or not

You're missing the point, it's only a testing excersize for the new model.

No, the point is that you can set up the testing exercise without using an LLM to do a simple find and replace.

... I'm not sure if you're trolling or if you missed the point again. The point is to test the contextual ability and correctness of the LLMs ability's to perform actions that would be hopefully guaranteed to not be in the training data.

It has nothing to do about the performance of the string replacement.

The initial "Find" is to see how well it performs actually find all the "spells" in this case, then to replace them. They using a separate context maybe, evaluate if the results are the same or are they skewed in favour of training data.

Its a test. Like all tests, its more or less synthetic and focused on specific expected behavior. I am pretty far from llms now but this seems like a very good test to see how geniune this behavior actually is (or repeat it 10x with some scramble for going deeper).

This thread is about the find-and-replace, not the evaluation. Gambling on whether the first AI replaces the right spells just so the second one can try finding them is unnecessary when find-and-replace is faster, easier and works 100%.

Reply on news.ycombinator.com