Hacker News Reader

Comment by mech422 | Hacker News Reader

"Airbnb recently completed our first large-scale, LLM-driven code migration, updating nearly 3.5K React component test files from Enzyme to use React Testing Library (RTL) instead. We’d originally estimated this would take 1.5 years of engineering time to do by hand, but — using a combination of frontier models and robust automation — we finished the entire migration in just 6 weeks."

Color me unimpressed - it converted some test files. It didn't design any architecture, create any databases, handle any security concerns or any of the other things programmers have to do/worry about on a daily basis. It basically did source to source translation, which has been around for 30+ years.

View on HN

If you told me five years ago that such a conversion had been done in six weeks, I would not have believed it. Even though some level of source-to-source existed. And I would definitely expect that such a conversion would have resulted in hideous, non-idiomatic code in the target language.

> And I would definitely expect that such a conversion would have resulted in hideous, non-idiomatic code in the target language.

and we don't know what is the quality of end code, it is possible that tech debt created by migration is well higher than 1.5 eng/y.

Have you tried it?

I’ve done something similar to this type of work with an llm. It produces code that is often too idiomatic, in that it introduces conventional approaches that are overkill for the task at hand. But this is almost an ideal scenario, because if the previous tests run clean, the newly converted tests can only fail by being wrong.

They can also silently reduce the tested scenarios, but that’s what code reviews are for.

I use LLM for coding every day. LLMs can totally make something super dumb: oh this test is broken, gatcha, lets replace it with something which is not broken (but very different).

> They can also silently reduce the tested scenarios, but that’s what code reviews are for.

code review of complicated scenarios is about as resource demanding as actually writing code, so shrinking time spent from 1.5y to 6 weeks totally can produce lower quality.

Do you really think Airbnb software engineers would accept the final code if it was low quality?

Seriously, the wishful thinking around this stuff is embarassing.

There are plenty of tech bros in corps who push some hyped stuff, add to resume and move to next gig. Not sure why do you think airbnb is different.

It's easy to tell yourself you're not losing by constantly pushing the goal posts.

great comeback - "Dude - Trust me" :-P got anything more compelling then your opinion ? Have you ever done source to source translation work (I have ...) Or is this just so exciting to you cuz you've never seen it before ?

I didn't say anything remotely close to "Dude Trust Me" - perhaps you need to try a bit of "vibe learning how to read english"

exactly, you didn't provide any evidence or even decent arguements to support your position. I provided 3 distinct areas (of many) that it didn't touch that programmers have to deal with everyday. I also pointed out that the capabilities to do the sorts of transformations it did have existed for decades.

But given your comment history, backing up your arguments doesn't seem to be your strong suit...

Reply on news.ycombinator.com