"Slack uses ASTs to convert test code from Enzyme to React with 22% success rate"
This article is a poor summary of the actual article, which is at least linked to Slack's engineering blog [0].
[0] https://slack.engineering/balancing-old-tricks-with-new-feat...
[updated]
The INFOQ article is for your C types. It's the spin on what buzz words they should be using with their peers and to make a splash in the market.
NFT, Crypto, Cloud, Microservice, SAS, Podcast (the first time when there wasnt video), Web 2.0, The Ad (double-click, pre google) market, The Dot Com Bubble...
Im sure I missed a few hype cycles in there.
Both articles show how deep we are in the stink of this one.
Im tired of swimming the bullshit, It keeps getting in my mouth.
We shouldn’t detach responsibility from the publisher and author.
So I guess the publisher and author should get credit. I'll leave others to discuss the misleading comment...
The comment: ""Slack uses ASTs to convert test code from Enzyme to React with 22% success rate""
To quote[1], this 22% comes from this part:
> We examined the conversion rates of approximately 2,300 individual test cases spread out within 338 files. Among these, approximately 500 test cases were successfully converted, executed, and passed. This highlights how effective AI can be, leading to a significant saving of 22% of developer time. It’s important to note that this 22% time saving represents only the documented cases where the test case passed.
So that 22% rate is 22% saving of developer time, measured on a sample. No reasonable reading of that makes it a "22% success rate".
Over the whole set of tests:
> This strategic pivot, and the integration of both AST and AI technologies, helped us achieve the remarkable 80% conversion success rate, based on selected files, demonstrating the complementary nature of these approaches and their combined efficacy in addressing the challenges we faced.
and
> Our benchmark for quality was set by the standards achieved by the frontend developers based on our quality rubric that covers imports, rendering methods, JavaScript/TypeScript logic, and Jest assertions. We aimed to match their level of quality. The evaluation revealed that 80% of the content within these files was accurately converted, while the remaining 20% required manual intervention.
(So I guess the "80% conversion success rate" is this percentage of files?)
The Infoq title "Slack Combines ASTs with Large Language Models to Automatically Convert 80% of 15,000 Unit Tests" certainly more accurately reflects the underlying article than this comment.
Edit: they do have a diagram that talks about 22% of the subset of manually inspected files being 100% complete. This doesn't appear to be what Slack considers their success rate because they manually inspect files anyway.
[1] https://slack.engineering/balancing-old-tricks-with-new-feat...
Well, 500/2300 is 22%, so calling it 22% seems pretty reasonable.
From what I get from the rest, the 78% remaining tests (the ones that failed to convert) were "80% accurately converted", I guess they had some metric for measuring that.
So it looks like it depends on how you interpret "automatically converted 80%". If it's taken to mean "80% could be used without manual intervention", then it's clearly false. If you take it to mean "it required manual intervention on just 20% of the contents to be usable", then it's reasonable.
I live in Poland and I know this version:
Is it true that they give away cars on Red Square?
Radio Yerevan answers: not cars, only bicycles, not on Red Square, but near the Warsaw station, and they don't give them away, they steal them.