604 karma · joined August 16, 2012
The project is abandoned, with the last update from Jan 20, 2023.
Automated systems, AI screening, and incompetent HR people are the bane of modern recruiting practices.
I asked it to refactor a medium-sized Python project to remove duplicated code by using a dependency injection mechanism. That refactor is not really straightforward as it involves multiple files and it should be possible to use different files with different dependencies.
Anyway, I explain the problem in a few lines and ask for a plan of what to do.
At first I was extremely impressed, it automatically used commands to read the files and gave me a plan of what to do. It seemed it perfectly understood the issue and even proposed some other changes which seemed like a great idea.
So I just asked him to proceed and make the changes and it started to create folders and new files, edit files, and even run some tests.
I was dumbfounded, it seemed incredible. I did not expect it to work with the first try as I had already some experience with AI making mistakes but it seemed like magic.
Then once it was done, the tests (which covered 100% of the code) were not working anymore.
No problem, I isolate a few tests failing and ask Claude Code to fix it and it does.
Now for a few times I found some failing tests and ask him to fix it, slowly trying to fix the mess until there is a test which had a small problem: it succeeded (with pytest) but froze at the end of the test.
I ask again Claude Code to fix it and it tries to add code to solve the issue, but nothing works now. Each time it adds some bullshit code and each time it fails, adding more and more code to try to fix and understand the issue.
Finally after $7,5 spent and 2000+ lines of code changed it's not working, and I don't know why as I did not make the changes.
As you know it's easier to write code than to read code so at end I decided to scrape everything and do all the changes myself little by little, checking that the tests keep succeeding as I go along. I did follow some of the recommended changes it proposed tough.
Next time I'll start with something easier.
That old project though was never fixed though, probably still running that way now.
At some point, the application had some bugs which were not appearing when the application was run in debug mode in Visual Studio. The solution was obvious: installing Visual Studio for each customer on site and teaching the users to run the app in debug mode from Visual Studio. I don't know how they convinced the users to do this and how they managed with the license but it was done.
What happened next was even worse.
There was no version control of course, the code being available on a shared disk on the local network of the company with the code copied over in multiple folders each having its own version, with no particular logic to it either, V1, V2, V2.3, V2a, V2_customer_name, V2_customer_name_fix, ...
After that, when there was a problem for a customer, the programmer went there to debug and modified the code on site. If the bug/problem was impacting other customers, we had to dispatch some guys for each customer to go copy/edit the code for all of them. But if the problem was minor, it was just modified there, and probably saved on the shared folder in some new folder.
What happened next was to be expected: there was no consensus on what was the final version, each customer having slightly different versions, with some still having bugs fixed years before for others.
Of course, when you have multiple levels or complex lambdas inside, then I agree that the for loop might be preferable.