If you told me in 2005 there would be something non-human that can get a coding question right with near-coin-flip accuracy I'd say you're crazy.
It outputs some boilerplate code, and inserts a method with comments saying "insert logic here" - entirely sidestepping the request.
You can say that's remarkably human-like, but if it was a human cow-orker, you would not keep them on the payroll. It is a cheat, like ELIZA, like SHRDLU, like a useless employee.
I asked ChatGPT-3.5 for the usual rejoinder to complainers:
"If you're looking for an enhanced experience, you might want to consider trying out the paid version called ChatGPT-4. It offers even more capabilities and improvements. Feel free to explore the options available to find the best fit for your needs!""