if [ -z "${Var}+x" ]
Where I can see what the author was trying to do, but the code is just wrong.
I dont mind people not knowing stuff, especially when it's essentially Bash trivia. But what broke my heart was when I pointed out the problem, linked to the documentation, but recieved the response "I dont know what it means, I just used copilot" followed by him just removing the code.
What a waste of a learning opportunity.
There were many times in my career when I had what I expected to be a one-off issue that I needed a quick solution for and I would look for a quick and simple fix with a tool I'm unfamiliar with. I'd say that 70% of the time the thing "just works" well enough after testing, 10% of the time it doesn't quite work but I feel it's a promising approach and I'm motivated to learn more in order to get it to work, and in the remaining 20% of the time I discover that it's just significantly more complex than I thought it would be, and prefer to abandon the approach in favor of something else; I never regretted the latter.
I obviously lose a lot of learning opportunities this way, but I'm also sure I saved myself from going down many very deep rabbit holes. For example, I accepted that I'm not going to try and master sed&awk - if I see it doesn't work with a simple invocation, I drop into Python.
E.g. maybe it would be very 'elegant' or rather concise awk if I could overcome the learning opportunity, but like you I would probably decide not to; I'll do it with the sed I do know even if it means some additional piping and cutting or grepping or whatever that awk could've done in one, because I already know it and it's going to be clearer to me and probably anyone else I'm working with.
I think we're saying quite similar things, but my point is I wouldn't be deleting it, dismissing the idea, and disappointing the colleague ready to teach me about it - because I never would've been willing to blindly try broken AI generated (or however sourced) code that I didn't understand in the first place.
An afternoon learning your way around bash or vim will save you countless of hours of work, just because you will know what the building blocks are, you will be able to ask the right questions directly instead of chasing down alleys.
It's not the same thing as learning yet another language. It's a separate type of tool. Developing software without knowing an editor or a shell is like refusing to learn what a file is or what an ip address is. Sure, you can probably get work done in roundabout ways, but it's certainly not rational.
It is not nonsense. You use that expression if you want to check if a variable exists or not (as opposed to being set to an empty string) which is an extremely common problem.
There was a real and correct analysis that, hey, I want to make sure the variable is set here. Only to then drop it because you get told the syntax is wrong. The response I'm looking for when I say "This syntax won't do what you're looking to do" would be something like "what am I trying to do? and why wont this do it?" not "well it's just some AI code, I'll just remove it".
But I agree I wouldn't extrapolate too much from one such brain fart
Generative AI: hold my bear
I think the problem with “AI” code is that many people have almost a religions belief. There’re weirdos on internet who say that AGI is couple years away. And by extension current AI models are seen as something incapable of making a mistake when writing code.
So I hope you're right, but the evidence is currently that you're wrong. Let's see how it plays out, I suppose.
- That said, surely traffic will decline for Q&A sites. "How do I connect tab A into slot B" is something that people are likely to query LLMs for; the response will surely sound authoritative, and could be even correct. That's definitely a task where LLMs could help: common questions that have been asked many times (and as such, are likely to be well-answered in the human-made training data). A 20001st question of "how do I right-align a paragraph in HTML" has not been posted? Good. Rote tasks are well-suited to automation. (Which, again, brings us back to the issue "how to distinguish the response quality?")
Now imagine that HTMZ comes along and people just go straight to asking how to full justify text in HTMZ for their smart bucket. What happens? I doubt we’ll get good answers.
It feels like the test of whether LLMs can stay useful is actually whether we can stop them from hallucinating API endpoints. If we could feed the rules of a language or API into the LLM and have it actually reason from that to code, then my posed problem would be solved. But I don’t think that’s how they fundamentally work.
So, I think the answer is that since all useful data is already in a LLM somewhere all new data will be stolen/scraped and inserted in real time. So if real people are answering the question it will work as normal. The real question is what happens when people are trying to mine karma by answering questions using an LLM that is hallucinating. We have seen such with the Bug Bounty silliness going on.
The problem, of course, is that this might work once in a while for low hanging fruit, until the web inherited things like DICOM and we now have medical imaging in the web browser (I've heard in Apple Vision Pro), where robotics implies the price of unforeseen bugs is not accidental death or dismemberment of one patient, but potentially many.
His clients were usually older small business owners that just wanted a web presence. His rate was $5000/site.
Within a few years, business dried up and he had to do something completely different.
He also hosted his own smtp server for clients.It was an old server on his cable modem in a dusty garage. I helped him prevent spoofing/relaying a few times, but he kept tinkering with the settings and it would happen all over again.
Unless he's so good at selling his services he can consistently find new clients. And if that's the case, he'd probably kill it in sales.
That business model only works until customers are sophisticated enough to understand tech debt. In the future, more customers will be less willing to pay the same good wages for low quality code.
> but his customer / employer bought low quality code with lots of tech debt.
I can completely believe someone had no idea what they were doing when copy/pasting, and working on wordpress.
Arguably the term for a bad idea that works is "good idea"
Asking it to refactor / fix it made it worse bc it'd get confused, and merge them into a single variable — the problem was they had slightly different uses, which broke everything
I had to step through the code line by line to fix it.
Using Claude's still faster for me, as it'd probably take a week for me to write the code in the first place.
BUT there's a lot of traps like this hidden everywhere probably, and those will rear their ugly heads at some point. Wish there was a good test generation tool to go with the code generation tool...
Having mistakes in context seems to 'contaminate' the results and you keep getting more problems even when you're specifically asking for a fix.
It does make some sense as LLMs are generally known to respond much better to positive examples than negative examples. If an LLM sees the wrong way, it can't help being influenced by it, even if your prompt says very sternly not to do it that way. So you're usually better off re-framing what you want in positive terms.
I actually built an AI coding tool to help enable the workflow of backing up and re-prompting: https://github.com/plandex-ai/plandex
But in my experience, if you continue iterating from that point, there's still a risk that parts of the original broken code can leak back into the output again later on since the broken code is still in context.
Ymmv of course and it definitely depends a lot on the complexity of what you're doing.
The challenge is writing in a tone that will gently move the conversation rather than refocus it. I can’t just inject “remember point n+1” and hope that’s not all it’ll talk about in the next frame.
If nothing else, LLMs have helped me understand exactly why GIGO is a fundamental law.
https://news.ycombinator.com/item?id=40922090
LSS: metaprogramming tests is not trivial but straightforward, given that you can see the code, the AST, and associated metadata, such as generating test input. I've done it myself, more than a decade ago.
I've referred to this as a mix of literate programming (noting the traps you referred to and the anachronistic quality of them relative to both the generated tests and their generated tested code) wrapped up in human-computer sensemaking given the fact that what the AI sees is often at best a lack in its symbolic representation that is imaginary, not real; thus, requiring iterative correction to hit its user's target, just like a real test team interacting with a dev team.
In my estimation, it's actually harder to explain than it is to do.
python -mvenv aider
aider/bin/pip install aider-chat
aider/bin/aider
Just don't mess with the system-wide installed version of python and it will be fine. This isn't a python specific issue though.
(https://aider.chat/docs/install/pipx.html will install it globally on your system within its own python environment. This way you can use aider to work on any python project, even if that project has conflicting dependencies.)
I run it with these settings: `aider --sonnet --no-auto-commits--cache-prompts`
if you want to run the most bleeding edge version at the same time without breaking anything,
`pipx install --suffix=@dev git+https://github.com/paul-gauthier/aider.git`
then run it with `aider@dev`
`aider@dev` ... with your desired settings
True, but the anecdote doesn't prove the point.
It's easy to miss that kind of difference even if you wrote the code yourself.
The developer in the story had no idea what the code did, hence they would not have written it themselves, making it impossible for them to “miss” anything.
Yes.
AI as a faster way to type: Great!
AI as a way to discover capabilities: OK.
Faster way to think and solve problems: Actively harmful.
Artificial Incompetence indeed!
Hear hear!
I feel like genAI is turning devs from authors to editors. Anyone who thinks the latter is lesser than the former has not performed both functions. Editing properly, to elevate the meaning of the author, is a worthy and difficult endeavor.
I'm the first to bash JS, but this is not a JS issue. It's 100% a "bad and lazy" human actor that is throwing spaghetti on the wall to see what sticks. In this case, they have a minigun cannon called ChatGPT/CoPilot that is letting them do more of it than what they used to.
Many statically typed languages do have escape hatches to do some dynamic typing at runtime, but this is not the default (hence the classification), and it requires some additional effort to use.
"Create Python code for iterating a file with a cache."
class CachedFileIterator:
def __init__(self, file_path):
self.file_path = file_path
self.cache = [] # List to store cashed lines
self.current_index = 0 # Index for the current line to read
def _read_lines_from_file(self):
"""Read lines from the file and cache them."""
with open(self.file_path, 'r') as openFile:
for line in f:
self.cache.append(line.strip()) # Strip newline characters and save to cache
def __iter__(self):
"""Return the iterator object itself."""
return self
def __next__(self):
"""Return the next line in the iteration."""
if self.current_index < len(self.cache):
result = self.cache[self.current_index]
self.current_index += 1
return result
else:
# If we reach the end of the cache, we read more lines from the file if necessary
if len(self.cache) == 0: # Cache is empty, read file
self._read_lines_from_file()
if self.current_index < len(self.cache):
result = self.cache[self.current_index]
self.current_index += 1
return result
else:
# If still no lines, we stop iteration
raise StopIteration
# Example usage:
if __name__ == "__main__":
file_path = 'example.txt' # Replace with your file path
iterator = CacheingFileIterator(file_path)
for line in iterator:
print(line)But this is actually so much worse for that same reason - the type of developer who'd submit Copilot output (I can call it that, as it's definitely not code) for a PR is unable to respond to any comment beyond asking Copilot again and wasting everyone's time with 6 more rounds of reviews. I've literally had to write test cases for someone else and told them "You can't ask for another code review until your code passes these."
With the big difference obviously being that typically the developer who writes the test also will write the code.
In some situations, this actually makes sense to do with junior developers as part of their training. Where a senior developer sits down with them and write out the tests together, then with the tests as a guide they are thrown into the waters to develop the functionality.
Of course, I suspect that in this case, you were not dealing with a junior. Rather the sort of person who looks at your tests, still is confused and asks for a "quick call" to talk about the tests.
Such a good idea :-) Maybe for job applications too and any at home work sample tests
The only thing I'd call an actual mistake is using an empty list to mean both an empty file and an uninitialised value.
def cached_file_iterator(file_path):
with open(file_path, 'r') as f:
lines = [ line.strip() for line in f.readlines() ]
yield from iter(lines)
# Example usage:
if __name__ == "__main__":
file_path = 'example.txt' # Replace with your file path
iterator = cached_file_iterator(file_path)
for line in iterator:
print(line)
for line in f:
iterating over the file object at all instead of just calling self.cache = openFile.readlines() means that calling strip() the line below removes data beyond just the trailing newlines.
with open(self.file_path, 'r') as openFile:
for line in f:
Certainly if you are in a conversation mode after a few back and forths this happens from time to time.
I am just not going to spend my time digging to previous prompts of code I might not want to share just to satisfy a random internet person .
Maybe you meant the latter?
Lots of mistakes, but never this one.
Well. We definitely agree on that.