Hacker News Reader

I often find it faster to write something from scratch rather than to work with someone else’s code to fix it. In the latter case I need to understand the intent, the whys behind the choices.

Well guess what, LLM-generated code is someone else’s code: an amalgamation derived from many peoples’ code. Except those people are ‘helpfully’ “abstracted away” from you by the middleman, so you can’t know their original intents and choices. What’s worse, it’s someone else’s code that will be treated as your code—unlike working with a legacy system that everyone knows was written by some guy, in this case any bugs will be squarely on you.

This offering, and the other half-dozen like it this past week or so, is like giving a kid a flamethrower.

It's all fun and games until they burn down your house.

> ... I need to understand the intent, the whys behind the choices.

As do I.

And that is something ChatGPT-X (for any given X) cannot provide, regardless of whether or not what is produced is correct. Perhaps with some form of backward chaining[0] a ChatGPT-X someday can explain how it arrived at what was produced works.

But "the why" is the domain of people.

0 - https://en.wikipedia.org/wiki/Backward_chaining

It's weird to see a forum for hackers, with hacker in the name, and with a line about encouraging curiosity in the charter, be so hostile to someone who hacked something together.

Sign of the times perhaps.

Though I guess it's not much different from the thread trashing Dropbox however many years back.

> It's weird to see a forum for hackers, with hacker in the name, and with a line about encouraging curiosity in the charter, be so hostile to someone who hacked something together.

My comment was in direct response to an overarching concern raised by the implications of incorporating "LLM-generated code." This is relevant here due to the "Show HN" description above, which reads thusly:

  Regex.ai is an AI-powered tool that generates regular
  expressions. It can accurately generate regular expressions
  that match specific patterns in text with precision.

If you interpreted my characterization of "... like giving a kid a flamethrower" as being hostile, then I extend my apologies to the OP as I was using this phrase as a literary tool detailed subsequently. I thought the subject expansion of "the other half-dozen like it this past week or so" was sufficient.

As to "encouraging curiosity", I point you to feedback I provided to the OP in a reply peer to this one.

I’m not critical of the hack itself (unless it uses OAI’s closed commercial LLMs). Just not a fan of some implications of using it in real circumstances: it might work for a personal thing but if you use it for anything important you still need to know how regular expressions work.

>so hostile to someone who hacked something together.

It's not hostile but I'm a bit tired of all those projects that sprout around AI.

If it was an open-source project full of bugs, I would understand, and encourage and give solutions to the creator of the project, maybe even create tickets or fix bugs.

But with AI, we are flooded with tons of closed-source frontends to a closed-source backend, and those projects are more than buggy since they confidently give bad solutions. It's not like a "DIY electric car project," it's someone putting pieces of cardboard on a Tesla and pretending it makes it safer or faster.

I'm dumbfounded and I don't know how I am supposed to react to this, I would certainly not release that to anyone since it's antithetical to what I do and believe what software should be.

Good point. I wish OpenAI released more of their work as open source. I wish people building on top of them did too. That said, I usually won't begrudge a small-time developer or entrepreneur from choosing whatever licensing model they think is going to make them the most money. An army of small-time entrepreneurs who build closed source can still have democratizing effects on a market that's been captured by a few large companies. I'm more frustrated when I see big, entrenched companies finding ways to capture value from the open source ecosystem and privatize it.

My view on v1s, prototypes and PoCs regardless of their licensing is that by design they're going to be a mess and have errors, if they don't you waited too long to ship. Maybe these folks should have been a little more honest in their marketing but man if we're going to get into a list of the offenders on that front I think they are way way down on that list.

Overall in my view LLMs are the most disruptive thing to come along since the Web itself. Business model's like Google's are facing a direct challenge from this technology. Why do I want to look at Google's first page full of shitty search ads when I can use a LLM to get an answer immediately? As far as I'm concerned at this stage I would love to see a billion projects from every corner of the world built on top of this technology. Whether they're great or they're crap, the avalanche is the first real opportunity in many years to disrupt some giants.

Are you trying to say that every sort of criticism equals hostility? If I dont like your half-thought-out idea, I am hostile. If I praise it, I feel like an idiot. Not much choice remaining after all....

You have a choice of not replying at all.

Sure, that seems to be the overall way forward. Keep quiet to avoid being attacked for speaking your mind.

If I was trying to say that, I imagine I would have just said that

> It's weird to see a forum for hackers, with hacker in the name, and with a line about encouraging curiosity in the charter, be so hostile to someone who hacked something together.

I guess people are getting tired of too many topics in one narrow space. I come to HN for variety. It does get tiring when every single day I see yet another LLM-based solution attempting to solve a problem I don't think I even have.

Overdose of a certain topic is not good for a general tech forum like this. Everything should be in moderation and all that.

This forum is also against decentralization and Web3, and often shills for large centralized corporations. The ethos of hackers was always ANTI that stuff.

So if I look at most codebases, someone would be able to explain what all the code does and why it does that way? I'm extremely sceptical of that, even if I myself wrote the code 3 weeks ago.

A person should be able to explain the code they're adding to a repo at the time they are adding it. Whether or not they can explain it at some arbitrary point in the future is a different question/issue.

You can ask it to explain why. It might not be a true representation of why those decisions were made but at least it’s a plausible explanation of why something could work like that which is better than nothing. I’m not sure why you think it can’t do that already?

or you know, you can ask it to insert helpful comments

It's even worse. When working with someone else's code e.g. StackOverflow there's a reputation system gating people from the platform and incentivizing them to provide correct answers. You can reasonably expect that someone else's code has at least been thought through to some extent to solve the problem at hand, and very likely tested.

With LLM-generated code, especially ChatGPT-style decoder models, none of that is true. All of the posts and comments I see about it here seem to be anecdotes "it can do all of my job for me" yet asking it to write the simplest code creates several issues on my end.

Personally I think a model geared towards code generation isn't an unsolvable task; the Spider dataset was released some time ago (text to SQL task) and the winning approach there was no fanciness on the model side, but rather to just test all the output queries to ensure it's at least valid SQL. That got a 20%+ boost in accuracy.

Your experience is no less anecdotal than the millions of people who successfully use Copilot and ChatGPT to write code on a daily basis. I am one of those and can't imagine coding without Copilot or an equivalent ever again.

Define successfully. You might verify what the LLM gives you, but lots of people who blindly copy and paste from stack exchange will do the same with chatgpt

Software exists over time. There is no “successful” unless you account for future bugs.

I do believe LLM code generators can be used with good results. I just know that for me that way is slower and more painful, because I need to switch between creative mode (when I make stuff) and debugging mode (when I need to figure out how someone else’s stuff works). I find keyboard typing speed is usually not what slows me down the most…

> the millions of people who successfully use Copilot and ChatGPT to write code on a daily basis

Where did you get that number from? Are you saying that roughly one in a thousand person on Earth, alive today, is using Copilot and ChatGPT to write code on a daily basis?

Not the parent but it's not completely impossible. According to [0], there's about 25-30 million software developers in the world. If about 7-8% of them use ChatGPT and Copilot every day, it's already (two) millions.

[0] https://www.bairesdev.com/blog/how-many-software-developers-...

I guess it's early for this to matter too much for the count, but people who are not "developers" have also used ChatGPT to write code. I've read anecdotes.

I'm (genuinely) curious what kind of code you write. I haven't tried Copilot and I haven't used ChatGPT very much, but I feel I would be pretty surprised if either of them made significant improvements to my workflow.

Copilot I could see, since I already use Intellisense, autocomplete, and snippets to great effect. I'd be annoyed if I had to work without them. But in general, knowing what I want the code to do is >90% of the work of writing new code.

I feel there are a few possibilities for why I'm confused:

1. I'm not a very good software engineer, at least in certain respects. Maybe I should have a better understanding of architecture patterns or something I might have learned in a CS degree. Maybe I am hacking everything together and maybe I am already a slow coder.

2. I'm not [being] creative enough as a prompt engineer. I typically can't think of any way that ChatGPT could help me without ingesting my entire repo and figuring out the correct patterns. It could be, however, that there are ways to get the answers I need with better questions.

3. We do completely different kinds of work, and some kinds of coding are better suited for AI assistance than others.

The opposite of 1 is also possible. You're a really good programmer and know the material better, and just don't need to ask the kinds of questions that other people are asking ChatGPT (or stack overflow, or man pages) for/are happy with your current reference materials.

Like autopilot in planes that fall back to experienced pilots, we're embarking on the most dangerous "uncanny valley" maneuver where these systems will be adopted by experienced pilots who know the limits but who will inevitably be followed by either no one or students whose conception is entirely synthetic.

At that point the plane AI better be 100% TRUSTWORTHY cause there's no safe fallback.

Thanks for your feedback! Updated ver 1.1 coming soon with more descriptions and better performance :)

If you have a choice between descriptions or performance, I humbly suggest detailed descriptions perhaps with links to tutorials and/or further reading. Who cares if the wrong thing is returned quickly if that means it lacks any context.

Also, consider how to express anchoring and/or grouping preferences in the UI or weighting based on highlight positioning. These are oft used features of regex languages.

I’d you don’t understand regexes well enough to write them yourself, you should not get some ai to generate them for you. You won’t be able to verify whether they do what you want and the bugs can be subtle and destructive

I read a few weeks ago here on HN about one large SAAS grinding to a halt because of a greedy selector in one line of regex. Not sure how people find old stories, it's lost to me now. But it was an excellent example of why regex is dangerous and requires a lot of care to write. I wouldn't trust an AI to write my regex unless I saw that people were finding it to be consistently better than they were are writing what they need.

You gave it an example where inferring the semantics you were after was basically a crapshoot. It’s not going to do well under those conditions. Nor will a random human who lacks insight into what specifically you are after. Did you want all the bazzes that are at the end of lines? The bazzes that follow bars? Who knows?

Try giving it examples where the data provides context cues.

Using your example and deselecting the email addresses I end up with these suggestions:

\b(foo|bar|baz)\b

\w(foo|bar|baz)\w

\bbaz\b

[fF][oO][oO]|[Bb][Aa][Rr]|[Bb][Aa][Zz]

It only lacks a dice button which randomly selects the "correct" answer.

There are tools that somewhat explain what each part of a regex does.