Hacker News Reader

Comment by nomilk | Hacker News Reader

I had an interesting experience with CoPilot yesterday. I asked it to add a 'deactivate' button to each of a collection of items. It did that, but unexpectedly, it conditionally displayed 'reactivate' if the `item.deactivated_at.present?` with appropriate bootstrap icons (a cross for deactivate and a circular arrow for reactivate).

What surprised me was it knew what I wanted better than I did (for this MVP, I hadn't even considered 'reactivate' functionality - I'd have been happy just with deactivate).

So it didn't just write code I asked for, but suggested (and was right about) something it thought I might like beyond what was actually asked for.

It was my first time using CoPilot chat so perhaps it does this a lot, but it was my first time experiencing it. It challenged my thinking and improved it.

View on HN

I find things like this too. I add detailed comments and get thoughtful autocompletes because many day to day things are solved problems.

It didn't "know what you wanted" - it simply did what has been done many times in the code it was trained on.

This feels like a distinction without a difference.

Depends entirely on how close your desires are relative to the center of the bell curve.

Great observation. When working on a scrappy project (e.g. MVP), solutions that just work might suffice (far left of bell curve).

If CoPilot bumps these crumby solutions into 'average' solutions then it's a big improvement under those circumstances.

But when making highly refined software, CoPilot's suggestions might be relatively mundane; a degradation in quality.

Even in not refined software: the other day I asked it to, in js, create a table taking (hour, power produced) data from a json response. It managed to come up with an n^2 way of doing what should have been a linear traversal through the json.

Sometimes it's great, sometimes the middle of the curve is itself bad. :)

Great point; sometimes average is not good enough. (imagine what could go wrong with a banking application or software for dangerous physical/industrial process). A simple inefficiency like the that could cause degradation in a safety system or similar.

This strengthens the case for careful assessment of when CoPilot is used vs not used, and in cases that it's used, how much effort goes into rigorously testing and scrutinizing its suggestions (as opposed to unthinkingly accepting them as one may for tasks known to have no downside risk).

Very good point, context is important.

Maybe there is an option to make Copilot prioritize security concern over other aspects when working in the context of security related apps or features.

This is how humans work, too.

Technically, this is fine tuning or prompt engineering.

It feels important to me, from the perspective of someone who doesn't want to be negatively surprised by auto-generated code. I'd expect there to be a significant difference in uncaught errors in Copilot code between people who believe Copilot knows what they want vs people who think copilot just does what it's been trained to do, mainly based on the level of scepticism people approach reviewing the code with.

I think it is a huge difference. It didn't know what you wanted. It knew what most people want. For a lot of generic things what you want is probably what most people want. Shoot, most times we ask ourselves what other people do in this situation. But in a specific case, what you want is probably not what other people want. As that is why you are writing this code, to do something different.

Some people say "she doesn't listen to me" when their Amazon Alexa fails to properly parse particular prompts.

I personally say "it" as I see it as a machine.

Learning from experience is learning!

How exactly had it improved your thinking? By your description it did it for you, you just sat back.

Doing nothing does not drive self improvement, I can't get stronger sitting on the couch watching videos on YouTube about weight lifting, even if I did type in the search term.

It challenges and improves your thinking just like a coding buddy improves and challenges you.

It's not like watching others lift, it's like having a sparring partner. ( A very fast, slightly stupid sparring partner...)

Coding a button isn't the part of software development that's intellectually stimulating, so I am glad we have robots to help us speed it up.

I feel like you're coming from a place of disliking chatbots and then working backwards.

My father was a wiz with a slide rule and understood the relations between logarithms and powers as a matter of necessity - he could look at a number and have a rough idea of what the natural log of that value was at a glance. I had a calculator - I am an expert at pushing the buttons but outside of a few select important numbers if you throw a value at me I'm not going to guess correctly - log10(7000) is clearly between 3 and 4 (it's in the thousands) - is it closer to 3.7, 3.5 or 3.9?

Tools don't improve skills they replace but, at the same time, are those skills that are being replaced important to practice? If you've worked even briefly with vector math or game programming you know sqrt(2) is 1.41 being able to summon that value at will is essential if you're regularly doing vector addition - but sqrt(23)? Probably never going to come up.

At the end of the day we need to decide whether these tools are helpful or not - in the very limited scope of UI building I think the answer is yes... a lot of front-end surface-level development has one right answer. For other things I am skeptical - and even for front-end work a developer that is considerate with their CSS classes and tag nesting will be a darling of the UI team compared to a developer that gets it to look right but does so with code that's difficult to understand or modify.

Because they learned for next time?

Are you suggesting that you wouldn't want anyone (or anything) to suggest improvements for you because then you would not learn yourself how to improve?

Learning is hard, what was described didn't sound hard to me?

You can call productive if you want, but sounds pretty far from self development to me.

Agree, that's why I only program with 1s and 0s, programming languages rob me of self development opportunities ;-)

Learning isn't hard wtf. I aced all my computer science shit and write trading code now. None of the learning was hard. It just was.

Learning is easy. It's just learning.

By "It challenged my thinking and improved it." OP probably meant 'improved upon it', or 'improved on it' where "it" was the originally desired output.

Who cares about developing skills machines can do better. I don’t spend hours with a handsaw so I can cut perfect perpendicular cuts in wood, I just use a miter saw that does it perfectly 99% of the time. That lets me concentrate on the next thing on the value chain that machines can’t do yet.

There’s plenty of woodworkers that work exclusively with hand tools for the joy of it and that where a lot of programming will be a decade in the future.

Because well, for me we are designing tools not doing manual labor. The tool is designed once and well. Then the computer executes it 50 million times. Automated tool design is cool but the ROI for us is not developer hours minimized but number of users.

The ROI for any business is number of users. What tools? Do you mean physical tools or are you referring to a program as a tool?

The software I make is the tool. The user uses my tool millions of times. The cost of adding a user is close to zero. So the power for me isn't in reducing developer costs

Yeah that’s all development/programming.

That's fair, so it isn't self development just like i said? glad you agree.

I think being able to get better at things higher in the value chain does count as self development.

You can't get stronger from watching videos. But you can learn things that you can apply next time you workout.

I have the same experiences, but i am using already for a year. Quite surprised now and then how its able to predict my next move. Sometimes remarkably accurate. I noticed here on hackernews there is a large group of copilot haters. I normally find this forum a place with the best open minded people out there. But not with copilot topics :)

Being open minded doesn't imply one has to accept incorrect terms like "knows" and "predicts" when talking about technical topics involving LLMs. HN of all communities should do better.

I mean, LLMs are basically text predictors, so it seems like a fitting term.

I was replying to:

> Quite surprised now and then how its able to predict my next move

Not text, the user's next move.

It is an aspect of an open mind to consider that a thing that looks good might not be.

Reply on news.ycombinator.com