Hacker News Reader

Comment by holoduke | Hacker News Reader

I am using copilots now since a few months and it really makes me a 2x more productive developer. Its like you become an orchestrator of a dev team. You still need to look into details, but things just flow much much faster. I can only imagine how it will be like if I also have access to a AI debugger / end to end tester. Completing the loop and making it super efficient. Also think that this is not only the case for developers. Even for lawyer it could be the same thing. Expected production output of a worker is going to rise. The ones who do not embrace AI assistants in the near future will have a hard time in the future.

View on HN

> Even for lawyer it could be the same thing

Funny you mention that, since lawyers have already gotten in trouble for citing fictional cases when submitting work performed by chatgpt.

It's useful for rote things, but for anything that you depend on, you still need to give it just as much attention as if you'd done it yourself.

I agree, but it does change the shape of the work you would be doing. Validating work isn't the same as generating new work.

Whether that's better, or useful, probably differs by situation.

> Validating work isn't the same as generating new work.

in many cases it's way harder

> you still need to give it just as much attention as if you'd done it yourself.

It's different than a Lawyers case, where the facts require manually cross referencing. In our case, that verification can come via directly and immediately running the code. How good the actual code is varies, but the fundamental difference remains.

> Funny you mention that, since lawyers have already gotten in trouble for citing fictional cases when submitting work performed by chatgpt.

For something like legal work, you don’t want a raw LLM. You want an agent integrated with a legal database, in such a way that it can’t cite cases which don’t exist in the database. And it can’t generate a direct quote from a case unless that text actually occurs in the case.

You still need a human lawyer familiar with the law to pick up on subtler errors, but better technology can prevent the grosser ones. And the subtler errors (e.g. misrepresenting what a case says through partial or out of context quoting) is the kind of error human lawyers sometimes make too - like every other profession, lawyers vary greatly in their competence, and often only the grosser cases of incompetence incur sanctions

Tried Lexis+ AI, and… it’s just not very good yet.

Much like ChatGPT, it can only handle recall and summation — and even then, I don’t fully trust it because it often misses key ideas.

And much like ChatGPT, it can’t do anything coherent at length without a lot of help and working around its faults.

And seems entirely unaware of similar sounding words having distinct legal meanings. Which is not good.

> Tried Lexis+ AI, and… it’s just not very good yet.

I wonder to what extent that’s due to current inherent limitations of the technology, and to what extent it is due to quality of implementation issues.

It is hard to say because (I presume) there is only limited public information on how it is actually implemented.

e.g. which LLM is it using? How much fine-tuning has been done? Are they using other potentially helpful techniques such as guided sampling? Or breaking down the task into parts and having multiple agents each specialised to handle one particular part?

> Much like ChatGPT

When you compare it to ChatGPT, do you mean GPT3.5 or GPT4?

I also would guess that certain areas of law (especially criminal law) may be prone to triggering “safeguards” which result in poorer performance than if those safeguards were absent. Arguing that what your client did was legal (even be it ethically unsavoury) is an essential part of a lawyer’s job

I still haven't figured out how people use this that much. I use LLMs almost daily via ChatGPT, Phind, Kagi Ultimate (my current one), etc. However i spend too much time pushing the LLM towards my goal that i can't imagine it speeding my coding up.

I clearly find value in LLMs to some degree, but speeding up my coding is not yet one.. i'd love to, but i just don't understand. I can only imagine typing more in explanation for the LLM than it would take me to write it to begin with.

IMO a chunk of it is like Peel developers: people that prioritize their own speed above all else: readability, team reviews, etc. I'd love to be proven wrong.

I rarely find interfacing with an external chat interface useful, but integration with the coding environment (e.g. Copilot) is an immediate productivity boost.

Some revered novelists spend all day to write a single page and others are able to get in a flow and produce a chapter in that same time.

Without getting into touchy questions of quality and talent and the whole 10x trope, there are a lot of working engineers out there that produce code a lot more slowly than others and that work on more common problems than others. My sense is copilot-like products provide the biggest boon to those people and are a lot harder for people who are more naturally prolific or who work on more esoteric things.

Have you ever tried it?. It sounds like you are a bit against the idea of an AI supporting your work.

It all seems to depend on what you are doing. The more niche and technical the task, the less the AI can help. If you are just generating crud and points for a common web language and framework, it can be an 80% boost.

I think the real problem with this is that people aren't differentiating these different types of work when they give these numbers.

It's not just the type of work but also experience. Here are two cases:

1. After working for many years in Java I needed to build a service. I spent few days designing it and then a month on implementation. I used DBs and libraries I knew very well. I didn't need to access google/stackoverflow, I didn't need to look up names of (std)lib methods and their parameters and if something wasn't working it was fairly obvious what I needed to change.

2. Recently I wanted to create a simple page which fetched a bunch of stuff from some URLs and showed the results, simple stuff. But with React since that was what frontend team was using. I never used React and rarely touched web in the recent years. Most of my time was spent googling about React and how exactly CORS/SOP work in the browsers, and with polishing it took a couple of days.

I'm pretty sure that in case 1) AI wouldn't help me much. Maybe just as a more fancy code completion.

In case 2) AI would probably be a significant time save. I could just ask it to write some draft for me and then I could make few tweaks, without having to figure out React.

But somehow nobody quantifies their experience with the languages/tools when they are using AI - I'm sure there's a staggering difference between 1 month and 10 yoe.

For myself I have several distinct cognitive weaknesses that I can often just unload onto an AI in a conversational style.

For instance I know I'm good at breaking up large tasks into smaller ones, but because planning and executive functioning is by far my worst energy consuming skill (adhd) I can save a lot of energy (but not time) by approaching it conversationally. I'd often use colleagues for the same thing, but then my productivity costs 2 salaries

Similarly, I have some trouble with memory and cognitive speed when I'm tired which is unfortunately often the case due to my health, I know well enough what I kinda "want" to do and I let the AI generate something that comes near what I need and I can work from there once I have the right starting point.

Just my personal experience I'd wanted to share.

Its usually with simple things like code completion on logging, boiler plate, repetitive tasks etc. I am surprised sometimes how copilot knows what my next step is. I am typing in the start of an algorithm and copilot gives me the rest. And it predicts a lot of things correctly. It saves me tons of time. Another thing what i like is that I no longer need to no all the programming language related syntax. In the past I looked up stuff on stackoverflow. Now i simply type a short comment like "reverse the array...". Copilot automatically suggests the right syntaxes. Sometimes need some adjustments, but thats fine.

Try this: in a situation where you need a small change to an existing class that you haven't looked at in a long time, dump the code and spec to ChatGPT with a request to add the feature or make the change along with a supporting spec. This can really speed up getting to the final result.

I feel like there's too many files (lots of out-of-file references) to do that easily. Probably is fine for something like Copilot, but manually feeding it to ChatGPT is not something i've had much faith in

So I wanted to take a heatmap and bin it into fifteen values on a logarithmic basis. So I asked ChatGPT to do it for me, and it just worked.

Would this have been difficult for me to just write? No. I wouldn't call it difficult. But it would have involved effort, which is a resource I'm happy to conserve. It's like the difference between a sandwich you make and a sandwich you ask someone to make.

Then I asked it to generate the integer ranges which correspond to each bucket. It screwed that one up, which I found out by copying the function to a scratch file and trying some representative values in the REPL. So I told it to iterate all the values between in and max and generate the ranges that way. That one worked. Net of less effort, plus I had a test for it which I could copy-paste from the REPL to the test suite.

Faster? Maybe, maybe not. But I'm rate-limited by gumption, not minutes. When it's easier to describe the function than write it, and it's simple enough that the robot won't screw the pooch, I hand it off to the LLM. It's a great addition to the toolkit.

The co-pilot equivalent tool I use makes me about 10-15% more productive. Noticeable but not life changing.

> The ones who do not embrace AI assistants in the near future will have a hard time in the future.

The exact opposite will be true and the funny (sad?) part is that they will lack the skills necessary (because they got lazy and over-trusted the AI) to fix mistakes/incompatible solutions.

My thinking as well. If coding assistants become so good that I'm at a competitive disadvantage, I'll just start using them. It's not rocket science. So far, everything I've tried largely slowed me down. As a Google/SO replacement for some types of questions, they sure save me maybe an hour per week, but that's really all I could extract so far.

Maybe my work is not too typical though, I spend only a fraction of my time actually typing in code. And I do eliminate the need for boilerplate through other means (picking frameworks/libraries that are a good fit for the problem, refactoring, meta programming, scripts, suitable tool chain etc).

1 hour per week of increased coding is a 5-7% boost in productivity, using the Amazon guidelines for how SDEs use their time — 50%/20hrs for SDE1 and 33%/13hrs for SDE3.

Is that enough to be competitive?

I’m not sure — but at scale that would be a 5% reduction in headcount for the same work, or ~$12M/yr for every 1,000 engineers.

If you can figure out how to get 2-3 hours more coding done a week, we’re talking real gains.

Depends on where the time is saved. If it's in figuring out how to do something in Django where StackOverflow is flooded with outdated answers, sure. I see those kinds of savings. But the tragic beauty of programming is that a little time saved today can very well mean lots of time lost later. The former you can measure, the latter is a tougher nut.

People said the same thing about garbage collectors.

And sure enough, people practiced only in garbage collected environments are the ones who struggle most to work with Rust's borrow checker or write sound embedded/IOT code, or to attend to refence leaks in things like event listeners.

Did they help people write lots of effective code faster? Yes.

Did they breed a generation of people with little intuition around how memory works? Also yes.

This statement boils down to "people who know C or C++ have an easier time learning Rust" which is not especially informative, or even interesting.

That's a terrible comparison. GC is there to take care of one specific implementation detail in a "good enough" way so that you can focus on more productive work.

People are starting to rely on LLMs to do almost everything they've been hired to do for them.

I’ve had it enabled for months across both Javascript and Kotlin codebases and it’s…fine? Good enough that I leave it enabled. But only barely. I’m certainly not orchestrating a dev team.

It has probably the same productivity boost that intellisense gave back when it came out. Which is good, but still marginal. Certainly not replacing anyone’s job.

in the future, it seems like we might just become PR reviewers

Reply on news.ycombinator.com