Hacker News Reader

I think all this can be true, and we are still in a massive AI bubble that may pop at any moment.

ChatGPT truly is impressive. Nonetheless, i still think most companies integrating "AI" into their products is buzzword bs that is all going to collapse in on itself.

How are you using AI to double your coding productivity? Are you using ChatGPT, or Claude, or GitHub Copilot? I am an AI-skeptic, so I am curious here. Thanks!

I've tried various AI coding solutions and have found at best a mild boost but not the amazing multipliers I hear about online.

Copilot gives you some autofill that sometimes can be helpful but often not that helpful. I think the best it did for me was helping with something repetitive where I was editing a big list of things in the same way (like adding an ID to every tag in a list) and it helped take over and finish the task with a little less manual clicking.

ChatGPT has helped with small code snippets like writing a regular expression. I never got 100% regex mastery, usually I would have to look up a couple things to write one but GPT can shortcut that process. I get a little paranoid about AI provided code not actually working so I end up writing a large number of tests to check it, which could be a good thing but can feel tedious.

I'm also curious how other people are leveraging them to get more than I am. I honestly don't try too hard. At one point I did try really hard to get AI to do more heavy code lifting but was disappointed with my results so I stopped... but maybe things have improved a bit since then.

I don’t know if I’ve done something wrong my my copilot is so wrong I just turned it off. I don’t understand the appeal at all.

I don’t remember the last time I thought one of its suggestions was useful. For me LSP has been the real game changer.

It can get a little tedious if you are just using ChatGPT or Claude as it is. Also you are limited by lack of context on existing codebase.

That's why there are a lot of tools that help to setup a proper workflow around these LLMs.

For terminal-based workflow, you can checkout aider or plandex.

For GUI-based workflow, you can try 16x Prompt (I built it).

> I get a little paranoid about AI provided code not actually working so I end up writing a large number of tests to check it, which could be a good thing but can feel tedious.

This is a good thing. We need more tests on such critical places like regexes because they can be finicky and non-obvious. Tedious or not, we are not artists; the job must be done. Kudos for sticking to the good practices.

What does it mean to be a skeptic here? Have you tried ChatGPT? Copilot?

Perhaps I should have said "AI hype-skeptic"? I am just not seeing the productivity gains that others claim ITT.

AI is a tool, if you don't know how to use a tool you can't expect to get good results with it. That means both how to interact with the AI and how to structure your code to make the AI's generations more accurate.

If all you got is a LLM hammer, then every problem is a nail.

GPT-4o is the greatest hammer ever invented.

Got it. Are you using the latest models? Like, GPT-4o ? I find it significantly more useful when I'm stuck than copilot's autocomplete.

I don't use AI at work at all.

I pay for Leetcode, which usually gives editorial examples in Python and Java and such, and paste it into ChatGPT and say "translate this to a language I am more familiar with" (actually I have other programs that have been doing this for some language to language conversions for years, without AI). Then I say "make it more compact". Then again "make it more compact". So soon I have a big O(n) time, big O(1) space solution to Leetcode question #2718 or whatever in a language I am familiar with. Actually sometimes it becomes too compact and unreadable, and I back it up a little.

Sometimes it hallucinates, but it has been helpful. In the past I had problems with it, but not recently.

I’m not the OP and I wouldn’t say that AI has doubled my productivity, but the latest Claude models in particular have made me less of a skeptic than I was a few months ago.

I’m an experienced backend dev who’s been working on some Vue frontend projects, and it’s significantly accelerated my ability to learn the complexities of e.g. Vue’s reactivity model. I can ask a complex question that involves several niche concepts and get a response that correctly synthesizes those concepts. I spent an hour the other night trying to understand a bug in a component to no avail; once I understood the problem well enough to explain it in a few sentences, Claude diagnosed the issue and explained it with more clarity than the documentation and various stack overflow answers.

My default is no longer to assume that the model has a coin flip’s chance of producing bs. I still verify and treat answers with a certain degree of skepticism, but I now reach for it as my first tool rather than a last resort or a gimmick.

Exactly. It’s insanely helpful when u are a dev with experience in another language. You know what you want, you just don’t know the name of the functions, etc. so you put a comment

// reverse list

And it writes code in the proper language.

I want to double tap this point. In my experience Claude out performs GPT-4o, Llama 3.1 and Gemma 1.5 significantly.

I have accounts for all three and will generally try to branch out to test them with each new update. Admittedly, I haven’t gotten to Grok yet, but Claude is far and away the best model at the moment. It’s not even close really.

> I now reach for it as my first tool

The manual is my first tool.

In this case, the manual is rather poor, so a tool that can cobble together an answer from different sections of the documentation plus blog posts and stack overflow is superior to the manual.

Ok I jumped on copilot when it first came out so I have been using it for a long time.

Since I have been using it so long, I have a really good intuition of what it is “thinking” in every scenario and a pretty good idea of what it can do for me. So that helps me get more use out of it.

So for example one of the projects I’m doing now is a flutter project - my first one. So I don’t remember all the widgets. But I just write a comment:

// this widget does XYZ

And it will write something that is in the right direction.

The other thing it knows super well is like rote code, and for context, it reads the whole file. So like Dart, for example is awful at json. So you have to write “toMap” for each freaking class where you do key values to generate a map which can be turned into json. Same goes for fromMap. So annoying.

But with copilot? You just write “toMap” and it reads all your properties and suggests a near perfect implementation. So much time saved!

I don't think you need an LLM just to parse class properties and turn them into a map. Not that familiar with Dart, but that's the kind of thing IDEs have been able to do for a while now just by parsing syntax the old-fashioned way.

The thing is, when you dig into the claims many people make when they say that they get a 10x productivity boost using "AI" its usually some basic tasks that either generates boilerplate code or performs a fancy autocomplete and while those are great, in no way it supports their original claim.

I think people just want to be part of the hype and use the cool new technology whenever possible. We've seen this over and over again: Machine Learning, Blockchains, Cryptos, "Big Data", "Micro Services", "Kubernetes", etc.

I just don't think the current design of "AI" will take us there..

> they get a 10x productivity boost using "AI" its usually some basic tasks that either generates boilerplate code or performs a fancy autocomplete and while those are great

And that are just a tiny upgrade over what IDEs can do. When I used Android Studio, the code basically write itself due to the boilerplate surrounding your business logic. And once I got a basic structure down, I feel like I only write 5 to 10 characters each line (less for data types). And the upgrade is both positive and negative at the same time, it boils down to luck to actually get good suggestions.

Good point about robots. But there will be a throughput issue. You cannot accelerate physical movement.

> Seriously I am a 2x coder compared to what I was because of this.

You probably shouldn't advertise that.

They're a 20x coder now.

I am highly skeptical that a competent coder sees a 2x boost.

You shouldn't be. For code bases where context is mostly local they destroy human throughput by comparison. They only fall down hard when used in spaghetti dumpster fire codebases where you have to paste the contents of 6+ files into context or the code crosses a multiple service boundaries to do anything.

A competent engineer architects their systems to make their tools as effective as possible, so maybe your idea of competent is "first order" and you need higher order conception of a good software engineer.

could you provide some examples, code or repos and questions where it does a good job for you (question can also just be completion). Obviously you're having really good experiences with the tools that other aren't having. I'd definitely appreciate that over a lot of assurances taht I'm doing it wrong with my spaghetti code.

Try these:

Take a SQL schema, and ask AI to generate crud endpoints for the schema, then sit down and code it by yourself. Then try generating client side actions and state management for those endpoints. Time yourself, and compare how long it takes you. Even if you're fast and you cut and paste from template work and quickly hand edit, the AI will be done and on a smoke break before you're even a quarter of the way through.

Ask the AI to generate correctly typed seed data for your database, using realistic values. Again, the AI will be done long before you.

Try porting a library of helper functions from one language to another. This is another task where AI will win handily

Also, ask AI to write unit tests with mocks for your existing code. It's not amazing at integration tests but with mocks in play it slays.

Your experience mirrors mine. I use ChatGPT or meta for boiler plate code like this. I write golang at my day job, and there's a lot of boiler plat for that language, saves a lot of time, but most importantly, does the tedious boring things I hate doing.

Why does no one uses snippets or create scaffold for projects? My main requirement for a tool is to be deterministic. So that I don't spend time monitoring it as failure is very distinct from success.

Most of the things you list can be done deterministically, without the risk of AI errors. The first one in particular is just scaffolding that Visual Studio has had for Entity Framework and ASP.NET MVC for a decade now. And if you were using, e.g., LISP, you'd just write a DEFCRUD macro for it once, and re-use it for every similar project.

Those things are a tiny part of the work though and are all about generating boilerplate code. Tons of boilerplate code isn't the hallmark of a great codebase, I don't think many programmers spends more than 10% of their time writing boilerplate code, unless they work at a very dysfunctional org.

It is true it is faster than humans at some tasks, but the remaining tasks were most of the time, you can't gain more than 11% speedup by speeding up 10% of the work.

What's your point? The other person is speeding themself up, and it works for them. What's the appropriate bar for speedups. What would be enough to satisfy you? What problems do you have that AI isn't speeding up that you still feel aren't worth spending your brain on?maybe list them and see if the other people has thoughts on how to go about it?

Things don't move forward by saying it can't be done or belittle others accomplishments.

Thank you. That was a lot more informative than the argument you were having. And it's now obvious to me the value you're getting and a few areas that would work for me that I'll try. I'm not slapping out crud apps day to day, but can see how I could accelerate myself.

I appreciate it.

> They only fall down hard when used in spaghetti dumpster fire codebases where you have to paste the contents of 6+ files into context or the code crosses a multiple service boundaries to do anything.

So humans do better than them in at least 80% of all code everywhere, if not 95% even? Cool, good to know.

Care to provide some examples to back your otherwise extraordinary claim btw?

You sure seem invested in hating AI, what's your problem brother?

I don't "hate" AI because (1) AI does not exist so how can I hate something that doesn't exist? And (2) I don't "hate" a machine, I cringe at people who make grand claims with zero proof. Yes. Zero. Not small, not infinitesimal -- zero.

I just can't make peace with the fact that I inhabit the same planet as people who can't make elementary distinctions.

I can't make peace that I'm on the same planet as people who can't use google worth a shit: https://mitsloan.mit.edu/ideas-made-to-matter/how-generative...

And yet somehow want to act high and mighty and be insulting as fuck.

You failed to mention the word "can", which is a theoretical.

We can all Google stuff because internet is big and supports anyone's views, which means it's more important than ever to be informed and be able to infer well. Something that you seem to want to defer to sources that support your beliefs. Not nice finding that on a hacker forum but statistical outliers exist.

Live long and prosper. And be insulted, I suppose.

You are stealing and laundering my open source code and take the credit, that is the problem.

Fortunately, as pdimitar pointed out, so far it is an ineffective scam that mostly produces LoC.

I use multiple daily and have definitely seen a productivity boost. If nothing else, it saves typing. But I'd argue they are in essence a better search engine - it answers "you don't know what you don't know" questions very well, providing a jumping off point when my conception of how to achieve something with code or tooling is vague.

Typing is, or at least it should be, the least of your time spent during the day doing programming. I don't find optimizing the 5-10% of my workday spent typing impressive, or even worth mentioning.

Granted there are languages where typing takes much more time, like Java and C# but... eh. They are quite overdue for finding better syntax anyway! :)

The languages where typing takes more time also tend to have IDE support to mitigate that --- in a deterministic way, unlike CoPilot.

True.

I didn't mean typing in the sense of autocomplete, I meant typing in the sense of stubbing out an entire class or series of test cases. It gives me scaffolding to work with which I can take and run with.

Yes that's fair. If it helps reduce writing boilerplate then I'm all for it.

Reminds me of The Primeagen quote: “If copilot made you 10x better, then you were only a 0.1x programmer to begin with”.

As someone who uses ChatGPT and Claude daily, but cancelled my Copilot subscription after a year of use because it intimately just wasn’t that helpful to me and didn’t provide enough benefit over doing it by hand, I kind of sort of agree. Maybe not entirely, but I can’t shake the feeling that there might be some truth in it.

The code that AI generates for me is rarely good. It’s possible to get good code out of it, but it requires many iterations of careful review and prompting, but for most cases, I can write it quicker by hand. Where it really shines for me in programming and what I still use ChatGPT and Claude for is rubber ducking and as an alternative to documentation (eg “how do I do x in css”).

Besides the code quality being mediocre at best and outright rubbish at worst, it’s too much of a “yes man”, it’s lazy (choose between A and B: why not a hybrid approach? That’s… not what I asked for), and it doesn’t know how to say “I don’t know”.

I also feel it makes you, the human programmer, lazy. We need to exercise our brains, not delegate too much to a dumb computer.

> I also feel it makes you, the human programmer, lazy. We need to exercise our brains, not delegate too much to a dumb computer.

I kinda feel like this isn't talked about enough, my main concern right from the beginning was that new programmers would rely on it too much and never improve their own abilities.

I am a competent coder. I have been a coder since I was in middle school. I know at least 10 languages, and I could write my own from scratch.

I know c++ dart golang java html css javascript typescript lua react vue angular angularjs c# swift sql in various dialects including mysql and postgres, and have worked professionally in all these regards. I love to challenge myself. In fact, if I done something before, I find it boring.

So copilot helps me because I always find something new to do, something I don't understand, something I'm not good at.

So yes, I'm confident I'm competent. But I always do things I'm not good at for fun. So it helps me become well rounded.

So your assertion it only helps me because I'm incompetent is true and false. I'm competent, I just like to do new stuff.

That's all very nice but it contains a fatal logical flaw: it assumes CoPilot actually gives you good code. :D

I mean it does, sometimes, but usually it's either boilerplate or something you don't care about. Boilerplate is mostly managed very well by most well-known IDEs. And neither them nor CoPilot are offering good algorithmic code... OK, I'll grant you the "most of the time, not never" thing.

That kinda proves my point. You find it useful when you’re doing something outside your core competencies.

I don't see the problem here. What's wrong with that? Tools are supposed to make your life easier.

That’s a different discussion. I’m disputing the claim that AI can make you a 2x dev when it seems like it’s mostly beneficial when you don’t know what you’re doing.

> I know at least 10 languages

This statement would have been a huge red flag for me, if I had interviewed you. Don't get me wrong, you could use and program in 10 languages. Maybe you can be proficient in many at different times of your life. But know them at once? No.

I think for more boilerplate-esque code-monkey type code it can be a boon.

I think the unfortunate reality is that this makes up a shockingly large amount of software engineering. Take this object and put it into this other object, map this data to that data, create these records and move them into this object when then goes into that other object.

2x loc!

I’ve seen copilot spit out garbage dozens of lines long for something I swore must be one or two stdlib functions. Yep, it was, after reading some real documentation and using my brain. It was some NodeJS stuff, which I never work with. Don’t get me wrong, I still find it a helpful tool, but it is not at all a good, seasoned programmer— it is an algorithm predicting the next token based on the current tokens.

> I still find it a helpful tool, but it is not at all a good, seasoned programmer

How quickly the goalposts move! Two years ago it was "AI will never be able to write code", now we're complaining that "AI is not a seasoned programmer". What are we going to be complaining about, two years from now?

> Two years ago it was "AI will never be able to write code"

GitHub Copilot was released nearly three years ago.

I’m not really complaining, I’m saying it’s useful, but don’t pretend it’s better than what it is. It’s cruise control on a car— makes long drives better, but doesn’t literally drive for you.

lol this made me chuckle

> Seriously I am a 2x coder compared to what I was because of this.

Isn't the energy consumption of this technology pretty catastrophic? Do you consider the issue of energy consumption so abstracted you don't worry about it? Do you do anything to offset your increased carbon emissions?

They certainly are not providing these services at less than electricity costs. So if you are spending $20 a month on it, they are spending less than that on electricity. It's very low compared to any person in the first world's energy spend.

How do you know?

Common sense, having been around cloud operations for a bit. The big cloud providers run gross margins around 30%. So if openai are using MSFT and getting a "good deal" maybe MSFT only get 20% GM. So, $1 of openai compute costs MSFT say $0.80. Of that cost of providing the service, something like 30-40% goes to electricity. So, lets say the electricity cost is $0.30 for $1 of OpenAI compute. (And that's probably a steelman, I think it's actually more like $0.20. )

There is about zero chance OpenAI are running their service at a 70% negative gross margin.

Doesn't seem anything definitive to me.

Not definitive. But if you need a water tight argument to change your mind on something, you'll never change it.

Actually, they probably are. OpenAI is projected to lose $5 billion this year in ChatGPT costs.

Spend, not lose. And it's mostly on training, not inference.

They would need 250 million GPT Plus $20 subscribers to recoup a $5 billion expense. They're far from that even when we count the free users (which are likely 99% of the user base?)

The math just doesn't work. They're hemorrhaging money as far as I can tell (not counting the Azure computing deal).

We can only guess, but my guess is that inference is still a good chunk of their costs. That's why they're trying to get the mini/turbo models into a usable state.

Even then, training is still an expense. And it's not like you can train and forget. Even if your model is already trained you still need to incorporate new knowledge over time.

Redo the math....

Ouch, $5b yearly of course!

Makes it seem more tenable huh. Honestly they could charge me $200 a month and I'd pay it.

isn't the energy consumption of travel, driving, and shipping food to you pretty catastrophic? Do you consider the issue of energy consumption so abstracted you don't worry about it? Do you do anything to offset your increased carbon emissions?

> Do you do anything to offset your increased carbon emissions?

Yes. Quite a lot. I walk the talk.

You think silicon valley types care?

Rest of the developing world care even less.

I’m working on getting a place with solar panels. I think that’s important for sustainability, plus who wants to have to be connected to the grid anyway?

Offsetting carbon emissions - what does it even mean? There is no feasible way to remove carbon from the athmosphere as far as I know. Would you care to explain what you mean by that?