If we are to believe the hype though, shouldn’t these tools be launching software into the stratosphere? Like the CEO of stripe said AI tools provide a x100 increase in productivity. That was 3-4months ago. Shouldn’t stripe be launching rockets in to space now since that’s technically 400months of dev time? Microsoft is reportedly all in on AI coding. Shouldn’t Teams be the best, most rock solid software in existence now? There is so much hype around these tools being a super charger for more than a year, but the actual software landscape looks kind of the same to me as it did 3-4 years ago.
I use it quite aggressively and I'd probably only estimate 1.5x on average.
Not world changing because we all mostly work on boring stuff and have endless backlogs.
Understanding the problem to solve is.
Or: buying a super car might make your commute feel faster. But if everyone did it, we'd have a lot more congestion and a lot more pollution.
And is this a good thing since you can (in theory) multitask and work longer hours, or bad because you're acquiring cognitive debt (see "Your Brain on ChatGPT")?
Or, as another commenter said, it's for investors, not developers, and certainly not the scum users.
80% (99%?) of what you hear about llms are from the first group, amplified by influencers.
I'm guessing people feel the productivity boost because documenting/talking to/guiding/prompting/correcting an LLM is less mentally taxing than actually doing the work yourself even though time taken or overall effort is the same. They underestimate the amount of work they've had to put in to get something acceptable out of it.
Ah, Stone Soup: https://en.wikipedia.org/wiki/Stone_Soup
I'm really not seeing these massive gains in my workflow either and maybe it's the nature of my general work but it's baffling how every use case for programming I'm seeing on YouTube is so surface level. At this point I've given up and don't use it at all
Not a "100x" boost, but a pretty good take on what tasks agents can do for even very good programmers.
Besides that, I think your best bet is to find someone on youtube creating something "live" using a LLM. Something like this: https://www.youtube.com/watch?v=NW6PhVdq9R8
i feel like some kind of shill, but honestly i'm anywhere from 1.5x to 10x on certain tasks. the main benefit is that i can reduce a lot of cognitive load on tasks where they are either 1) exploratory 2) throwaway 3) boilerplate-ish/refactor type stuff. because of that i have a more consistent baseline.
i still code "by hand". i still have to babysit and review almost all the lines, i don't just let it run for hours and try to review it at the end (nightmare). production app that's been running for years. i don't post youtube videos bc i don't have the time to set it up and try to disprove the "naysayers" (nor does that even matter) and its code i can't share.
the caveat here is we are a super lean team so probably i have more context into the entire system and can identify problems early on and head them off. also i have a vested interest in increasing efficiency for myself wheras if you're part of a corpo ur probably doing more work for the same comp.
This may sound more mean than I intend, but your comment is exactly the kind of thing the GP post was describing as useless yet ubiquitous.
Without concrete details about the exact steps you're taking, these conversations are heat without light.
nobody can be bothered to show how these coding llms fall flat on their face apparently with their own real detailed examples and people can't be bothered to setup some detailed youtube video with all source code because in the end i'm not trying that hard to convince people to use tools they don't want to use.
i think with all the comments maybe the more people who stumble through this the better. the cloudflare example is a decent starting point and i've already given you the general approach. i'm fine with that being a copout lol.
Of course.
By your own reckoning. There was that recent study showing how senior devs using AI thought they were 20% faster, when they were objectively 20% slower.
and this is exactly what i'm talking about. in the end who gives a shit about some study that may or may not apply to me as long its actually working.
people literally frothing at the mouth to tell people who find it useful that they are utterly wrong or delusional. if you don't like it then just go about your day, thx.
Here are some projects Claude has helped create:
1. Apache Airflow "DAG" (cron jobs) to automate dumping data from an on-prem PGSQL server to a cloud bucket. I have limited Python skills, but CC helped me focus on what I wanted to get done instead of worrying about code. It was an iterative process over a couple of days, but the net result is we now have a working model to easily perform on-prem to cloud data migrations. The Python code is complex with lots of edge conditions, but it is very readable and makes perfect sense.
2. Custom dashboard to correlate HAProxy server stats with run-time container (LXC) hooks. In this case, we needed to make sure some system services were running properly even if HAProxy said the container was running. To my surprise, CC immediately knew how to parse the HAProxy status output and match that with internal container processes. The net for this project is a very nice dashboard that tells us exactly if the container is up/down or some services inside the container are up/down. And, it even gives us detailed metrics to tell us if PGSQL replication is lagging too far behind the production server.
3. Billing summary for cloud provider. For this use case, we wanted to get a complete billing summary from our cloud provider - each VM, storage bucket, network connection, etc. And, for each object, we needed a full breakdown (VM with storage, network, compute pricing). It took a few days to get it done, but the result is a very, very nice tool that gives us a complete breakdown of what each resource costs. The first time I got it working 100%, we were able to easily save a few thousand $$ from our bill due to unused resources allocated long ago. And, to be clear, I knew nothing about API calls to the cloud provider to get this data much less the complexities of creating a web page to display the data.
4. Custom "DB Rebuild" web app. We run a number of DBs in in our dev/test network that need to get refreshed for testing. The DB guys don't know much about servers, containers, or specific commands to rebuild the DBs, so this tool is perfect. It provides a simple "rebuild db" button with status messages, etc. I wrote this with CC in a day or so, and the DB guys really like the workflow (easy for them). No need to Github tickets to do DB rebuilds; they can easily do it themselves.
Again, the key is focusing my energy on solving problems, not becoming a python/go/javascript expert. And, CC really helps me here. The productivity our team has achieved over the past few weeks is nothing short of amazing. We are creating tools that would require hiring expert coders to write, and giving us the ability to quickly iterate on new business ideas.