Google Colab will soon introduce AI coding features
blog.google
blog.google
This looks like an announcement of an announcement, which isn't on topic for HN. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
On HN, there's no harm in waiting for the actual thing: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
The way I've understood the "announcement of an announcement" rule in the past is that it is for reports/rumors that somebody is company to release something, or a company sending a teaser for a future event where they'll announce something.
The standard has never been that the product needs to already be generally available to be on-topic. If that were the standard, when could we discuss say a new iPhone? Not when it's talked about on the big Apple stage. Or as another example consider yesterday's discussion of the changes to the Google account deletion policy. No accounts will be deleted for 9 months, was it an announcement of an announcement?
The comments in this thread are nearly entirely generic, though.
(1) run notebook in VS-code https://code.visualstudio.com/docs/datascience/jupyter-noteb...
(2) install Github Copilot extension https://marketplace.visualstudio.com/items?itemName=GitHub.c...
and then it can quite often achieve what you want with just a comment or some previous code snippets.
Then if you load in pandas and a DataFrame you want to analyse, it quite often suggests the next stage of your data-analysis work, either right away, as you write the first step in some chain of pandas methods.
I think you meant "with the Github Copilot plugin installed".
I think Colab doesn't support this, but Paperspace Gradient for example does.
https://docs.paperspace.com/gradient/notebooks/notebooks-rem...
I have been using CoPilot for about a year (mostly in Emacs, sometimes in VSCode) and I find it very useful, so I am especially excited to see what Google offers.
I also find myself using Bing+ChatGPT a lot for writing short functions in a variety of programming languages. All of these tools, at least for me, eliminate most of the tedium in my programming projects.
You can do GPT-4 powered coding chats in your terminal today with aider. It's not free in the sense that you need a gpt-4 api key, which openai charges you for. But it's a pretty great way to collaborate with AI on code.
Do you know how it deals with source files that are larger than the context window?
Right now aider doesn't even try and deal with files larger than the context window. For gpt-4, that's 8k tokens or about 32kbytes. Which is pretty reasonable.
According to the data in [1], the average source file in github is less than 14KB. It's worth noting that they explicitly discarded small files with less than 10 lines. So the true average is probably much lower.
Regardless, I haven't found the gpt-4 context window issue to be problematic in practice yet. You do need to be careful about how many files you "add to the chat" at once. But that's not hard. For sure there are cases where you would need to refactor a large file before you could use it with aider.
I am certainly interested added context window management features to aider. I have previously explored a bunch of approaches for this with gpt-3.5-turbo. I shared some notes about these experiments previously on HN [2].
[1] https://hoffa.medium.com/400-000-github-repositories-1-billi...
Seems like that could work even better for code, where gpt could compactly summarize what functions do without memorizing all the code.
In the [2] link I included above, I discuss the approach you suggest of summarizing the meat of most code. It didn't work well with gpt-3.5-turbo, but I do plan to revisit it again with gpt-4 at some point.
Again, I haven't actually exhausted the context window often in my own use of aider. So it hasn't been a problem in practice. But no doubt it will be an issue for certain code bases and source files. I am quite interested in finding an effective solution, and plan to put in more effort here in the future.
> Complex Multi-file Change with Debugging: A complex code change involving multiple source files and debugging.[0]
Ya, that one is some real work I was doing on the tool itself by using the tool. I use aider extensively while I code now. Aider has authored 150+ commits in that repo in the last week or so.
Many of the other transcripts are more "toy problems" to help convey the experience of working with aider to folks unfamiliar with the tool.
Edit:
My prediction is essentially that:
You give it a set of requirements for an api, with edge conditions written in plain English. Test cases are provided. And the vanilla api is generated. We aren’t that far from this. It’s going to happen. Programmers may go in and tweak the generated code. When the requirements change, you pass in the old code as context.
Complex software will require human coding. But most programmers are in denial about how complex the code they’re writing actually is.
Requimemts gathering is notoriously tricky, but you won’t need engineers for it. Welcome to the age of the PM
Think of it like hedge fund traders vs stay-at-home retail traders.
The closer you are to the core AI stack, the more you will get paid - kind of like hedgefund traders.
If you are just copy pasting ChatGPT mostly, then you will get disintermediated at some point.
It will be slowed by bureaucracy.
Example: To be SOC 2 compliant, you need to have change management controls in place. How do humans manage change management if an AI is doing everything for the humans? Would humans still be doing code reviews? (AI might be so great where code reviews are obsolete, but compliance / regulations may still require humans in the loop)
There's also a -huge- subset of software that is extremely mission critical or heavily regulated by compliance where all current compliance frameworks assume there's a human in the loop. Ripping the human out of the loop would require re-writing the regulations / standards (which I guess AI could help with) but I think the change will come slower than you're predicting.
Change won't be slower because of any technological barrier necessarily (this is debatable), but because it will absolutely take a very long time for humans to fully trust AI and its output.
The era we're in currently feels like where Tesla was a few years ago. Really cool concepts and proof that self-driving "works", but there are so many edge cases which limit its complete roll-out and makes the original promise of "everything will be self driving in 5 years" seemingly achievable, but 5 years later it hasn't been achieved, and remains simply an assistive technology with many limitations.
Orgs that are agile and unbothered by middle managers will thrive by adopting change, while the slow ones will lose competitive edge.
Eventually even the slow ones will have to adopt change.
And I’m sitting around here architecting code, rewriting graphs, and gathering input from stakeholders.
Only time will tell (and I think it will tell really really soon).
Anybody making a prediction about the future might be right or might be wrong, and we won't know until it happens. This is just a snarky way of saying "nah dude you're wrong"
Only time will tell :)
Natural language is translated into Python which is interpreted as C which was compiled into bytecode which is...
We still need experts at every layer.
This is absolutely something that already happens with fully human developers, but it seems likely to be much more frequent and not caught as soon with AI assistance.
This also seems like a failure mode that could go pathologically wrong on a regular basis for TDD types.
I feel like companies will start using watermarking to avoid the AI dumping GPL into their codebase.
Nah. It will just be tons and tons of trivial, straight forward code, calling other trivial code, that calls more trivial code.
Yes, some of it will be created manually, but if you go with "the IA created it" you will get it right 99% of the time. And obviously, the more code, the more it will fail; but people always try to fix this with more code.
Remember the times of Enterprise Java?
Those people have nowhere to go to match what they're earning now. Maybe support review roles (which won't pay particularly well), where they approve decisions by the AI that are held up for human approval.
The bottom half of software developers won't have AI assistants. The AI will have them as human assistants (required by corporations for control/safety/oversight purposes).
The ~50%-25% bracket will build software using AI tools and will rarely write the actual code.
In the top ~25% bracket (in terms of skill) you'll have software developers that are still paid very well and they'll directly write code, although not always. That group will be the only one remaining that is paid like today's software developers get paid.
Software developer in the future will most commonly mean someone who builds software via AI tools (with the AI writing nearly all of the actual code). Human software developers will be glorified prompt wizards (with required degrees; it won't be a great job).
For the median software developer, the peak has already been reached (in terms of pay and job security).
Emerging market software developers will be hammered before their economies can fully benefit from the relatively high pay of the industry (from off-shoring work from big tech).
The golden run is over for the bottom 3/4 of software developers. Prepare for it. Get used to it. In the developed world the ladder up and out of the middle class via software development is going to go away (and quickly).
To regularly write code in the future you'll have to be damn good. Good enough, and knowledgeable enough, to be better at what you're doing than an AI with a handler (human assistant). You'll be writing the AI systems that govern everything and it'll be increasingly regulated, with more government licensing (plausibly formal AI engineer licensing and actual accountability, because the risks will go way up).
This reminds me of self driving car predictions. It also ignores that "building" software encompasses many different tasks.
What you're referring to is one of the AI systems that I mention that the top ~25% will still write code for directly. It's a governing AI system.
Most software development doesn't involve tasks that can very easily kill people with N thousand pounds of fast moving metal.
Most software development is trivial by comparison in terms of challenge/complexity. It'll be wiped out accordingly. Why would you need anything more than a human handler to sign off on AI development as it goes down a path? It'll be able to build drastically faster than a median developer can and it can do it without getting tired (its productivity won't implode after 4-6 hours). All you'll need are some human handlers to approve key decisions during the process of development (to ensure you get to the end product that is desired).
As with most technology job predictions, it won't happen the way most predict. Yes what defines a developer/swe will change, but our history has shown us that we are likely to see an increase of jobs in technology for the long term. I remain optimistic that most people will find new roles as they emerge.
Being a prompt wizard won't pay as well as directly writing code for the AI systems. They're different layers. There won't be more people directly developing software than there are today in the US market, there may be more overall jobs in and around the process however (ie the tech industry will continue to expand to more people in terms of employment; benefits will weaken, median pay will fall).
Most software developers will be prompt wizards, with required degrees that say they know how to be effective prompt wizards. Then there will be a lot of supporting roles, oversight roles.
More jobs in the industry, lower skill levels, less pay.
We simultaneously won't need and won't want the majority of software developers that exist today (the sheer number of them), writing code in the future. That would be a bad outcome.
They're going to end up more valuable as prompt wizards and checkpoint decision makers, because the AI will be drastically better at writing code (in all respects) than they could ever be. And they're going to get paid less because more people will be able to do it, software development will become a lot less intimidating as a field. It'll be more mass market as a field, akin to being a nurse (4.2 million registered nurses in the US).
How is this not a risky prediction?
1. Are you talking about AI (i.e. AGI) or LLMs? If you think we will have AGI in 10 - 15 years maybe you are right, but AGI is always just around the corner.
2. LLMs don't seem to be the magic multiplying force people are insinuating. GPT-3 has been around for almost 3 years, and while great (I was an early adopter) it's just another tool for me.
Over the past 10 - 15 years software has gotten easier to develop. And make software easier to develop has just gone to serving greater and greater demand. The invention of C didn't reduce the number of engineers because it was simpler than ASM. The invention of Python didn't reduce the number of Java engineers. Every year CPUs get faster and faster (at an exponential rate almost), and yet software somehow manages to get slower and slower. No other tool in the short history of software development ever did anything close to "wipe out at least half of all software developers" despite newer tools because easier and easier.
I'm talking about AI - made up of multiple modules that focus on different aspects - that can comprehensively build software products, with nothing more than human sign-offs to get from start to finish. Nothing even remotely close to the difficulty of building AGI (which I don't think will happen in the next 30 years at least).
In this scenario the AI has human assistants that sign off on decisions before the AI can proceed further, before it can continue writing more code. The human developers that build this system will include a judgment for checkpoints, for the AI to judge when it thinks it's necessary to ask permission from a human to proceed (hey human, do you want me to go this way or that way?). The AI will occasionally present the human prompt clicker to choose from multiple viable development paths (for example overnight it'll build three different approaches to solving a problem after you leave work at 5pm; when you come in in the morning your task will be to pick the one you think is best, and the AI will proceed from there). I think a pattern of: AI development -> checkpoint approval by human -> continued AI development, will be the superior way to utilize AI to build software (rather than letting it get too far down the wrong path by trying to let it build without oversight most or all of the way).
The human decision making process at checkpoints will become by far the slowest part of building software. The AI will spend most of its time waiting for someone to sign off on a path.
I'm not sure if I've ever seen a situation where all the code for a company gets written and then the engineers just pack up and leave. Typically the more code you write the more code you plan to write.
The only prediction I feel is easy to make is that nobody can accurately predict the effects of an emerging technology 10-15 years down the line. That is a long time, and prognosticators almost always get it wrong. Coding will absolutely change, but I'm very skeptical that anyone can predict how with any sort of specificity.
Sometimes I wonder where some of the posters here work or if I'm working in a dystopia.
>You give it a set of requirements for an api with edge conditions written in plain English
This part is the job! If my job was 100% writing logic, it would be infinitely easier. Defining the requirements, evaluating the tradeoffs, discovering the edge conditions is where the bulk of my time goes. The only time someone did this for me was when I was a junior developer. Maybe I'm overestimating things, but I find it hard to believe that most engineers pulling huge salaries are just shuffling around fields on JSON API. Do you really need AI to expose a CRUD interface to Postgres?
Edit:
This idea that LLMs will replace engineers (or lawyers, or any traditionally "skilled" field) is hype. It's the same mistake that the customer makes when he's shocked that he gets a bill for $10,000 for replacing a screw in his car engine. You are conflating the actual physical labour requirements of the job (sitting down and coding) with the actual knowledge value that is being used when you do the job.
For example, take a look at Redis. Redis is a great codebase, especially for those that want to learn C. It's simple - there are exceedingly few mind-bending, hardcore, engineering algorithms in Redis. Antirez is an amazing software engineer, but Redis is not the fastest database, nor is it most durable. But what you see is the meticulous application of understanding engineering tradeoffs; there are things Redis does incredibly well and things that it doesn't that is made easier by the overall architecture of the code. How would you even begin to prompt this to an LLM? Again the code isn't complex, but the engineering is, and the act of turning those ideas and communicating them either to an LLM or to a C compiler, is engineering.
No one comes home from a long day and says "Honey, I'm tired I spent all day typing JSON schemas and function signatures".
"Requirement gathering" is almost always influenced by technical capabilities. All but the simplest projects, require back and forth with guestimates about the direction and approach.
Going to guess you're both young and working at a tech company as opposed to a big company that also does some tech things.
Prior to the rise of Agile and the decrease in waterfall-style development the role of Business Analyst was very popular. It still is at many companies that do need to do waterfall development, but less so these days. The Business Analyst (BA) role is semi-technical and requires some subject matter expertise, but generally doesn't do much actual writing of code. Instead it's focused on requirements gathering, wireframe creation, test case creation & running, bug logging, and status reporting. I know you're probably saying "that's at least half of my job!" and you're right, but now go and look at the difference in pay between a BA and a developer. It turns out that if you remove the "actually writes code" part of the job, it's not worth nearly as much and I think that's what the comment above is trying to express.
I think Product Managers often take on a lot of these BA responsibilities, they've got the domain knowledge, but there's another step of translation from "build an X that does Y" to "build an X, in A, that does Y, but not Z, in B time, integrating with systems I, J, K". I think that takes an engineering perspective.
I've not worked with anyone titled Business Analyst that does the role you've described, but working with PMs, BI people, and others, there's a large gulf between requirements and tests that they would specify at their level, and requirements and tests that a programmer could work from.
Already we have prompt engineering. And also we could have said the same thing for people who write in assembly language, and each higher level language and also programs that are designed to be low code . I see that this is a new type of spreadsheet but scaling these up or using these programs aren’t going to make coders / software engineers / data scientists go away in fact historically we have seen the opposite.
In an adjacent discussion about ChatGPT, we will look forward to when nobody learns to write in plain English any more.
I think it's just a matter of time till all software becomes complex, e.g a codebase worked on by a few dozens people for 10 years will have a non trivial level of complexity. Not disagreeing or anything, the machines might be able to do this well, we'll see. It will have to improve by a lot, like almost no hallucinations. But there could be many shades in between. Instead of completely replacing developers in 5 years which I personally find unlikely, it can replace 50% of them.
"most programmers are in denial about how complex the code they’re writing actually is"
I can taste the salt
Forget about being employed producing software in a world where these are the norms, I don't think I'd want to be a software _user_.
I'd love for programming to be higher level and accessible, but I wish the process were:
- write a _specification_ that describes functionality, invariants, etc
- generate signatures compatible with that specification
- interactively seek implementation-relevant information from the designer, e.g.
- "I see Users can have an unbounded collection of Bars, which can each reference an unbounded number of Wugs. How many Bars do you suppose a typical User would have? How many Wugs per Bar and how many Bars will reference each Wug? How large is a Wug typically?"
- "Which of these DB queries do you expect to be run most/least frequently?"
- "This API method allows for pagination. How often do you expect a user to page beyond k?"
- generate code aligning with signatures, _and which cause tests to pass_ (i.e. program synthesis from examples)
- static analysis / abstract interpretation / model checking to confirm that some invariants are respected
- explicitly reporting which invariants or properties were not able to be confirmed automatically which a human may need to confirm via an ad-hoc analysis
Software is one of the domains where we can actually do a lot of sophisticated automated reasoning, but the current trend of ML code generation as text completion ignores basically all of that, and this seems like a giant waste.
For example parsing files, pulling data from a table and dumping it in a CSV, monitor performance counters or network activity, backing up and copying files etc.
I don't consider this part of my job, but it's stuff I need to do often and I'm glad it can now be delegated to a bot.
It also means new hires and co-ops have more interesting things to do.
Today for the first time I used code from ChatGPT in a working solution.
Nothing too advanced, a small routine to interpolate a vector.
It took me literary a minute to write the prompt and another maybe 5-10 to adapt it and test it.
I see myself using ChatGPT more and more for this type of things, automation scripts and very specific, simple routines in production code.
However this is not 5% of the stuff I do, for the other 95% there's simply no AI advanced enough to do it and there won't be any for several years or decades or maybe ever.
Now those working purely in CRUD applications from functional specs given my others, better to start switching to something that demands design skills and process know-how.
Am I missing any?
The UX: Click a button, type in a prompt, click another button. The speed just to generate 2-3 lines of code.
And AI generated code with no basic error handling.
Clever use of intentional UX friction IMO. It does cost money to run these models.
Certainly google should monetize; but not off removing artificially bad UX.
Make fun of my stupid blog post all you want, but my eng partners are awesome!
Source: currently a Colab pro subscriber based in the U.S., but don't see the AI features in my Colab notebook
"Google Colab will soon introduce..."
"Access to these features will roll out gradually in the coming months, starting with our paid subscribers in the U.S. and then expanding into the free-of-charge tier. We'll also expand into other geographies over time. Please follow @googlecolab on Twitter for announcements about new releases and launches."
Also, I'm sure they have ways around it, but I'd imagine colabs are a poor source of "good" code to use for further model training, both because of the kind of code you write in notebooks and the demographic that would make up colab users. It sort of fits with the idea that autocomplete might be good at writing short functions that do some specific thing, but not much help actually writing a full program.
Does it provide any attribution?
As more people use automated programmaking, most of the code will be low level spaghetti in procedural languages, because it is easier for the model to reason with it (without needing huge buffers to know about abstractions). We will see a lot of generated Javascript, PHP, even a return of Win32. Who cares if they are tedious for humans to read and fix, now the AI can do all kinds of advanced search/replace in the code. And maybe at some point they 'll be generating machine code directly
And strictly I use Pycharm from the IntelliJ suite.
If you mount the remote FS locally (I use NFS) you can even stop writing locally and continue on your server's Jupyter web interface.
(sorry frustrated bard-non-user in the EU speaking ...)
blame the lawyers and TPU capacity
"OK, team. With our founding cash cow threatened, we're at a critical juncture that could preserve or break us. If we're to survive, we must make the most of every factor of success at our disposal. This includes branding, and that's why we hire you, some of the best in the world at it. First major disruptive tech up for branding is a code LLM. Brainstorm now."
"Codey McCodeface!"
"Looks like we can go home early."
Ugh, cringe. Just say this is a panic swing at an existential threat (OpenAI) and you're trying to commiditize them.