GitHub Copilot loses an average of $20 per user per month
thurrott.com
thurrott.com
I resent some LLM implementations on principle, but decided to give these code helpers a try. What I found was they’re reasonably bad, and I kept telling them the solution doesn’t work, only to be presented with a little tweak.
So I don’t see the point of outsourcing my thinking, I’d rather remain intelligent and do the search/try/tweak on my own, instead of pretending a half-assed LLM is genius.
That doesn’t mean they don’t have good use cases, or aren’t an improvement on previous tech. But we definitely should stop calling them mind-blowing. Jaron Lanier had long ago predicted we’d willingly downplay human intelligence to pretend AI was… I.
You seem to be doing what GP is pointing out.
GP's claim is that the relative improvement itself is mind-blowing, not that the tech is mind-blowing in an absolute sense.
I tend to agree: much of the detraction hangs on current-state rather than a probable potential-state informed by recent relative advancements.
In other words, many proponents are chuffed because of that potential; not because actuality. Likewise, skeptics are reserved because of the actuality, and not because of potential, for whatever reason (there are at least a few main ones, I think).
I feel like people are taking Moore's Law, which is definitely a real thing, and thinking that everything else is going to advance like semiconductors did, and I just am not seeing it in any other field. It's not true in software development (where gains, such as they are, are more linear than exponential) its definitely not true in rockets or civil engineering or steel or anything like that. So I am afraid a whole lot of people are expecting Moore's Law type improvements in AI, when really AI advances more like punctuated equilibrium: a sudden dramatic improvement, then a long period of consolidation and stasis, then another sudden dramatic improvement, often in a totally different unpredictable area.
But I've just been keeping tabs on AI since the hot way to do it was Expert Systems back in the 1990's, and I'm aware of its history since Norbert Weiner wrote Cybernetics back in 1947, and this seems to be a repeating pattern: a single major breakthrough (in this case, honestly, the combination of large quantities of data with NN's- with driving and natural language being the two easiest to get, and so the most prominent examples) followed by a lengthy fallow period where not much appreciable progress happens, then another breakthrough, often orthogonal to where earlier breakthroughs happened.
If that’s what you want to call it.
I see hypemen overpromising and product underdelivering. And when pressed about specifics, attempts to drown queries in jargon or an ass-covering retreat to treating it like it’s just a tech demo not intended to be used for anything ever.
And it seems likely that with an order of magnitude better hardware it might be good at some things that it seems really bad at now. So yes, it's a tech demo for a lot of things that aren't quite ready, and it's also very useful as it is.
i work at a large consulting firm everyone knows and am seeing this first hand. I'm not on the bandwagon until i see real money in the bank from large AI projects succeeding. It's not happening yet. I not a naysayer but am still very skeptical.
Edit: I haven't even started on the social and political impact this has either.
Yes if you tell it to write an entire program it will get it wrong and you'll spend some time verifying things. But that's not a sane way to use it. As a very clever auto-complete it's fantastic. It's also pretty great at getting past "blank page syndrome". Even if what it spits out is wrong it's still helpful to get you started.
Do you really think it's fine blowing 400 watts because you can't be arsed to think or do not have the creative intelligence to get over the blank page syndrome and have to lean on a crutch?
Yes, I think it is absolutely 100% fine.
From my past history:
"Is "192.168.1.4" included in the subnet "192.168.0.0/16"?" -> No
"Check whether a widget overflows in flutter" -> returns a function that cannot be made to works even with a lot of massaging (uses stuff that does not exist)
"Write a parser for this multiline format in C++ (describe format)" -> parser only read first line
Admittedly a trick one: "Can you give me a C++ function to merge 2 uint32_t and one uint16_t into a unique uint64_t?" -> happily gives an answer
Sometimes it is salvageable, and sometimes it can provide ways I did not consider to solve a problem (though the proposed solution is usually broken), but usually I would have been faster to do it myself than to try to fix whatever it gives me.
I have basically given up on it, except for some generic "how would you solve problem X?", and when I see people talking about it, it feels like a totally different world.
> "Is "192.168.1.4" included in the subnet "192.168.0.0/16"?" -> No
ChatGPT is not good at numbers or complex maths like this.
> Check whether a widget overflows in flutter
I mean this would be closed as unclear on StackOverflow, but again, this is basically asking ChatGPT to write an entire function. It can do a good stab but it's not going to get it correct.
Copilot isn't for that sort of thing. Let me give you a more realistic autocomplete example from my code:
std::fs::write(&sv_path, sv).expect("error writing top.sv");
std::fs::write(&rs_path, rs).expect("error writing top.rs");
std::fs::write(&cargo_toml_path, cargo_toml).expect("error writing Cargo.toml");
std::fs::write(&cpp_path
However even for "do it all for me" queries it can still be useful. For example I asked:
> I have a C++ process paused in a debugger (lldb). It is consuming a lot of memory. Is there any way I can see what is using the memory? E.g. a heap profiler that can attach to the process?
It got it wrong and told me to run the process under Valgrind. I said:
> Those instructions aren't for attaching to an existing process.
And it told me about the Heaptrack project with can do exactly what I want. I can see why search engines are integrating this asap. Even though it hallucinates, it does that infrequently enough that it's still really useful. I mean it hallucinates frequently but even so it is very useful.
Perhaps a better example is this:
> Write a C++ program to run a child process and send data to its stdin and stdout while it is running.
After 2 more prompts I got it to output the code below. This code is not correct. But the fixes to make it work are relatively minor, and EASILY less work than writing it all from scratch.
In a few years we will look on people that don't use Copilot (or similar) like people that don't use IDEs. They're hurting their own productivity out of principal.
-------
#include <iostream>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <vector>
class ChildProcess {
public:
ChildProcess() : pid(-1), pipefd{-1, -1} {}
~ChildProcess() {
if (pid != -1) {
close(pipefd[1]); // Close write end of the pipe
waitpid(pid, nullptr, 0);
}
}
bool create(const std::string& command, const std::vector<std::string>& arguments) {
if (pipe(pipefd) == -1) {
std::cerr << "Failed to create pipe." << std::endl;
return false;
}
pid = fork();
if (pid < 0) {
std::cerr << "Failed to fork process." << std::endl;
return false;
} else if (pid == 0) {
// Child process
close(pipefd[0]); // Close unused read end of the pipe
// Redirect stdin and stdout to the pipe
if (dup2(pipefd[1], STDIN_FILENO) == -1) {
std::cerr << "Failed to redirect stdin." << std::endl;
return false;
}
if (dup2(pipefd[1], STDOUT_FILENO) == -1) {
std::cerr << "Failed to redirect stdout." << std::endl;
return false;
}
// Convert arguments to a C-style array
std::vector<char*> args;
args.reserve(arguments.size() + 2);
args.push_back(const_cast<char*>(command.c_str()));
for (const std::string& arg : arguments) {
args.push_back(const_cast<char*>(arg.c_str()));
}
args.push_back(nullptr);
// Execute the child process
execvp(command.c_str(), args.data());
// execvp() only returns if there's an error
std::cerr << "Failed to execute child process." << std::endl;
return false;
} else {
// Parent process
close(pipefd[1]); // Close unused write end of the pipe
}
return true;
}
void write(const std::string& data) {
if (pid != -1) {
::write(pipefd[1], data.c_str(), data.size());
}
}
std::string read(size_t numBytes) {
std::string output;
if (pid != -1) {
char buffer[numBytes + 1];
ssize_t bytesRead = ::read(pipefd[0], buffer, numBytes);
if (bytesRead > 0) {
buffer[bytesRead] = '\0';
output = buffer;
}
}
return output;
}
std::string readLine() {
std::string output;
if (pid != -1) {
char buffer;
ssize_t bytesRead;
while ((bytesRead = ::read(pipefd[0], &buffer, 1)) > 0) {
output.push_back(buffer);
if (buffer == '\n') {
break;
}
}
}
return output;
}
private:
pid_t pid;
int pipefd[2];
};
int main() {
ChildProcess childProcess;
std::vector<std::string> arguments = {"arg1", "arg2"};
if (childProcess.create("child_process", arguments)) {
childProcess.write("Hello, child process!");
std::string output = childProcess.read(1024);
std::cout << "Child process output: " << output << std::endl;
std::string line = childProcess.readLine();
std::cout << "Child process line: " << line << std::endl;
}
return 0;
}Aren't you overgeneralising a bit? Not even I would say that CNNs for image classification, or Deep-RL for board game-playing are a "continual disappointment" and they certainly predate LLMs. Are you talking about NLP? Even Neural Turing Machines were quite capable in language pairs with large parallel corpora (and similar linguistic structure).
Basically, what do you mean by "continual disappointment"? What I'm aware of is an incessant hype crescendo that crashing over everything like a relentless wave.
RLHF trained GPT3 and then DALLE/Midjourney/Stable Diffusion changed all that. Suddenly AI not only got good, but the field broke loose of the inane and insane obsession with pseudo-safety that had been holding it back. Now the rest of us can use it without dropping $5M on a GPU cluster and hiring a dozen researchers first. AI is no longer a disappointment.
One thing I always find funny is the general expectation that machine learning models are both incredibly generalised and designed based on the way biological systems work, but should also be 100% perfect and never be wrong just like a machine and NOT like a biological system, those things are mutually exclusive; even the best, smartest most physically capable humans will still sometimes spill their coffee, yet we expect coffee-bot 2024 not to do this.
Certainly machines can be much better at a task than humans are, but if that tasks requires generalisation then it's still gonna fuck up from time to time.
Machine learning programs may think a dog is actually a cat sometimes, but afaik they ain't ever called their teacher "Mum" yet.
For all of the criticism of Github Copilot, for a lot of developers (but not all), the value Github Copilot is incredibly high, much more than $20/month.
The current rock bottom pricing is low compared to the value provided for those users.
As such there is a big opportunity to multiply the price here being charged. Probably an increase between 3x to 5x.
The excuse is because they are losing money, but the underlying reason is that the value it is providing is so high in terms of developer productivity.
For score keeping, let's define "soon". Does "by the end of 2025" sound reasonable?
LLMs are really good at fooling us into thinking that code comments make sense.
That's why I prefer if devs don't use copilot, but instead cherry pick good output from ChatGPT.
Anyway, I have to review all of his code, and I've been doing this the whole time he worked for us. But, starting around the beginning of the year, I started to notice problem after problem coming through his merge requests (MRs). At first I gave the guy the benefit of the doubt, like I said this was unusual for him. He always needed moderate coaching, but the mistake ratio was way higher than usual.
Anyway, I did some basic coaching and whatever. I talked to him about the increase in errors and he said he was stressed with moving to a new house at the time. It made sense, so I decided not to even record the first performance review with HR.
But the problems kept coming and getting worse. It was getting to the point that he had so many problems, I wouldn't even address them all. I would only kick back the top dozen or so. I was noticing that his code had entirely different styling and voice across methods like it wasn't written by him. I also started to see weird lines that were trying to catch unusual edge cases that would never even apply in our scenario. There was one time where there was this insanely complex filter that fed into an insane regex that I couldn't decipher. I asked him about what it is even trying to do and he had no clue, he couldn't even explain the purpose of the line, let alone how it worked. I pushed him on where it came from and he said "StackOverflow". But I reversed searched it and couldn't find it.
We ended up doing performance reviews almost weekly for a while, and now I was formally writing them up. I told him that he was more of a burden on the company at this point. He was offering a negative value. I was getting stressed out because I was spending so long fixing his code and coaching him, that I was working longer hours because of him. Furthermore, I could have fired him and not even replaced him, just taken over his job myself and it would have been less work than this back and forth of trying to repair his work.
Eventually I did fire him. After he left and I talked to the team about him being let go, one of his friends told me that this guy had been using ChatGPT and Copilot for all of his work. It was a secret because our company doesn't allow AI tools in our codebase right now because of compliance reasons. Which is why he would rather say his mistakes were "copypasta from StackOverflow" as he always put it, than admit it was copilot.
That's not to say that CoPilot is bad. But its not ready to replace developers just yet. Even junior developers still need to wrangle and check the work coming out of the AI. And yes, people do notice. Even after this employee polishing the work coming out of the AI model, it still wouldn't work in our codebase.
And to the original point. Yes, these AI tools really are just passing increased burden onto the experienced developers. The juniors are using these tools as crutches to speed things up so they can watch more YouTube videos. But the code gatekeepers who have to defend the application are the ones with increased burden and workloads of fixing the mistakes the AI is generating. I've been saying for a long time that AI will simply separate the good from the bad in software development. For the past 15 years you could make $150k or more a year as a developer that is only capable of producing sub-par code. But those days are gone. The sub-par code is being produced by AI now. So you need to be at least mediocre now.
I really doubt this.
IMO, the jury is still out deciding if the value is above zero for enough people to matter.
Sounds to me like the verdict on its value is in already.
As a data point, this matches my personal observations. But reducing the time spent on that bolierplate plus not needing to search where the "copy" portion comes from, may justify $30/month (and probably much more than that). My 2c.
Any new product that gained +1M paying subscribers (Github Copilot) in its first year is a success. You cannot like it, that is allowed. And it may not help you, but +1M subscribers is a lot of people.
They are definitely here to stay until they are superseded by even better technology or it gets sued into oblivion.
Not a native speaker, but to me this sounds much more ambiguous than up to 50% of code is produced by Copilot.
Also, how different from previous solutions is this actually? I use autocomplete and code snippets extensively. Never measured it , but I wouldn't be surprised if my IDE had generated more source code than I myself typed over the last 10 or so years.
It doesn't say anything about its value.
The value, for me, is extremely high.
My teammates feel the same. Our shared opinion/experience is that ChatGPT 4 is better than Copilot in general but Copilot shines in-editor because it's aware of your project. So we use both in tandem. They mostly use Chat GPT and I split about 50%/50%. (Note: I'm using the Copilot X beta which I believe uses GPT-4)
People say they're "only good for boilerplate code" but well, that's the vast majority of what anybody is writing IMO.
If I need to traverse a tree or list or something, I'm letting AI write that code. Could I write it myself faster? No, and it's going to have an off-by-one error some non-zero portion of the time if I write it. I also find it's superior to e.g. memorizing all 10,000 CSS properties along with all the classes that pertain to Boostrap or Tailwind or whatever.
I see the AI code assistant hate here and it just baffles me. It's so obviously useful to me, and I really can't imagine I'm that atypical.
Edit 1: AI help is especially pertinent if you are a "full stack coder" who is working on everything from database to frontend. Since frontends really multiplied in complexity about 15 years ago, I have not met a single "full stack engineer" who is truly fluent and expert in the entire db->app->frontend stack, because complexity and choice has proliferated at each of those levels.
Edit 2: While most of us are (hopefully) not literally writing tree or list traversals by hand in our actual daily programming lives, I hope my meaning is still clear -- I'm talking about that mundane sort of code, iterating over things, etc.
Many languages/companies have existing well understood solutions that _won't_ have errors. Maybe that is the disconnect? I can't remember the last non-interview time I had to write a non-trivial traversal.
Many languages/companies have existing well understood
solutions that _won't_ have errors.
In a literal sense it has been years since I wrote a tree or list traversal by hand and I would be very surprised and concerned to see a PR where somebody is doing it by hand rather than using a library.
But, I hope my meaning comes through despite that. I mean the sort of mundane "iterate through a thing, and do a thing with some of the things" sort of code that many/most of us are writing on a regular, hour-to-hour basis.
Maybe that is the disconnect?
I'm generally a "full stack" web developer (currently switching between Python and Ruby on the backend) and I don't mind admitting: front end crap changes fast enough that I can't possibly keep up with it. In my experience nobody is expert in the whole stack. Altogether it's just a really big surface area of Shit I Need To Know. AI is very welcome here for me.
Other coders might have a smaller surface area of shit they need to know, and they already know it inside and out, and therefore see no real value add from an AI buddy who is not correct and optimal 100% of the time.
Between better quantization, pruning, smaller models through better training/distilling/architecture, hardware price drops and purpose-built hardware, it will probably be much cheaper to run such a model in the future.
Prompting allows you to direct the model into a different way than the training data. If it were not so, LLMs would never solve problems that were not explicitly in their training set.
This is an issue with your prompting not the models.
If you tell someone "Do this thing" they'll just do it, LLM's too. And how it is done will probably be terrible.
If you ask someone "come up with 5 ways to accomplish this, compare and contrast between them, list pros and cons, think through best practices, maintainability, readability, security, performance and cost. Come to a final decision and then do it" their response will be different. So too will the LLM's. You can even have the LLM present its opinions and recommendation but pause and wait for your choice on the go-ahead.
Even if you do prompt it correctly, and it responded in the abstract with all these pros and cons, you cannot be reasonably sure that the code it also provided actually follows these best practices. It will just anytime mindlessly wander from "the best experts on the Internet are saying this" territory to "I just made this up" territory.
Compare that with googling a human-written stack overflow answer on the topic, there usually is some good soul who pointed out the inconsistency, if there is one.
Look, I like grinded meat. But the fact is, without a detailed analysis, it's hard to tell what's actually in it.
And?
As a Sr. Engineer, this is exactly how all the slop my Jr's and Mids send me looks. Copy pasted, Stack Overflow, and when I read through Jr code my mind is boggled. I want to shake them "did you even read your own code. do you even know what your code is doing? Why did you do this?" Forget style, best practices, etc,
In my opinion, LLM's produce code that is as good or better than most Jr engineers and as a Sr it is my responsibility to audit, review and test all code. As a Sr level engineer I spend 90% of my time judging/fixing/improving others code, and less than 10% of my time writing my own.
LLM is just another source, and unlike the Jr, I can quickly ask it why it did what it did or to refine it. You ask the Jr to iterate on the project and you won't hear from them until they mention a blocker at tomorrow's stand-up (or you just pair it out and spend 2 hours teaching them, while the LLM turns it around in 15 seconds. There is value in teaching of course, but we build quickly, too).
If it was a boolean, the limits you describe would certainly exclude some of the human programmers whose nonsense I have had to fix.
[0] and "generality", though "AGI when?" is a separate topic
The second issue I could work around but combined with the fact that it doesn't have context of my current codebase, my impression was that it would be a fantastic tool for someone going to school for CS but practically useless in a professional codebase.
If you understand how it works you can sometimes lay out your code in a way that makes it more likely to include the relevant examples to get the effect that you want.
There is no guidance or documentation for this at all!
I've been running a hack to help me see the prompts in order to better understand what's going on: https://gist.github.com/simonw/30fec0ac20244b4b921532b63b8b0...
In neovim w/ a medium-sized go project, I"m finding that it's really good at providing meaningful and at least directionally-accurate contextual suggestions. It seems that the plugin is providing a good bit of context, but maybe the vscode one doesn't?
I’m pretty sure it pulls context from at least the other open files in VS Code, if not the ones on disk. Hard to tell for sure.
It’s kind of like having an eager, well-read, but naive and over-caffeinated intern as a pair programmer.
Who's gonna pay for this as soon as the VC money runs out?
But it doesn’t matter, I’m they never would have gotten the valuations they did if it weren’t for the insane hype around the (free) ChatGPT offering. It more than paid for itself just in that regard.
I assume the best strategy is to shrink the models and tune them more.
1. GPT4 is on a different scale to CodeLlama
and
2. If open models do catch up, they will have moved on to GPT5 or GPT6 and the gap will have widened again
They are proving the market to generate demand to invest to vertically integrate, which then drives down costs while revenue remains flat or (hopefully) increases.
Eventually reality will force cost to the consumer towards cost of serving. Cost of serving LLMs will decline alongside this, so maybe the magic number stays at $20
GitHub Copilot loses an average of $20 a month per user (https://news.ycombinator.com/item?id=37821756) (45 points | 16 comments)
I’m also using it with intellij instead of vscode, so for all I know I could be using an old version still.
If I'm still unsure how I might approach a certain problem, or if it isn't immediately clear to me how I'd write the function I want, I might type in a prompt to ChatGPT and see what it comes up with. But it would really slow down my workflow if I had to prompt ChatGPT for every mundane function I plan to write.
[1]: https://github.blog/2023-07-28-smarter-more-efficient-coding...
[2]: https://github.com/orgs/community/discussions/56975#discussi...
By default, ChatGPT can only give a generic answer. Do you just paste the entire file in there?
For example let’s say I want to write a function foo() that calls functions bar() and baz() defined in the same file and uses a library Y that I already imported. If I just write the name of the function, Copilot will often autocomplete a reasonable body for me. If I wanted to use ChatGPT then I would have to first tell it about foo() and bar() and the dependency on Y, and by the time I’ve finished telling it about all of that I could have written the function by hand twice over.
a) They're much slower on my 6GB laptop GPU
b) The seem to not be as good, functionally
b) I can't make use of larger models
I haven't done more than just some experimentation but I can see how this would make sense to put on a server.
I guess we’ll see in a year or two. This must be on everyone’s radar now, Apple won’t be the odd man out.
Lisa Su was interviewed at Code recently, and the AI discussion focused on competing against NVIDIA in big datacenters, but I hope they're also thinking about client-side stuff like that.
For whom? For those selling them it absolutely does, because that's how they get to charge a huge ongoing markup on hardware and electricity costs. That's why everything is a service now.
I’d be surprised to see the models go the ground breaking commercial AIs shipping but then I’m surprised at how much we’ve gotten open source anyway.
I know they have github, but tiny scripting glue tasks like copilot excels at aren't the best represented on GitHub.
The main functionality I get out of copilot in vscode is autocomplete on steroids, so it's getting all code as I write it. I don't use the question based generator most of the time.
All in all it saves me a little bit of typing a couple of times a week.
I'm often writing legal rules in code and from the name of my function, it predicts the way the law is written. It's pretty incredible how well it works. Clearly it's just stealing someone else's work from github, but the autocomplete is very awesome.
For code completions it uses an improved version of Codex[1], and for chat beta (part of Copilot X), it uses 3.5-Turbo[2].
However, they're claiming to have "early adoption of OpenAI's GPT-4" for Copilot X on their marketing page[3], which is confusing/deceptive. To be fair, it's still in beta, but they should state outright what model you currently get if you sign up for it.
[1]: https://github.blog/2023-07-28-smarter-more-efficient-coding...
[2]: https://github.com/orgs/community/discussions/56975#discussi...
That's not too much if it's adding a lot of value but it's also definitely a lot of money. It would be the most expensive SaaS my team pays for (and we have a lot of subscriptions).
Or create a pricing model where the monthly sub covers a certain amount of tokens and then beyond that is pay-as-you-go. Sucks for users, but a scalable model.
Before LLM, it's perfectly possible to spin up a SaaS on 5$ Digital Ocean VM and charge $4.99 per seat monthly. If you're using low overhead techs like Go and SQLite you might get away pretty far with a decent user base.
But LLM is inheirently costly compared to those traditional apps. No matter if you're calling OpenAI or DIY your own GPU cluster it's gonna be way more expensive. Spin your own GPU might ended to be more expensive because utilization problems and upfront costs.
The subscription model was kind of the silver-bullet for SaaS but it's probably not going to work well in the AI era.
OpenAI, Elevenlabs, Runway, and Midjourney: they have subscription model but the quota is strict and tight. The "unlimited" plan is simply pay-as-you-go.
Early wave of LLM products with unlimited subscription models like Github Copilot and Notion AI are probably pricing way too low. $7 or $10 is way too low to support heavy usage.
But charging $50 might scare most user away because it exceeded people's expectation for SaaS. And probably still end up losing money. And hobby users may ended up paying too much for the core users - that will lead us back to sophisticated pricing tiers like Elevenlabs and Runway.
Are there alternatives? I dunno. Maybe implement bring-your-own-key properly? Like OAuth but for LLMs? It's definitely interesting to see how things will turn out eventually.
Right now, a GPU is an NPU. Does anyone how an NPU would differ from a graphics card?
But NPUs (like Googles NPU) are systolic arrays designed for 16x16 or even 256x256 float16 or even int8 matrix multiplications instead.
-------
NVidia builds larger matrix multiplications out of the 4x4 float16 base that a SM is designed for. After all, a 8x8 matrix multiplication is just four of the smaller 4x4 matrix multiplications.
Isn't it 8 of them? That's why you can't go the other way around and use your 8x8 multiplier to efficiently multiply 4x4s in parallel.
probably intel is working on the same. I assume the next generation of mobile chips (laptop or phone) will all include special LLM processors.
Compare NVIDIA to Groq TSP (Tensor Streaming Processor)
This is how an AI chip should be designed, you write the neural net compiler first, then design the chip. These guys have combined a number of smart ideas in their product: synchronous operation makes memory and network operations simpler to plan and optimise; all software defined memory access, no caches; a simple set of primitive operations that can express every model architecture - so they can simply make it pytorch compatible, no need to write kernels, no need to have 100 versions of 2D conv, for all sizes and shapes.
https://groq.com/groq-tensor-streaming-processor-architectur...
Groq recently demonstrated 240T/s for LLaMA-2 70B. Probably world record, the next best I know is 40T/s at labs.perplexity.ai
It also supports voice, which is a fun way to code!
There are several ChatGPT-like UIs that you can self host and pay only for API and not for ChatGPT plus. For example, I'm using https://github.com/Yidadaa/ChatGPT-Next-Web. It would be nice to use Copilot in the same way.
for mere code completion there is this: https://github.com/FarisHijazi/PrivateGitHubCopilot
but I haven't tested it.
It’s become such an integral part of my workflow, that until Zed got Copilot integration I couldn’t switch to it.
Of course I’d prefer to continue paying $10 a month.
You do realize any data sent to MS/openai/anyone will be stored forever and probably be used against you in the (near) future?
However. It will eventually, chunk by chunk, upload the full source code of the app you are writing. Complete with all typos and mis-pastes. Like youtube blocks videos with just a hint of copyrighted music, they may be able to detect use of "patented" algos or something like that and block you/ send a lawyer/etc.
The full code that’s already uploaded via “git push”?
HN has never been a privacy-minded crowd. It's been a collection of random people, and depending on the threads of topic, you'll get more people speaking out than others. See the Dropbox post from 2007. Nary a privacy concern. HN isn't a single voice.
Why do you think it's a privacy-minded crowd? Because you probably react and respond to similar posts, and you see people with a like-mind there.
> How came that usually privacy-minded hn crowd uses run-by-someone-else models so cheerfully and carelessly?
Let's answer this question: they probably aren't doing it carelessly. Consider that Copilot is run by Microsoft, who owns GitHub and your code already. So, they already have the code. I'm not sending them something they don't already have.
Anyways, when I see something I didn't expect to see on HN, my initial thought isn't how it's wrong, but rather, what am I missing? And rather than post an antagonistic question, I instead try to research more to get a better understanding of what I'm missing.
Have you double-checked the terms of service? Are you absolutely certain that there's no risk of your copilot use impairing your intellectual property rights in the future?
Given what happened to Unity, how comfortable are you with becoming dependent on copilot and then having the rules change underneath you?
IMHO If you are a code monkey copilot makes you a 5x developer, 10x if you have 5+ years of experience on the business
Maybe selling tokens/search could be benefitial for copilot, like only pay for what i use will be an interesting approach.
If you need to constantly use chatgpt/copilot to do your job properly, maybe its time to learn something else because your job will be automated sooner or later, what will be more cheaper a 100/hr or the 20/month service.
I find it hard to believe there is a scenario where it actually helps you, but only provides $5 of value in a month.
What exact kind help is that? If it is able to provide $5 in help, you could likely scale that to $20.
Maybe if you are website designer... ohh m sorry i should say the modern way website designers call themselves today "frontend engineer", maybe if you are a designer it could help you with all the boilerplate your react/astro/svelte requires just to add a <table> with two columns.
I think that AI will be the biggest loss leader ever invented. We'll buy a robot one time for $10,000 that will perform $1 million of mental and physical labor over its lifetime. Except nobody will pay for that $990,000. There will just be this expectation of something for nothing, so the anticipated payment never comes. And it will happen so quickly, so completely, that we'll wonder how it was ever possible to pay people to do all this free stuff.
No, the GitHub Copilot model of charging for AI services isn't going to scale. People are going to open source AI and own it themselves and stop paying for goods and services. Why would they when the AI can provide everything they need? AI is the beginning of the end of money. The headline foretells the end of the era of artificial scarcity, because capitalism can't compete with free.
I think Github Copilot paired with ChatGPT fails the most in idiosyncratic code bases where you are doing maintenance or complex refactors. I think Github Copilot with ChatGPT shines in writing green field code in areas where you are will to use the best practices and the open sources libraries that Chat GPT recommends. Basically, if you align with where Chat GPT recommends to go, then it will help you a lot more than if you try to fight it all the way.
Seriously! I write a documentation comment like:
/// Given: blah
/// When: blah
/// Then: blah
I find that ChatGPT/Github Copilot knows certain types of patterns, libraries better than others. If you get it to suggest the path forward, and then do that path, it will be better at maintaining that code. Basically ChatGPT sort of has happy paths which you want to follow.
Or you could say it in another way: AI designed code / architecture is easier for an AI to maintain.
Due to hallucinations, beginners can get knee deep very quickly, but experts can see through this and use these tools will skill.
I don't write as much code as I used to, but I still write code daily and appreciate how they can turn what was a 10-60 minute task into 1 minute of prompting plus 30 seconds of generation/reading to make sure it is valid.
I am not sure why so many people have hard time getting value out of it.
Cut to few months later with OpenAI API release, so many code completers released in many useful form factors willing to go wherever I code even if its Android Studio or Xcode. The product has stagnated completely.
I do expect it to change significantly in the future though, might go back then.
or are they suffering similiar lose?