GitHub Copilot loses an average of $20 a month per user | Hacker News Reader

GitHub Copilot loses an average of $20 a month per user

wsj.com

51 pointsskilled22 comments

I have enterprise access to Github Copilot and it is surprisingly bad for code generation compared to the free version of ChatGPT (I mainly use it for generating simple functions in C). It always generates incomplete functions and there seems to be no way to increase the text generation window or even follow up. On top of it, I am only able to use it through VS Code and I couldn't ever figure how to use their chat interface plugin.

Overall it just feels like an incomplete product. I would've been happy with a chat interface on the web. If they're already losing money I can't believe how bad it would become once the enshittification begins.

On the flip side it works really well for me for both autocomplete and the chat interface. I mostly work in Python so perhaps the support is better, have not used it with C. It definitely does not feel complete but I do get tons of value out of it for the price.

The one major complaint I have is the filtering it does to prevent copy protected code. Sometimes it will just get stuck and there is no way to get it out of sending the same matched public code so I end up stuck with starting the conversation over.

My experience is that it completes boilerplate very well. I don't generally ask it to do anything super complex, but what it does I'm very happy with.

It's gotta run for some time to see your code and see what youre trying to do. For chat you just highlight code and ask it to do stuff with it, translate, transformstions, explain it etc.

You gotta run it at a loss to get market share before you can enshittify it.

Evergreen.

> Evergreening is any of various legal, business, and technological strategies by which producers (often pharmaceutical companies) extend the lifetime of their patents that are about to expire in order to retain revenues from them. Often the practice includes taking out new patents (for example over associated delivery systems or new pharmaceutical mixtures), or by buying out or frustrating competitors, for longer periods of time than would normally be permissible under the law. Robin Feldman, a law professor at UC Law SF and a leading researcher in intellectual property and patents, defines evergreening as "artificially extending the life of a patent or other exclusivity by obtaining additional protections to extend the monopoly period."

https://en.wikipedia.org/wiki/Evergreening

I have Copilot. It's okay I suppose, but nothing game-changing at least for my purposes. It's great at DRY stuff within the codebases in which I work.

I achieved similar results using Llama 2 running on my Mac Mini M2 Pro, which I did basically as a PoC, but again, nothing game changing for me. That's not to say that it might well be the killer app for many others out there. I'll give it another whirl I suppose.

I think the loss is simply because Copilot is currently free for all students, educators (?), and notable OSS devs.

The speed at which the models are progressing and new optimizations for reducing hardware requirements makes $20/mo per user not too bad. They'll be able to reduce those costs pretty soon in my opinion, and the users will continue to use the service for years to come.

The flip side to that is it's going to be easier to self-host a coding LLM. That's free for the user (if they already have a good enough GPU) and it gives much better flexibility.

That's a good point, the large corporation model vs smaller specialized finetuned models are vying for dominance right now, I'm interested to see which paradigm we end up with.

Won't be winner takes all. The people who use local models will have requirements or desires that lean them that way. The rest will use the easier to stand up large corp offerings.

Only way this changes is if there is a very large performance gap between the two. Small ones just won't matter.

https://archive.ph/3WDSv

I would be curious to see a study that compares users' impressions of the value of Copilot/ChatGPT for code generation, as a function of which IDE/editor they use, and also their typing speed.

I have a suspicion that the perception of value would be greater among slower typists who don't really unlock much from modal editing and rely a lot on their IDE to compensate.

For $30/month I could buy a GPU all for myself, they can serve a huge number of users from one GPU, how can it be that expensive?

An A100 GPU costs ~$10,000. You'll struggle to run even a small LLM with a GPU under $1k right now (there are some exceptions, but it's hard to find a lot of memory under that)

Copilot is worth more than $100/month. Junky enterprise tools get to charge $500/year for seats and Copilot is way more useful. I’m just talking about the standard autocomplete, not the Copilot X labs thing with chat

I have to turn it off anytime I do anything that isn't fairly boilerplate driven

I couldn't disagree more. If someone turned it off tomorrow I'd barely notice.

I don't know about the price but you are not wrong. A lot of naysayers just don't know how to use it well enough in their workflow. This is definitely the future of the industry though.

The people who know 'how to use Copilot well' are the ones using it just for boilerplate. For almost any other code requiring some logic, copilot is useless. More than that, who owns the code produced by copilot, you or github/microsoft ?

Reply on news.ycombinator.com