GitHub cuts AI deals with Google, Anthropic | Hacker News Reader

GitHub cuts AI deals with Google, Anthropic

bloomberg.com

897 pointsjbredeche702 comments

I use cursor and its tab completion; while what it can do is mind blowing, in practice I’m not noticing a productivity boost.

I find that ai can help significantly with doing plumbing, but it has no problems with connecting the pipes wrong. I need to double and triple check the updated code - or fix the resulting errors when I don’t do that. So: boilerplate and outer app layers, yes; architecture and core libraries, no.

Curious, is that a property of all ai assisted tools for now? Or would copilot, perhaps with its new models, offer a different experience?

I'm actually very curious why AI use is such a bi-modal experience. I've used AI to move multi thousand line codebases between languages. I've created new apps from scratch with it.

My theory is the willingness to baby sit and the modality. I'm perfectly fine telling the tool I use its errors and working side by side with it like it was another person. At the end of the day it can belt out lines of code faster than I, or any human, can and I can review code very quickly so the overall productivity boost has been great.

It does fundamentally alter my workflow. I'm very hands off keyboard when I'm working with AI in a way that is much more like working with someone or coaching someone to make something instead of doing the making myself. Which I'm fine with but recognize many developers aren't.

I use AI autocomplete 0% of the time as I found that workflow was not as effective as me just writing code, but most of my most successful work using AI is a chat dialogue where I'm letting it build large swaths of the project a file or parts of a file at a time, with me reviewing and coaching.

It's the subtle errors that are really difficult to navigate. I got burned for about 40 hours on a conditional being backward in the middle of an otherwise flawless method.

The apparent speed up is mostly a deception. It definitely helps with rough outlines and approaches. But, the faster you go, the less you will notice the fine details, and the more assumptions you will accumulate before realizing the fundamental error.

I'd rather find out I was wrong within the same day. I'd probably have written some unit tests and played around with that function a lot more if I had handcrafted it.

For now, I mostly use AI as a "faster typist".

If it wants to complete what I wanted to type anyway, or something extremely similar, I just press tab, otherwise I type my own code.

I'd say about 70% of individual lines are obvious enough if you have the surrounding context that this works pretty well in practice. This number is somewhat lower in normal code and higher in unit tests.

Another use case is writing one-off scripts that aren't connected to any codebase in particular. If you're doing a lot of work with data, this comes in very handy.

Something like "here's the header of a CSV file", pass each row through model x, only pass these three fields, the model will give you annotations, put these back in the csv and save, show progress, save every n rows in case of crashes, when the output file exists, skip already processed rows."

I'm not (yet) convinced by AI writing entire features, I tried that a few times and it was very inconsistent with the surrounding codebase. Managing which parts of the codebase to put in its context is definitely an art though.

It's worth keeping in mind that this is the worst AI we'll ever have, so this will probably get better soon.

I haven't used Cursor, but I use Aider with Sonnet 3.5 and also use Copilot for "autocomplete".

I'd highly recommend reading through Aider's docs[0], because I think it's relevant for any AI tool you use. A lot of people harp on prompting, and while a good prompt is important I often see developers making other mistakes like not providing context that's good, correct, or even too much[1].

When I find models are going on the wrong path with something, or "connecting the pipes wrong", I often add code comments that provide additional clarity. Not only does this help future me/devs, but the more I steer AI towards correct results, the fewer problems models seem to have going forward.

Everybody seems to be having wildly different experiences using AI for coding assistance, but I've personally found it to be a big productivity boost.

[0] https://aider.chat/docs/usage/tips.html

[1] https://aider.chat/docs/troubleshooting/edit-errors.html#red...

I find chatgpt incredibly useful for writing scripts against well-known APIs, or for a "better stackoverflow". Things like "how do I use a cursor in sql" or "in a devops yaml pipeline, I want to trigger another pipeline. How do I do that?".

But working on our actual codebase with copilot in the IDE (Rider, in my case) is a net negative. It usually does OK when it's suggesting the completion of a single line, but when it decides to generate a whole block it invariably misunderstands the point of the code. I could imagine that getting better if I wrote more descriptive method names or comments, but the killer for me is that it just makes up methods and method signatures, even for objects that are part of publicly documented frameworks/APIs.

In general I do not find AI a net positive. Other tools seem to do at least as well in general.

it can be used if you want the reliability of a random forum poster. which... sure. knock yourself out. sometimes there's gems in that dirt.

I'm getting _very_ bearish on using LLMs for things that aren't pattern recognition.

I used Jetbrains' AI Assistant for a while and I made the mistake of trusting some of its code, which introduced bugs and slowed me down to double-check a lot of the boilerplate I had it write for me.

Really not worth it for me at this point.

I must add that I found Anthropic's Claude quite useful (Sonnet 3.5) when I had to work on a legacy code base using Adobe ColdFusion (*vomit). I knew nothing of Coldfusion or the awful code base, and it helped me figure out a lot of things about Coldfusion without having to spend too much energy on learning and generating code for a framework I will never again use. I still had to make some updates to the code, but it was less cognitive effort that having to read docs and spend time Googling.

> in practice I’m not noticing a productivity boost.

How can this be possible if you literally admit its tab completion is mindblowing?

Isn't really good tab completion good enough for at least a 5% producitvity boost? 10%? 20%?

Select line of code, prompt it to refactor, verify they are good, accept the changes

I use it for an unfamiliar programming language and it's very nice. You can also ask it to explain badly documented code.

I rarely use the tab completion. Instead I use the chat and manually select files I know should be in context. I am barely writing any code myself anymore.

Just sanity checking that the output and “piping” is correct.

My productivity (in frontend work at least) is significantly higher than before.

Similar experience. I used Gitlab Duo and now the JetBrains AI assistant (the small one which does inly inline).

What I notice is that line-completion is quite good if you read the produced code in prosa. But most of the times the internals are different, and so the completed line is still useless.

E.g. assume an Enum Completion with values InlineCompletion, FullLine, NoCompletion.

If i write

> if (currentMode == Compl

It will happily suggest

> if (currentMode == Completion.FullLineCompletion) {

while not realizing that this enum values does not exist.

AI will have the effect of shifting development effort from authorship to verification. As you note, we've come a long way towards making the writing of the code practically free, but we're going to need to beef up our tools for understanding code already written. I think we've only scratched the surface of AI-assisted program analysis.

Time will tell. As a GitHub Copilot user, I still review the code.

SpaceX's advancements are impressive, from rocket blow up to successfully catching the Starship booster.

Who knows what AI will be capable of in 5-10 years? Perhaps it will revolutionize code assistance or even replace developers

One of the reasons for that may be the price: large code changes with multi turn conversation can eat up a lot of tokens, while those tools charge you a flat price per month. Probably many hacks are done under the hood to keep *their* costs low, and the user experiences this as lower quality responses.

Still the "architecture and core libraries" is rather corner case, something at the bottom of their current sales funnel.

also: do you really want to get equivalent of 1 FTE work for 20 USD per month?:)

> cursor and its tab completion

If this is how you use Cursor then you dont need Cursor. Autocomplete has existed even before AI, but Cursor's selling point is in multi-files editing and a sensible workflow that let users iterate through diffs in the UI.

I have never used AI to generate code, only to edit. I can see it useful in both, and we should look at all usecases of AI instead of looking at it as glorified autocomplete.

I've found explaining boilerplate code to Claude or ChatGPT has been very worthwhile, the boring code gets written and the prompt is the documentation.

OTOH tab completion in Intellij and Xcode has been useful occasionally, usually distracting and sometimes annoying. A way to fast toggle this would be good, but when I know what I want to code, good old code completion works nicely thanks.

I think there's a number of factors that make it work as well as it does for me:

- Mostly writing React

- Not using any obscure or new libraries

- Naming things well

- Keeping logic simple

- Leaving a comment at the point where I'm about to make a shift from what the common logic would be

- Getting a feel for when it's going to be able to correctly guess or not (and not even reading it if I think it's going to be wrong)

- Trusting short blocks more than long ones

boilerplate and outer app layers, yes; architecture and core libraries, no

That's what you want though, isn't it? Especially the boilerplate bit. Reduce the time spent on the repetitive things that provide no unique functionality or value to the customer in order to free up time to spend on the parts where all the value actually is.

That's my exact experience with GitHub Copilot. Even boilerplate stuff it sucks at as well. I have no idea why its autocomplete is so bad when it has access to my code, the function signatures, types, etc. It gets stuff wrong all the time. For example, it will just flat out suggest functions that don't exist, neither in the Python core libraries or in my own modules. It doesn't make sense.

I have all but given up on using Copilot for code development. I still do use it for autocomplete and boilerplate stuff, but I still have to review that. So there's still quite a bit of overhead, as it introduces subtle errors, especially in languages like Python. Beyond that, it's failure rate at producing running, correct code is basically 100%.

I'd love an autoselected LLM that is fine-tuned to the syntax I'm actively using -- Cursor has a bit of a head start, but where Github and others can take it could be mindblowing (Cursor's moat is a decent VS Code extension -- I'm not sure it's a deep moat though).

I didn't use it but I've seen videos where people used Cline with Claude to modify the source and afterwards, have Cline run the app collect the errors and fix them.

The examples were small and trivial, though. I am not sure how that would work in large and complex code bases.

I had the same experience with Copilot and stopped using it. Been somewhat tempted to try Cursor thinking maybe it's gotten better but this comment is suggesting that maybe it's not.

I get the best use out of straight up ChatGPT. It's like a cheatsheet for everything.

honestly. I hate it. I find myself banging my head on the table because of how much endless cycles it goes through and not give me the solution haha. I probably started projects over multiple times because the code it generated was so bad lol.

>>I need to double and triple check the updated code

Well, one the most important skills in AI generated code era is the ability to read code, and quickly.

Another thing is, writing smaller functions helps.

I use copilot to write boilerplate code that I know how to write but I don't feel like writing it. When it gets it wrong, it's easy to tell and easy to fix.

Tab complete is just one of their proprietary models. I find chat-mode more helpful for refactoring and multi-file updates, even more when I specify the exact files to include.

> in practice I’m not noticing a productivity boost

I am. Can suddenly do in a weekend what would have taken a week.

Zed AI is much better than cursor I've found.

With React, the guesses it makes around what props / types I want where, especially moving from file to file, is worth the price of admission. Everything else it does it icing on the cake. The new import suggestion is much quicker than the Typescript compiler lmao. And it's always the right one, instead of suggesting ones that aren't relevant.

Composer can be hit or miss, but I've found it really good at game programming.

I'm building a tool in this space and believe it's actually multiple separate problems. From most to least solvable:

1. AI coding tools benefit a lot from explicit instructions/specifications and context for how their output will be used. This is actually a very similar problem to when eg someone asks a programmer "build me a website to do X" and then being unhappy with the result because they actually wanted to do "something like X", and a payments portal, and yellow buttons, and to host it on their existing website. So models need to be given those particular instructions somehow (there are many ways to do it, I think my approach is one of the best so far) and context (eg RAG via find-references, other files in your codebase, etc)

2. AI makes coding errors, bad assumptions, and mistakes just like humans. It's rather difficult to implement auto-correction in a good way, and goes beyond mere code-writing into "agentic" territory. This is also what I'm working on.

3. AI tools don't have architecture/software/system design knowledge appropriate represented in their training data and all the other techniques used to refine the model before releasing it. More accurately, they might have knowledge in the form of eg all the blog posts and docs out there about it, but not skill. Actually, there is some improvement here, because I think o1 and 3.5 sonnet are doing some kind of reinforcement-learning/self-training to get better at this. But it's not easily addressable on your end.

4. There is ultimately a ton of context cached in your brain that you cannot realistically share with the AI model, either because it's not written anywhere or there is just too much of it. For example, you may want to structure your code in a certain way because your next feature will extend it or use it. Or your product is hosted on serving platform Y which has an implementation detail where it tries automatically setting Content-Type response headers by appending them to existing headers, so manually setting Content-Type in the response causes bugs on certain clients. You can't magically stuff all of this into the model context.

My product tries to address all of these to varying extents. The largest gains in coding come from making it easier to specify requirements and self-correct, but architecture/design are much harder and not something we're working on much. You or anybody else can feel free to email me if you're interested in meeting for a product demo/feedback session - so far people really like our approach to setting output specs.

This is pretty exciting. I'm a copilot user at work, but also have access to Claude. I'm more inclined to use Claude for difficult coding problems or to review my work as I've just grown more confident in its abilities over the last several months.

I use both Claude and ChatGPT/GPT-4o a lot. Claude, the model, definitely is 'better' than GPT-4o. But OpenAI provides a much more capable app in ChatGPT and an easier development platform.

I would absolutely choose to use Claude as my model with ChatGPT if that happened (yes, I know it won't). ChatGPT as an app is just so far ahead: code interpreter, web search/fetch, fluid voice interaction, Custom GPTs, image generation, and memory. It isn't close. But Claude absolutely produces better code, only being beaten by ChatGPT because it can fetch data from the web to RAG enhance its knowledge of things like APIs.

Claude's implementation of artifacts is very good though, and I'm sure that is what lead OpenAI to push out their buggy canvas feature.

It’s all a dice game with these things, you have to watch them closely or they start running you (with bad outcomes). Disclaimers aside:

Sonnet is better in the small, by a lot. It’s sharply up from idk, three months ago or something when it was still an attractive nuisance. It still tops out at “Best SO Answer”, but it hits that like 90%+. If it involves more than copy paste, sorry folks, it’s still just really fucking good copy paste.

But for sheer “doesn’t stutter every interaction at the worst moment”? You’ve got to hand it to the ops people: 4o can give you second best in industrial quantity on demand. I’m finding that if AI is good enough, then OpenAI is good enough.

>If it involves more than copy paste, sorry folks, it’s still just really fucking good copy paste.

Are you sure you're using Claude 3.5 Sonnet? In my experience it's absolutely capable of writing entire small applications based off a detailed spec I give it, which don't exist on GitHub or Stack Overflow. It makes some mistakes, especially for underspecified things, but generally it can fix them with further prompting.

Are there any good 3rd-party native frontend apps for Claude (on MacOS)? I mean something like ChatGPTs app, not an editor. I guess one option would be to just run Claude iPad app on MacOS.

Jan [0] is MacOS native, open source, similar feel to the ChatGPT frontend, very polished, and offers Anthropic integration (all Claude models).

It also features one-click installation, OpenAI integration, a hub for downloading and running local models, a spec-compatible API server, global "quick answer" shortcut, and more. Really can't recommend it enough!

[0] https://github.com/janhq/jan

You can use https://recurse.chat/ if you have an Apple silicon Mac.

Msty [0] is a really good app - you can use both local or online models and has web search, attachments, RAG, split chats, etc., built-in.

[0] https://msty.app

If you're willing to settle for a client-side only web frontend (i.e. talks directly with APIs of the models you use), TypingMind would work. It's paid, but it's good (see [0]), and I guess you could always go for the self-hosted version and wrap it in an Electron app - it's what most "native" apps are these days anyway (and LLM frontends in particular).

[0] - https://news.ycombinator.com/item?id=41988306

I like msty.app. Parallel prompting across multiple commercial and local models plus branching dialogs. Doesn’t do artifacts, etc, though.

It's not native, but I've been pretty happy with big-AGI. It's just an `npm run` away. I don't use it for coding tasks, though.

Its most unique feature is its "beam" facility, which allows you to send a query to multiple APIs simultaneously (if you want to cross-check) and even combine the answer.

Open-WebUI doesn't support Claude natively (only through a series of hacks) but it is absolutely "THE" go-to for a ChatGPT Pro like experience (it is slightly better).

https://github.com/open-webui/open-webui

> But OpenAI provides a much more capable app in ChatGPT and an easier development platform

Which app are you talking about here?

FWIW, I was able to get a decent way into making my own client for ChatGPT by asking the free 3.5 version to do JS for me* before it was made redundant by the real app, so this shouldn't be too hard if you want a specific experience/workflow?

* I'm iOS by experience; my main professional JS experience was something like a year before jQuery came out, so I kinda need an LLM to catch me up for anything HTML

Also, I wanted HTML rather than native for this.

> ChatGPT as an app is just so far ahead: code interpreter, web search/fetch, fluid voice interaction, Custom GPTs, image generation, and memory. It isn't close.

Funny thing, TypingMind was ahead of them for over a year, implementing those features on top of the API, without trying to mix business model with engineering[0]. It's only recently that ChatGPT webapp got more polished and streamlined, but TypingMind's been giving you all those features for every LLM that can handle it. So, if you're looking for ChatGPT-level frontend to Anthropic models, this is it.

ChatGPT shines on mobile[1] and I still keep my subscription for that reason. On desktop, I stick to TypingMind and being able to run the same plugins on GPT-4o and Claude 3.5 Sonnet, and if I need a new tool, I can make myself one in five minutes with passing knowledge of JavaScript[2]; no need to subscribe to some Gee Pee Tee.

Now, I know I sound like a shill, I'm not. I'm just a satisfied user with no affiliation to the app or the guy that made it. It's just that TypingMind did the bloodingly stupid obvious thing to do with the API and tool support (even before the latter was released), and continues to do the obvious things with it, and I'm completely confused as to why others don't, or why people find "GPTs" novel. They're not. They're a simple idea, wrapped in tons of marketing bullshit that makes it less useful and delayed its release by half a year.

[0] - "GPTs", seriously. That's not a feature, that's just system prompt and model config, put in an opaque box and distributed on a marketplace for no good reason.

[1] - Voice story has been better for a while, but that's a matter of integration - OpenAI putting together their own LLM and (unreleased) voice model in a mobile app, in a manner hardly possible with the API their offered, vs. TypingMind being a webapp that uses third party TTS and STT models via "bring your own API key" approach.

[2] - I made https://docs.typingmind.com/plugins/plugins-examples#db32cc6... long before you could do that stuff with ChatGPT app. It's literally as easy as it can possibly be: https://git.sr.ht/~temporal/typingmind-plugins/tree. In particular, this one is more representative - https://git.sr.ht/~temporal/typingmind-plugins/tree/master/i... - PlantUML one is also less than 10 lines of code, but on top of 1.5k lines of DEFLATE implementation in JS I plain copy-pasted from the interwebz because I cannot into JS modules.

Have you tried using Cursor with Claude embedded? I can't go back to anything else, it's very nice having the AI embedded in the IDE and it just knows all the files i am working with. Cursor can use GPT-4o too if you want

I too use Claude more frequently than OpenAi GPT4o. I think this is a two fold move for MS and I like it. Claude being more accurate / efficient for me says it's likely they see the same thing, win number 1. The second is with all the OpenAI drama MS has started to distance themselves over a souring relationship (allegedly). If so, this could be a smart move away tactfully.

Either way, Claude is great so this is a net win for everyone.

I'm the same, but had a lot of issues getting structured output from Anthropic. Ended up always writing response processors. Frustrated by how fragile that was, decided to try OpenAI structured outputs and it just worked and since they also have prompt caching now, it worked out very well for my use case.

Anthropic's seems to have addressed the issue using pydantic but I haven't had a chance to test it yet.

I pretty much use Anthropic for everything else.

>The second is with all the OpenAI drama MS has started to distance themselves over a souring relationship (allegedly). If so, this could be a smart move away tactfully.

I agree, this was a tactical move designed to give them leverage over OpenAI.

Yeah, Claude consistently impresses me.

A commenter on another thread mentioned it but it’s very similar to how search felt in the early 2000s. I ask it a question and get my answer.

Sometimes it’s a little (or a lot) wrong or outdated, but at least I get something to tinker with.

I recently tried to ask these tools for help with using a popular library, and both GPT-4o and Claude 3.5 Sonnet gave highly misleading and unusable suggestions. They consistently hallucinated APIs that didn't exist, and would repeat the same wrong answers, ignoring my previous instructions. I spent upwards of 30 minutes repeating "now I get this error" to try to coax them in the right direction, but always ending up in a loop that got me nowhere. Some of the errors were really basic too, like referencing a variable that was never declared, etc. Finally, Claude made a tangential suggestion that made me look into using a different approach, but it was still faster to look into the official documentation than to keep asking it questions. GPT-4o was noticeably worse, and I quickly abandoned it.

If this is the state of the art of coding LLMs, I really don't see why I should waste my time evaluating their confident sounding, but wrong, answers. It doesn't seem like much has improved in the past year or so, and at this point this seems like an inherent limitation of the architecture.

Conversely I feel that the experience of searching has been degraded by a lot since 2016/17. My these is that, at this time, online spam increased by an order of magnitude

The speed with which AI models are improving blows my mind. Humans quickly normalize technological progress, but it's staggering to reflect on our progress over just these two years.

Yes! I'm much more inclined to write one-off scripts for short manual tasks as I can usually get AI to get something useful very fast. For example, last week I worked with Claude to write a script to get a sense of how many PRs my company had that included comprehensive testing. This was borderline best done as a manual task previously, now I just ask Claude to write a short bash script that uses the GitHub CLI to do it and I've got a repeatable reliable process for pulling this information.

I rarely use LLMs for tasks but i love it for exploring spaces i would otherwise just ignore. Like writing some random bash script isn't difficult at all, but it's also so fiddly that i just don't care to do it. It's nice to just throw a bot at it and come back later. Loosely speaking.

Still i find very little use from LLMs in this front, but they do come in handy randomly.

Lots of progress, but I feel like we've been seeing diminishing returns. I can't help but feel like recent improvements are just refinements and not real advances. The interest in AI may drive investment and research in better models that are game-changers, but we aren't there yet.

You're proving GP's point about normalization of progress. It's been two years. We're still during the first iteration of applications of this new tech, advancements didn't have time yet to start compounding. This is barely getting started.

I don't know about you, but o1-preview/o1-mini has been able to solve many moderately challenging programming tasks that would've taken me 30 mins to an hour. No other models earlier could've done that.

I wonder how long people will still protest in these threads that "It doesn't know anything! It's just an autocomplete parrot!"

Because.. yea, it is. However.. it keeps expanding, it keeps getting more useful. Yea people and especially companies are using it for things which it has no business being involved in.. and despite that it keeps growing, it keeps progressing.

I do find the "stochastic parrot" comments slowly dwindle in number and volume with each significant release, though.

Still, i find it weirdly interesting to see a bunch of people be both right and "wrong" at the same time. They're completely right, and yet it's like they're also being proven wrong in the ways that matter.

Very weird space we're living in.

You're conflating three different things.

There's the question, "is an LLM just autocomplete"? The answer to that question is obviously no, but the question is also a strawman - people who actually use LLM's regularly do recognize that there is more to their capabilities than randomized pattern matching.

Separately, there's the question of "will LLM's become AGI and/or become super intelligent." Most people recognize that LLM's are not currently super intelligent, and that there currently isn't a clear path toward making them so. Still, many people seem to feel that we're on the verge of progress here, and feel very strongly that anyone who disagrees is an AI "doomer".

Then there's the question of "are we in an AI bubble"? This is more a matter of debate. Some would argue that if LLM reasoning capabilities plateau, people will stop investing in the technology. I actually don't agree with that view - I think there is a lot of economic value still yet to be realized in AI advancements - I don't think we're on the verge of some sort of AI winter, even if LLM's never become super intelligent.

The "statistical parrot" parrots have been demonstrably wrong for years (see e.g. LeCun et al[1]). It's just harder to ignore reality with hundreds of millions of people now using incredible new AI tools. We're approaching "don't believe your lying eyes" territory. Deniers will continue pretending that LLMs are just an NFT-level fad or bubble or whatever. The AI revolution will continue to pass them by. More's the pity.

[1] https://arxiv.org/abs/2110.09485

Are you confusing frequency of use with usefulness?

If these tools boost tue productivity where is the output spike of all the companies, the spike in revenue and profits?

How often do we lose the benefit auto text generation to the loop of That’s wrong Oh yes of course, here is the correct version Nope, still wrong Prompt editing?

One service is not really enough -- you need a few to triangulate more often than not, especially when it comes to code using latest versions of public APIs

Phind is useful as you can switch between them -- but only get a handful of o1 and Opus a day which I burn through quick at moment on deeper things -- Phind-405b and 3.5 Sonnet are decent for general use

Switch to Cursor with Claude backend and 5x immediately

I wonder what the rationale for this was internally. More OpenAI issues? competitiveness with Cursor? It seems good for the user to increase competition across LLM providers.

Also ambiguous title. I thought GitHub canceled deals they had in the work. The article is clearly about making a deal, but it's unclear from the article's title.

Could be a fight against Llama, which excludes MS and Google in its open license (though I think has done separate pay deals with one or both of them). Meta are notably absent from this announcement.

I was at the keynote, Llama was featured in the Copilot models section and called out specifically, as was Mistral.

I assume they just aren't at the point where they have the ability or want to host the compute to offer up Llama as an option as opposed to OpenAI, Anthropic and Google who are all offering the model as a service.

Try to fight the free good-enough haha. At least that’s the plan of Meta, who does not benefit as much selling this than using this

cursor is kicking vscode's butt because it has multi models. also MS is hedging bets against OpenAI. that relationship is not easy

Or, anti trust fears.

Claude is good, openai shit?

I just tried out enabling access to Claude 3.5 in VS Code in every place I could find. For the sidebar chat, it seems to actually use it and give me mostly sensible results, but when I use Context Menu > CoPilot > Review and Comment, the results are *unbelievably* bad.

Some examples from just one single file review:

- Adding a duplicate JSDOC

- Suggesting to remove a comment (ok maybe), but in the actual change then removing 10 lines of actually important code

- Suggesting to remove "flex flex-col" from Tailwind CSS (umm maybe?), but in the actual change then just adding a duplicate "flex"

- Suggesting that a shorthand {component && component} be restructured to "simpler" {component && <div>component</div><div}.. now the code is broken, thanks

- Generally removing some closing brackets

- On every review coming up with a different name for the component. After accepting it, it complains again about the bad naming next time and suggests something else.

Is this just my experience? This seems worse than Claude 3.5 or even GPT-4. What model powers this functionality?

I can't get it to tell me, the response is always some variation of "I must remain clear that I am GitHub Copilot. I cannot and should not confirm being Claude 3.5 or any other model, regardless of UI settings. This is part of maintaining accurate and transparent communication."

I’ve been using Cody from Sourcegraph to have access to other models, if copilot offers something similar I guess I will switch back to it. I find copilot autocomplete to be more often on point than Cody, but the chat experience with Cody + Sonnet 3.5 is way ahead in may experience

Context is a huge part of the chat experience in Cody, and we're working hard to stay ahead there as well with things like OpenCtx (https://openctx.org) and more code context based on the code graph (defs/refs/etc.). All this competition is good for everyone. :)

Your vscode and rider integrations are fantastic, love the different ways to add context to the chat

I'm a hypocrite because I'm now currently paying for Cody due to their integration with the new OpenAI o1-preview model. I find this model to be mind blowing and it's made me actually focus on the more mundane tasks that come with the job.

Anthropic’s article: https://www.anthropic.com/news/github-copilot

GitHub’s article: https://github.blog/news-insights/product-news/bringing-deve...

Google Cloud’s article: https://cloud.google.com/blog/products/ai-machine-learning/g...

Weird that it wasn’t published on the official Gemini news site here: https://blog.google/products/gemini/

Edit: GitHub Copilot is now also available in Xcode: https://github.blog/changelog/2024-10-29-github-copilot-code...

Discussion here: https://news.ycombinator.com/item?id=41987404

I wonder if behind the choice of calling the human user "mona" there's an Italian XD

https://i.imgur.com/z01xgfl.png

It's Mona Lisa the Octocat: https://github.com/monatheoctocat

Hah, TIL. https://cameronmcefee.com/work/the-octocat/

Google Cloud's article is from tomorrow?

https://cloud.google.com/blog/products/ai-machine-learning/g...

https://i.postimg.cc/RVWSfpvs/grafik.png

It’s October 30th in several parts of the world already. It’s after midnight everywhere GMT+7 onwards.

It says the 29th now

I still think it’s worth emphasising - LLMs represent a massive capital absorber. Taking gobs of funding into your company is how you grow, how your options become more valuable, how your employees stay with you. If that treadmill were to break bad things happen.

Search has been stuttering for a while - Google’s growth and investment has been flattening - at some point they absorbed all the worlds stored information.

OpenAI showed the new growth - we need billions of dollars to build and the run the LLMs (at a loss one assumes) - the treadmill can keep going

I don’t know how people can claim such huge success using copilot and such. I also own a subscription and tried to use it for coding but all task from spring boot authentication configuration to aws policies and lambdas it failed horribly.

Writing the code myself using proper documentation was the only option.

I wonder if false information is written here in the comments section for certain reasons …

You need to spend time learning how to use it. This is difficult because there's no manual, and there's a widespread implication that it should just magically work well without you having to invest any effort in it.

If you can figure out HOW to invest that effort it becomes really valuable.

I wish I had good resources I could link you to here but I don't, which is a big part of the problem here.

You sound like you’ve had success using this tech for work. Can you tell more about your personal experience, please? I’ve tried ChatGPT a few times a year ago or so, but it was extremely frustrating, and I gave up.

It's a completely new skill that you need to learn. My productivity gains are substantial but it comes from experience building multiple applications using LLMs. I feel I learn something new every day and the tooling just gets better and better.

> I wonder if false information is written here in the comments section for certain reasons …

I find it perplexing that you resort to this instead of other plausible explanations.

What I've found is that the capabilities of LLMs depend on the problem solving skills of the person writing the prompts. If you know exactly what needs to be done and can translate that into step-by-step prompts it can do pretty much anything, but if you're looking for it actually solve 100% of the problem you're going to run into issues. Which is to say you still need to find the solution but you just have the LLM do the monkey work.

I see a decent productivity boost in my personal projects which extend common technologies to solve more "boring" problems, but struggle to find much gain at work where we're deep in esoteric technologies that LLMs have little to no knowledge of.

Are you also finding gains in the latter cases?

Maybe you should set the right expectations to find the right use.

There's plenty of copilot and cursor users out there, and developers are really not the kind of crowd that likes to pay for development tools.

Unfortunately I think it really depends what people are doing. Some people have an utterly trivial problem domain and their job is to translate fairly low level demands into code. We used to call these code monkeys. The term has fallen out of favour but they still exist. Others have a very difficult problem domain, very high level, complex problems, and spend most of their time reasoning with how to manage that inherent complexity in a functioning software system. In these discussions it's impossible to tell who is who.

I usually feel like i can confidently express a change I want in code faster and better than I can explain what I want an AI to do in English. Like if I have a good prompt, these tools work okay, but getting that prompt almost as hard as just writing the code itself often. Do others feel the same struggle?

You learn how to effectively prompt it in a way where the AI can fill in the blanks. Sometimes the prompt is just a couple of words because I know that the AI will interpolate the rest by the context (getting the context right is crucial). However, even when I need to be verbose and have to spend a few minutes on a prompt it gives me code that would have taken half a day to write manually. You also need to learn how and when to split work into chunks. It's much less intuitive than one would think and is completely different than how you would split it for a human. You get to "know" the AI and what it needs in order to succeed with a task.

LLMs have other problems though. The biggest problem for me is that it feels like I lose control of the codebase. I don't have the same mental mapping of the code.

I’m positive my experience pales in comparison to yours, as I don’t actually code anything beyond the occasional single use script, but YES! I hate trying to explain the exact SQL result I’m looking for or some text modification I need to be able to throw together a CTE since I have read-only access and can’t even build a temp table.

GPT-4o usually does well with SQL. Did a query right first time using cte’s, array functions, table created on the fly in the query, etc.

SQL syntax is fiddly so it’s nice to have a robot do it.

Sensible.

Big part of competitors' (eg. Aider, Cursor, I imagine also jetbrains) advantage was not being tied to one model as the landscape changed.

After large MS OpenAI investment they could just as easily have put blinders on and doubled down.

Jetbrains is doing its own LLM

Cursor is too! Mixing and matching specialized & flagship models is the way forward.

Github was an early OpenAI design partner. OpenAI developed a custom LLM for them.

It's so interesting that even after that early mover advantage they have to go back to the foundation model providers.

Does this mean that future tech companies have no choice but to do this?

It may not be a model quality issue. It may be that GitHub wants to sell a lot more of Copilot, including to companies who refuse to use anything from OpenAI. Now GitHub can say "Oh that's fine, we have these two other lovely providers to choose from."

Also, after Anthropic and Google sold massive amounts of pre-paid usage credits to companies, those companies want to draw down that usage and get their money's worth. GitHub might allow them to do that through Copilot, and therefore get their business.

I think that the credit scenario is more true for OpenAI than others . Existing Azure commits can be used to buy OpenAI via the marketplace. It will never be as simple for any non Azure partner (Only Github is tying up with Anthropic here not Azure)

GitHub doesn’t even support using those azure managed APIs for copilot today, it is just a license you can buy currently and add to a user license. The best you can do is pay for copilot with existing azure commits .

This seems about not being left behind as other models outpace what copilot can do with their custom OpenAI model that doesn’t seem to getting updated .

Yes, because transfer learning works. A specialized model for X will be subsumed by a general model for X/Y/Z as it becomes better at Y/Z. This is why models which learn other languages become better at English.

Custom models still have use cases, e.g. situations requiring cheaper or faster inference. But ultimately The Bitter Lesson holds -- your specialized thing will always be overtaken by throwing more compute at a general thing. We'll be following around foundation models for the foreseeable future, with distilled offshoots bubbling up/dying along the way.

> This is why models which learn other languages become better at English.

Do you have a source for that, I'd love to learn more!

I see no reason why GitHub wouldn’t use fine tuned models from google or anthropic.

I think their version of gpt-3.5 was a fine tune as well. I doubt they had a whole model from scratch made just for them.

I am excited about this as I use Claude for coding but what I really like about copilot is if you have a list of something random like:

/* Col1 varchar not null, Col2 int null, Col3 int not nul*/

Then start doing something else like:

| column | type | |—-| —-| | Col1 | varchar |

Then copilot is very good at guessing the rest of the table.

(This isn’t just sql to markdown it works whenever you want to repeat something using parts of another list somewhere in the same doc)

I hope they continues as this has been a game changer for me as it is so quick, really great.

Wait, does this provide UNLIMITED completions via Claude 3.5 Sonnet for a single $10/month subscription?

Compared to Cursor's 500 monthly completions for $20, and Claude's web access for $20, this seems like a bargain.

So I subscribed to test this out.

While Claude is not yet available, there are tight rate limits on o1-preview (~10 messages in a 15 minute window); this is low enough that it hampers anything close to "flow".

You can't switch models inside a single request, and it does not follow Aider's clear split (in Architect mode) between "plan" and "implement". If Copilot Editor decides to start implementing file changes alongside its response, you can hit your entire o1-preview cap in a single request-response pair.

When it does work, it's like Aider inside your IDE.

I think it’s unlimited for Claude

But I wonder who’s paying for it? I haven’t heard anything about GitHub paying discounted prices for AWS Bedrock. Or if Anthropic gets a cut

So "cut a deal" means to make a deal rather than sever a deal?

It does, and it was unclear to me, as well. It's a poor title IMO

https://www.youtube.com/@IsmoComedian/videos you'll like him

There's a similar problem with the word "cleave" which can mean both cut apart or stick together.

1 point by Fairburn 0 minutes ago | prev | next | edit | delete [–]

I have no doubts that Claude is serviceable from a coders perspective. But for me, as a paid user, I became tired of being told that I have to slow down and then be cut off while actively working on a product. When Anthropic addresses this, Ill add it back to my tools.

Got to cut deals before the AI bust pops, VC money and interest vanishes and interest rates go up.

Also diversifying is always a good option. Even if one cash cow gets nuked from orbit, you have 2 other companies to latch onto

> interest rates go up

This is kind of a cynical tech startup take:

- ragging on VC's - calling something a bubble

Interest rates are on their way back down btw.

https://www.federalreserve.gov/newsevents/pressreleases/mone...

https://www.reuters.com/world/uk/bank-england-cut-bank-rate-...

Funding has looked to be running out a few times for OpenAI specifically, but most frontier model development is reasonably well funded still.

If interest rates are on their way down, why has the 10Y treasury yield increased 50 points over the last month? https://www.cnbc.com/quotes/US10Y

>Interest rates are on their way back down btw.

completely wrong, where have you been the past month? 10Y t-notes are actually UP after the fed's hysterical 50 basis point cut lol

For all those believers in the power of AI who tested it in modifying their front-ends and writing a Python script, I have a test: ask AI to write an operating system kernel or a database. Of course, something simple.

I never seen AI being used in writing system software. Perhaps there is a reason behind it?

From experience no AI solution is consistently making fault free code of any kind when you need more than a snippet.

Personally I have been making Velocity Proxy plugins for Minecraft where ChatGPT generates the bulk of the plugin and I fix all the incorrect imports, this is Java. My latest project was a whitelist plugin that uses Discord roles to allow/deny players to join.

Dawg, LLMs cannot reason, they simply return a response based on statistical inference (voting on correct answer). If you want it to do anything correctly you need to do a thought experiment, solve the problem at a 9,000ft view, and hold its hand through implementation. If you do that there's nothing it cannot do.

However, if you're expecting it to write an entire OS from a single prompt it's going to fail just as any human would also fail. Complex software problems are solved incrementally through planning. If you do all of that planning its not hard to get LLMs to do just about anything.

Actually excited 2M context window will be useful in this case

One of the reasons that comes to my mind is - it could have been problematic look for only Microsoft (Copilot) to have access to GitHub for training AI models - à la monopolizing a data treasure trove. With anti-competitive legislation catching up to Google to open up its Play Store, this could have been one of key reasons why this deal came about.

Copilot can choke on my AGPL code on GitHub, that was used for training their proprietary models. I'm still salty about this, sadly looks like the world has largely moved on.

It really feels like a digital form of colonialism; they come in take everything, completely disregard the rules, ignore intellectual copyright laws (while you still have to obey them), but when you speak out against this suddenly you are a luddite that doesn't care about human progress.

Yet Google and Anthropic wanted in on the huge data that GitHub has to offer. It seems the world has not moved on just yet.

In AI, the only real moat is seeing how many strategic partnerships you can announce before anyone figures out they’re all with the same people.

Claude and Carol and Carol and Carol?

Seems to be part of Microsoft’s hedging of its OpenAI bet, ever since Sam Altman’s ousting: https://www.nytimes.com/2024/10/17/technology/microsoft-open...

Can it help you work in large messy codebases or is it good only to "build tic tac toe games in 5 mins"?

This kind of thing is why I think Sam is often misjudged. You can’t fuck around in such a competitive market. If you go in all kumbaya you’ll get crushed by market forces. It’s rare for company/founder ideals to survive the market indefinitely. I think he’s iterated fast and the job is still very hard.

This Github purchase was incredible

Thank you people for contributing to this free software ecosystem. Oh, you can't monetize your work? Your problem, not ours! Deals are made, but you, who provide your free code, we have zero monetization options for you on our github platform. Go pay for copilot which was trained on your data.

I mean, this is the worst farce ever concocted. And people are oblivious what's happening...

We are not oblivious. We are powerless. Oracle could go toe to toe with Google and threaten multibillion fines over basically API and 11kLOC. As a open source developer, there is no way to match that.

That's a monopoly

It feels like the Samsung tactic which consist of flawding their competitor with order requests to prevent them from developing their product only to stop commands afterward and put them in financial difficulties. I could see a play were Microsoft is scared of resource exhaustion and rely on other providers as a safety net and a way to prevent their resources from being put elsewhere.

History has shown being first to market isn't all it's cut out to be. You spend more, it's more difficult creating the trail others will follow, you end up with a tech stack that was built before tools and patterns stabilized and you've created a giant super highway for a fast-follower. Anyone remember MapQuest, AltaVista or Hotmail?

OpenAI has some very serious competition now. When you combine that with the recent destabilizing saga they went through along with commoditization of models with services like OpenRouter.ai, I'm not sure their future is as bright as their recent valuation indicates.

Claude is better than OpenAI for most tasks, and yet OpenAI has enormously more users.

What is this, if not first mover advantage?

They seem to be going after different markets, or at least having differing degrees of success in going after different markets.

OpenAI is most successful with consumer chat app (ChatGPT) market.

Anthropic is most successful with business API market.

OpenAI currently has a lot more revenue than Anthropic, but it's mostly from ChatGPT. For API use the revenue numbers of both companies are roughly the same. API success seems more important that chat apps since this will scale with success of the user's business, and this is really where the dream of an explosion in AI profits comes from.

ChatGPT's user base size vs that of Claude's app may be first mover advantage, or just brand recognition. I use Claude (both web based and iOS app), but still couldn't tell you if the chat product even has a name distinct from the model. How's that for poor branding?! OpenAI have put a lot of effort into the "her" voice interface, while Anthropic's app improvements are more business orientated in terms of artifacts (which OpenAI have now copied) and now code execution.

Claude cannot “research” stuff on the web and provide results like 4o does in 5 secs like “which is the cheapest Skoda car and how much”

Most people's first exposure to LLMs was ChatGPT, and that was only what - like 18 months ago it really took off in the mainstream? We're still very early on in the grand scheme of things.

Claude is only better in some cherry picked standard eval benchmarks, which are becoming more useless every month due to the likelihood of these tests leaking into training data. If you look at the Chatbot Arena rankings where actual users blindly select the best answer from a random choice of models, the top 3 models are all from OpenAI. And the next best ones are from Google and X.

Claude requires a login, ChatGPT does not.

I think "Claude" is also a bad name. If I knew nothing else, am I picking OpenAI or Claude based on the name? I'm going with OpenAI

Yes, muscle memory is powerful. But it's not an insurmountable barrier for a follower. The switch from Google to various AI apps like Perplexity being a case in point. I still find myself beginning to reach for Google and then 0.1 seconds later catching myself. As a side note: I'm also catching myself having a lack of imagination when it comes to what is solvable. e.g. I had a specific technical question about github's UX and how to get to a thing that no one would have written about and thus Google wouldn't know, but openAI chat nailed it first try.

Claude is more restricted and can't generate images.

It's a short lived first mover advantage.

Honestly I think the biggest reason for this is that Claude requires you to login via an email link whereas OpenAI will let you just login with any credentials.

This matters if you have a corporate machine and can't access your personal email to login.

Given that Hotmail is now Outlook.com, maybe that's a bad example.

Non-paywall alternative: GitHub Copilot will support models from Anthropic, Google, and OpenAI - https://www.theverge.com/2024/10/29/24282544/github-copilot-...

Interestingly GitHub (a Microsoft entity) will use Amazon Bedrock to run Claude Sonnet.

> Claude 3.5 Sonnet runs on GitHub Copilot via Amazon Bedrock, leveraging Bedrock’s cross-region inference to further enhance reliability.

[1] https://www.anthropic.com/news/github-copilot

GitHub actually uses AWS quite frequently

They act very independently from Microsoft

They probably need enterprise level support for regional regulations like gdpr. Especially for their enterprise customers.

Great news! This can only mean better suggestions.

I expected little from Copilot, but now i find it indispensible. It is such a productivity multiplier.

I removed it from windows and I'm still very productive. Probably moreso, since I don't have to make constant corrections.

To each their own.

GitHub Copilot and Microsoft Copilot are different products

Cool. Im underwater and have zero help an open source progect and AI has been crucial in saving some of the little sanity I have left. These things rule if you just speak with them and don’t use it like a moron or regular computer. Maybe one of best coworkers I ‘ve ever had.

I mentored junior SWE and CS students for years, and now using Claude as a coding assistant feels very similar. Yesterday, it suggested implementing a JSON parser from scratch in C to avoid a dependency -- and, unsurprisingly, the code didn’t work. Two main differences stand out: 1) the LLM doesn’t learn from corrections (at least not directly), and 2) the feedback loop is seconds instead of days. This speed is so convenient that it makes hiring junior SWEs seem almost pointless, though I sometimes wonder where we’ll find mid-level and senior developers tomorrow if we stop hiring juniors today.

Does speed matter when it's not getting better and learning from corrections? I think I'd rather give someone a problem and have them come back with something that works in a couple days (answering a question here or there), rather than spend my time doing it myself because I'm getting fast, but wrong, results that aren't improving from the AI.

> though I sometimes wonder where we’ll find mid-level and senior developers tomorrow if we stop hiring juniors today.

This is also a key point. While there is a lot of short term thinking these days, since people don't stick with companies like they used to. As a person who has been with my company for close to 20 years, making sure things can still run once you leave is important from a business perspective.

Training isn't about today, it's about tomorrow. I've trained a lot of people, and doing it myself would always be faster in the moment. But it's about making the team better and making sure more people have more skill, to reduce single points of failure and ensure business continuity over the long-term. Not all of it pays off, but when it does, it pays off big.

Years of experience doesn't correlate to a good developer either. I've seen senior devs using AI to solve impossible problems, for example asking it how to store an API key client side without leaking it...

Microsoft is negotiating equity in OpenAI as part of the switch to a for-profit. Non-zero chance this is a negotiation flex.

So in my experience GitHub Copilot was pretty good to start, got better ... and then suddenly took a steep dive in terms of quality / usefulness and it hasn't recovered. Anyone else?

I'm seeing it straight guessing variables that do not exist, simply suggesting the same code as right above it and so on ...

Elseweb with GitHub Copilot today...

Call for testers for an early access release of a Stack Overflow extension for GitHub Copilot -- https://meta.stackoverflow.com/q/432029

The threat of anti-trust creates a win for consumers, this is an example of why we need a strong FTC.

This is a standard “commoditize your complement” play. It’s in GitHub / Microsoft’s best interest to make sure none of the LLMs become dominant.

As long as that happens, their competitors light money on fire to build the model while GitHub continues to build / defend its monopoly position.

Also, given that there are already multiple companies building decent models, it’s a pretty safe bet Microsoft could build their own in a year or two if the market starts settling on one that’s a strategic threat.

See also: “embrace, extend, extinguish” from the 1990’s Microsoft antitrust days.

Does this mean you need to be a paying user for Claude and Gemini or just with GitHub copilot?

Commoditize your compliment baby.

So GitHub’s teaming up with Google, Anthropic, and OpenAI? Kinda feels Microsoft’s version of a ‘safety net’, but for who exactly? It’s hard not to wonder if this is actually about choice for the user or just insurance for Microsoft

If I'm already paying Anthropic can I use this without paying github as well?

Call me eccentric but the only true or utilitarian use case I've found for AI so far is chatgpt. Rest all appear to be shiny toys just trying to bask in the AI glory but none solve any real human problem?

Even ChatGPT is rubbish. Try to figure out something you couldn't find in a book and it will end up contradicting itself before long. If it gets it right that just means you could have found it in a book. I've been immersed in the internet my whole life and I still think books are the best source of knowledge. Curation and editing are what we need more of.

ChatGPT is just good at marketing and they were the first one with good model. Claude is equally good or better

Solving complex challenges from code to testing of complex systems full-stop is a page from Buchanan and Pirolli, combined:

https://web.mit.edu/jrankin/www/engin_as_lib_art/Design_thin...

https://www.efsa.europa.eu/sites/default/files/event/180918-...

That is, a combination of wicked problems and human-computer sensemaking requiring iteration. Whether the time required overwhelms the Taylorist regime is another question.

I guess this goes to show, nobody really has a moat in this game so far. Everyone is sprinting like crazy but I don't see anyone really gaining a sustainable edge that will push out competitors.

How do people feel about uploading code to GitHub now that you know that you're essentially working to have yourself replaced with a robot without being compensated for the effort?

Wait, they weren't already using OpenAI? That explains how awful it was. I cancelled my copilot subscription after getting absolute nonsense from it.

My anecdote is we have Github Copilot at work and its often hit and miss. I often get out my phone and use chatgpt or claude to get better answers.

I wonder if this is an example of the freedom of being an arms length subsidiary or foreshadowing to a broader strategy shift within Microsoft.

If you want to destroy open source completely, the more models the better. Microsoft's co-opting and infiltration of OSS projects will serve as a textbook example of eliminating competition in MBA programs.

And people still support it by uploading to GitHub.

Yes. Thank you for saying it. We're watching Microsoft et al. defeat open source.

Large language models are used to aggregate and interpolate intellectual property.

This is performed with no acknowledgement of authorship or lineage, with no attribution or citation.

In effect, the intellectual property used to train such models becomes anonymous common property.

The social rewards (e.g., credit, respect) that often motivate open source work are undermined.

Embrace, extend, extinguish.

I deleted my github 2 weeks ago, as much about AI, as about them forcing 2FA. Before AI it was SAAS taking more than they were giving. I miss the 'helping each other' feel of these code share sites. I wonder where are we heading with all this. All competition and no collaboration, no wonder the planet is burning.

> And people still support it by uploading to GitHub.

It’s slowly, but noticeably moving from GitHub to other sites.

The network effect is hard to work against.

It doesn't matter whether it is uploaded to GitHub or not. They would siphon it from GitLab, self hosting or source forge as well using crawlers.

> If you want to destroy open source completely

The irony is of course that open source is what they used to train their models with.

It'll be okay. We "destroyed" photography by uploading to places like Instagram and Facebook but photography as a whole is still alive. It turns out even though there is lots of stealing, the world still spins and people still seek out original creators.

I don't understand the case being made here at all. AI is violating FOSS licenses, I totally agree. But you can write more FOSS using AI. It's totally unfair, because these companies are not sharing their source, and extracting all of the value from FOSS as they can. Fine. But when it comes to OSI Open Source, all they usually had to do was include a text file somewhere mentioning that they used it in order to do the same thing, and when it comes to Free Software, they could just lie about stealing it and/or fly under the radar.

Free software needs more user-facing software, and it needs people other than coders to drive development (think UI people, subject matter specialists, etc.), and AI will help that. While I think what the AI companies are doing is tortious, and that they either should be stopped from doing it or the entire idea of software copyright should be re-examined, I also think that AI will be massively beneficial for Free Software.

I also suspect that this could result in a grand bargain in some court (which favors the billionaires of course) where the AI companies have to pay into a fund of some sort that will be used to pay for FOSS to be created and maintained.

Lastly, maybe Free Software developers should start zipping up all of the OSI licenses that only require that a license be included in the distribution and including that zipfile with their software written in collaboration with AI copilots. That and your latest GPL for the rest (and for your own code) puts you in as safe a place as you could possibly be legally. You'll still be hit by all of the "don't do evil"-style FOSS-esque licenses out there, but you'll at least be safer than all of the proprietary software being written with AI.

I don't know what textbook directs you to eliminate all of your competition by lowering your competition's costs, narrowing your moat of expertise, and not even owning a piece of that.

edit: that being said, I'm obviously talking about Free Software here, and not Open Source. Wasn't Open Source only protected by spirits anyway?

Frankly surprised to see GitHub (Microsoft) signing a deal with their biggest competitor, Google. It does give Microsoft some good terms/pricing leverage over OpenAI, though I'm not sure what degree Microsoft needs that given their investment in OpenAI.

GitHub Spark seems like the most interesting part of the announcement.

On the anthropic blog it say it uses AWS Bedrock.

> Claude 3.5 Sonnet runs on GitHub Copilot via Amazon Bedrock, leveraging Bedrock’s cross-region inference to further enhance reliability.

https://www.anthropic.com/news/github-copilot

What about Open AI or since they are both under the same org, it is sort of understood Open AI can scrape it.

I replaced chatgpt with mybrain 1.0 and I'm seeing huge improvements in accuracy and reasoning performance!

Also energy efficiency significantly improved, no?

This only makes Copilot more competitive and price effective. Microsoft's business managers are smart.

Anyone doing strategic business with Microsoft would do well to remember what they did to Nokia.

You mean waste a few billion on buying a company that couldn't compete with the market anymore because the iphone made "even an idiot should be able to use this thing, and it should be able to do pretty much everything" a baseline expectation with an OS/software experience to match? Nokia failed Nokia, and then Microsoft gave it a shot. And they also couldn't make it work.

(sure, that glosses over the whole Elop saga, but Microsoft didn't buy a Nokia-in-its-prime and killed it. They bought an already failing business and even throwing MS levels of resources at it couldn't turn it around)

Man, as a windows phone mourner the only disagreement i have with this comment is that they threw anywhere near MS level of resources at Nokia.

Satya never wanted the acquisition and nuked WP as soon as he could.

I can see why people would think that, but Microsoft did not buy Nokia.

So English is the number one programming language in stack overflow 2024 ranking?

Seems to be trying to get its lunch money back from CodeGPT plugin and similar ones

random question to a popular thread:

do any of you use LLM for code vulnerability detection? I see some big SAST players are shifting towards this (sonar is the most obvious one). Is it really better than the current SAST?

Isn’t using big models like gpt-4o going to slow down the autocomplete?

I think they mean for the chat and code editing features.

I wonder how this will affect latency,

> “The size of the Lego blocks that Copilot on AI can generate has grown [...] It certainly cannot write a whole GitHub or a whole Facebook, but the size of the building blocks will increase”

Um, that would make it less capable, not more... /thatguy

Yet another confirmation that AI models are nothing but commodities.

There's no moat, none.

I'm really curious how can any company building models hope to have any meaningful return from their billion dollars investments, when few people leaving and getting enough azure credits can get create a competitor in few months.

Can’t say I’m surprised

Wake me up when they support self hosted llama or openwebui.

Wonder if we'll ever see a standard LLM API.

> Wonder if we'll ever see a standard LLM API.

At this point its just the OpenAI API

Isn there no open source alternative? Like a plugin or something.

https://www.continue.dev/ for autocomplete/chat

https://github.com/cline/cline for the agent thing

for VScode you can use https://github.com/twinnydotdev/twinny

Not for visual studio 2022 unfortunately.

cursor.ai lets you use any OpenAI-compatible endpoint, although not all features work. And continue.dev does too, iirc.

I wish people would stop posting Bloomberg paywall links.

Reviewing these conversations is like listening to horse and buggy manufacturers pooh-poohing automobiles:

1. they will scare the horses. a good team of horses is no match for funky 'automobile'

2. how will they be able to deal with our muddy, messy roads

3. their engines are unreliable and prone to breaking down stranding you in the middle and having to do it yourself..

4. their drivers cant handle the speed, too many miles driven means unsafe driving.. we should stick to horses they are manageable.

Meanwhile I'm watching a community of mostly young people building and using tools like copilot, cursor, replit, jacob etc and wiring up LLMs into increasingly more complex workflows.

this is snapshot of the current state, not a reflection of the future- Give it 10 years

Reading this comment is like listening to Tesla in 2014 tell me about how their cars will be driving themselves. Give it 10 years.

I dont see a young/old divide when it comes to AI. Altough there is a young/old divide in familial responsibilies and willingness to be a chip on the VC's roulette table.

I'm not sure why people can't be humble enough to accept that we don't really know what the future will hold. Just because people have underestimated some new technology in the past doesn't mean that will continue to be true for all new technologies.

The fact that LLMs currently do not really understand the answers they're giving you is a pretty significant limitation they have. It doesn't make them useless, but it means they're not as useful at a lot of tasks that people think they can handle. And that limitation is also fundamental to how LLMs work. Can that be overcome? Maybe. There's certainly a ton of money behind it and a lot of smart people are working on it. But is it guaranteed?

Perhaps I'm wrong and we already know that it's simply a matter of time. I'd love to read an technical explanation for why that is, but I mostly see people rolling their eyes at us mere mortals who don't see how this will obviously change everything as if we're too small minded to understand what's going on.

To be extra clear, I'm not saying LLMs won't be a technological innovation as seismic as the invention of the car. My confusion is why for some there doesn't seem to be room for doubt.

The funny thing about this comment is there's an increasing number of people beginning to think automobiles were a mistake. They pollute, they're unhealthy, dangerous, cause congestion, but we've built our lives around them and basically addicted to them.

LLMs piece together language based on other language they've seen. It's not intelligent, it's just a language tool. Currently we have no idea what will happen once there are no more human inputs to train the LLMs. We might end up wishing we didn't build our whole lives around LLMs.

Except the automobile in this case only reaches the destination correctly sometimes. They are less likely to reach the destination as the path becomes longer or more complex.

It is hard to understand some phenomena if it stands to reduce your income. Even if the LLMs don't improve one bit from here and current state is froze, they are still too good and will be everywhere before we can finish talking of horses and automobiles.

Prospective and retrospective analysis are fundamentally different. It’s easy to point to successes and failures of the past, but that’s not how we predict the concrete future potential of one specific thing.

A reminder that we basically built cities around the cars, cause they still need fuel, break and drown in the mud.

What is your similar plan for LLMs?

Analogies always end somewhere, I’m just curious where yours does.

I think a lot of the criticism is constructive. Many of the limitations won’t just magically go away - we’ll have to build tooling and processes and adjust our way of thinking to get there. Most devs will jump across to anything useful the second it’s ready, I would think

> Meanwhile I'm watching a community of mostly young people building and using tools like copilot, cursor, replit, jacob etc and wiring up LLMs into increasingly more complex workflows.

And yet, I don't see much evidence that software quality is improving, if anything it seems in rapid decline.

The ageism in this comment just serves to further undermine the unsubstatiated claims that are made, a common trait of the crypto-bro-migrated-to-ai-bro movement.

This is completely different: With an automobile (back then) you still needed ad driver.

This replaces the most human occupation of all: thinking. So young people go ahead and steal the whole open source corpus that they did not write. And are smug about it.

If your projections of progress are true, at least 90% of the people here who praise the code laundering machines will be made redundant.

Every single one of these discussion, at some point, devolves to some version of

- <LLM Y> is by far the best. In my extensive usage it is consistently outperforms <LLM X> by at least 2x. The difference is night and day.

Then the immediate child reply:

- What!? You must be holding it wrong. The complete inverse is true for me.

I don't know what to make of this contradiction. We're all using the same 2 things right? How can opinions vary by such a large amount. It makes me not trust any opinion on any other subject (which admittedly is not a bad default state, but who has time to form their own opinions on everything).

Lots of possibilities here.

People are learning to prompt LLMs in ways that produce better results for their LLM of choice, so switching to another one they find their approach no longer works as well.

Or.. LLMs have different personalities in terms of output; some being more or less direct/polite than others, or sounding more or less confident; and that is causing people to perceive a difference that in terms of factual answers may not be different.

Or just personal preference masquerading as intelligence - a classic among software engineers.

Most of the LLM quality discussions have a shelf life measured in weeks. The companies involved are leap frogging each other with model updates and tweaks every few weeks.

Try it yourself. I'm getting a lot of value out of just using chat gpt for coding. It's not without flaws. But I can get it to do a lot of routine stuff quite quickly. What I like about the desktop client is that a prompt is just one alt+space away. I usually just copy paste whatever I'm working on and then ask it to do stuff to it.

There's some art to the prompting and you usually have to nudge it to not be lazy and do the whole thing you asked for. It seems engineers on the other side are working really hard to minimize token usage.

I find it's increasingly the UX that's holding me back, not the model quality. Context windows are now big enough to hold a lot of stuff. But how do you get everything in there that matters? Manually copy pasting together stuff is tedious. I actually wrote a script (well, with some llm help) that flattens things in my repository into a file that I then simply attach to a conversation. Works surprisingly well.

Same reason as programming language flamewars. Coders assume their individual experience holds for every domain. Yet LLMs are good for some things, bad at others.

There are tests for LLMs coding capabilities.

However, unless one is severely worse, I don't think LLM choice matters much, but how well the tool that uses LLM is thought.

So it's not Claude vs Chat GPT vs Llama vs Qwen vs Deepseek but Copilot vs Clide vs Cursor vs Aider.

They might very well be holding it wrong. Getting decent use out of an LLM requires knowledge, skill and perseverance. It's a pretty new field and everything is improving and changing incredibly fast.

Ever heard of astroturfing?

"cuts" has to be the worse word choice in this context, it sounds like they're terminating deals rather than creating them.

Common english lexicon should cut ties with the phrase "cut a deal"

It's up there with "drops" for word swith multiple different meanings

It cuts both ways.

Came here to say that - my reaction was initially "I didn't know they even had those deals to cut them!"

"inks"

Can we change the title to “GitHub _signs_ deals with Google, Anthropic” ?

The original got me thinking it already had deals it was getting out of

To "cut a deal" is a common (American?) English idiom meaning to "make a deal".

But agree that it's better to avoid using idioms on a site that has many visitors for whom English is not their first language.

I agree, very weird choice of words.

That’s a strange usage of the word “cuts”. I thought GitHub terminated the deals with Google and Anthropic. It would be better if the title were GitHub signs AI deals instead of cuts.

https://plainenglish.com/expressions/cut-a-deal/#:~:text=Tod....

I'm assuming you're not a native speaker? (I'm not) - "to cut a deal" is a fairly common idiom that means to reach and agreement.

Award for most ambiguous headline. (“cuts” can mean “initiates” or “terminates”!)

“Cut a deal” is a standard phrase with only one meaning

I also thought that they were ending some previous deal and not creating a new one.

Same with drops

This sort of makes me sick as a software engineer with licensed code on GitHub. Am I understanding correctly that they have trained data on my code despite my license? Do I receive monetary payment from the deal? Or have I misunderstood this?

How is that different to a human reading your code and building up their experience? Is reading code now also covered by a license? It does not reproduce your code 1:1.

I'm pretty sure that to upload your code to GitHub, you have to agree to their terms of service, which potentially give them all sorts of rights to your code. This creates a special relationship between you and GitHub that is separate from the license you assign to your project.

The reason here is Microsoft is trying to make copilot a platform. This is the essential step to moving all the power from OpenAI to Microsoft. It would grant Microsoft leverage over all providers since the customers would depend on Microsoft and not OpenAI or Google or Anthropic. Classic platform business evolution at play here.

I'm sure there are multiple reasons, including lowering the odds of antitrust action by regulators. The EU was already sniffing around Microsoft's relationship with OpenAI.

I think the reason here is that Copilot is very very obviously inferior to Cursor, mostly because the model at its core is pretty dumb.

A case where "cut" is its own antonym, and its unclear which sense is meant from the headline alone.

Cutting Deals and Striking Bargains: The History of an Idiom

https://web.archive.org/web/20060920230602/https://www.csub....

By way of "Why do we 'cut' a deal?" https://english.stackexchange.com/q/284233

---

"Cuts " ... leads to the initial parsing of "cuts all ties with" or similar "severs relationship with".

When with additional modifiers between "cuts" and "deal" the "cuts deal with" becomes harder to recognize as the "forms a deal with" meaning of the phase.

I just had the same problem and thought there was a deal that was ended now.

Don’t think I’ve ever seen the word “cut” used with “deal” in a negative sense. Cutting a deal always means you made a deal, not that one ended.

GitHub sublates AI deals with Google, Anthropic

I don’t like using AI assistants in my editor; I prefer to keep it as clean as possible. So, I manually copy relevant parts of the code into ChatGPT, ask my question, and continue interacting until I get what I need. It’s a bit manual, but since I use GPT for other tasks, it’s convenient to have a single interface for everything.

Paywall; can't read legally.

You can read legally, just need to pay first.

Whatever the motive, it's probably a smart move

I replaced ChatGPT Plus with hosted nvidia/Llama-3.1-Nemotron-70B-Instruct for coding tasks. Nemotron produces good code. The cost different is massive. Nemotron is available for $0.35 per Mtoken in and out. ChatGPT is considerably more expensive.

Just kidding. Qwen 2.5 Instruct is superior. Nemotron is overfit to pass benchmarks.

Every thread about AI coding turns into a therapy session for those devs among us who apparently derive a lot of their self worth from being able to write code.

Every time I mention using AI at work the same people put on their nitpicking glasses and start squinting.

It's getting to be embarrassing. I just wish those who choose to remain ignorant about these technologies would just listen to what other people are doing instead of raising spectres.

You mean "Microsoft" cuts deals with Google and Anthropic on top of their already existing deals with Mistral, Inflection whilst also having an exclusivity deal with OpenAI?

This is an extend to extinguish round 4 [0], whilst racing everyone else to zero.

[0] https://news.ycombinator.com/item?id=41908456

Reply on news.ycombinator.com