GitHub Models: A new generation of AI engineers building on GitHub | Hacker News Reader

GitHub Models: A new generation of AI engineers building on GitHub

github.blog

100 pointsdberhane14 comments

Best explanation I found was this video: https://www.youtube.com/watch?v=WiBB8Lsgl7I

It's three things, all currently waitlisted:

1. A new model playground (like the ones by OpenAI and Anthropic) that lets you try prompts against a large range of models

2. Client libraries for the same using what looks like a universal API proxy, though I think these are existing Azure libraries

3. A "gh models run" command (similar to my LLM CLI tool) that lets to interact with models on the command line

So it's mostly a wrapper around existing Azure stuff that makes it MUCH easier to use, because GitHub are better than Azure at things like authentication systems that don't strike fear into your mortal soul.

I hate to bag on Azure because competition in cloud is critical and they keep at least some price pressure on AWS, but god damn is it near impossible to even log in on a new machine.

And it’s not like AWS is winning any UX competitions.

Seems like this is a sales funnel for Azures OpenAI/LLM gateway with GitHub as a proxy(?). It's a bit unclear. Regardless, I'd be pretty wary of adding either Azure or GitHub as a core dependency to any of my apps at this point with how poor uptime seems to be at both lately.

Also, the pricing is pretty shady. It seems like your GitHub PAT gives you free limited access, but if you ever want to move to a paid model, you have to transition to Azure. The shady part is that it's pretty hidden. You have to go to a specific model in the Marketplace, (for example: https://github.com/marketplace/models/azureml-mistral/Mistra...), then go to the bottom of the "Chapters" to the "Going beyond rate limits" section. There it just directs you straight to the Azure portal.

I'm not sure what the point of this is. Aren't there enough LLM playgrounds out there?

Maybe if they wanted to develop it into a useful feature it would create unique LLM benchmarks from your pull requests and issues, so you can easily test which model performs best on your codebase. Then augment that with fine-tuning to your code style, etc.

The point is to funnel users into spending money on Azure, like most of Microsoft's dev tools these days.

"And finally, go to production with Azure AI by replacing your GitHub personal access token with an Azure subscription and credential."

"We believe every developer can be an AI engineer with the right tools and training [...] GitHub is the creator network for the age of AI. [...] Just in the last year, more than 100K generative AI projects were created on GitHub. [...] In the years ahead, we will continue to democratize access to AI technologies to generate a groundswell of one billion developers."

If I've learned one thing in the software industry, then that with a sufficiently creative definition of the word 'engineer' everything is possible. It feels like every week the AI "industry" turns more into some sort of cursed influencer economy. I guess if you've got to explain to your shareholders how you're gonna make the 10 billion back you spent on graphics cards you gotta be optimistic.

I find the gatekeeping of the term engineer a much more interesting phenomena. People planning, building, launching things are engineering things.

GitHub has been running too many waitlist scams lately. Why do they need a waitlist for this? Waitlists are a scam because I'm already waiting to be approved for my last waitlist application from several months ago. Fool me once.

> One data pack costs $5 per month, and provides a monthly quota of 50 GiB for bandwidth and 50 GiB for storage. You can purchase as many data packs as you need. For example, if you need 150 GB of storage, you'd buy three data packs.

They really need to move on from this pricing structure. It's too expensive to host and distribute large binary files from GitHub.

Is this just Azure AI Studio?

My experience is that Azure OpenAI is waay behind on API and model parity compared to the same thing from OpenAI.

AOAI literally has these compliance stuff (or what they market as 'enterprise-level security' features) so they're constantly 1-3 months behind OAI.

And no. This is not Azure AI Studio. You can't get a decent free trial working in it without getting through a pack of resource creation wizards.

So the tl;dr is they're now competing directly with Hugging Face?

Yes and no. It’s indeed a playground to test out various models and gives you an endpoint to play with it, but it’s not that developers can upload their own custom models. Instead, it’s currently only a curated library of certain popular models like those from OpenAI, Microsoft, and Meta.

(I don’t work at GitHub but was quoted in the article).

I'd store full datasets on GH if they made LFS suck less lol

Reply on news.ycombinator.com