Hacker News Reader

Comment by ilaksh | Hacker News Reader

Is anyone else here building AI programming services based on models like this? I see a lot of comments saying the models can't do much programming. But I just suspect there must be a silent contingent that is also working on services like that. And maybe less likely to promote the abilities of these models because it encourages competition.

View on HN

We are at Codeium (codeium.com)! Not the SantaCoder model specifically, but the same types of LLM architectures. We've started with AI-based code autocomplete, but we think there is a lot more we can do.

We wrote up some of our learnings so far in @swyx's blog recently: https://lspace.swyx.io/p/what-building-copilot-for-x-really

What I would really like is something I saw someone talking about here; I'd like the editor to brighten text it finds "unexpected" which could immediately alert to bugs, or to the fact that the code I'm writing looks weird in some way and might either be restructured or accompanied by a comment.

Yep, these kinds of applications are on our mind! We consider autocomplete to be the "baseline" task since there are plenty of benchmarks and research to compare our model's performance to, but there's lots of things like highlighting code, upgrading to new libraries/conventions, etc that we can do with a good base model.

my unsolicited advice: pick an X. what is the one best use case for this other than code? law? finance? focus on that vertical. if you have no idea what that could be or if that market is too small, you're already in trouble.

I don't use anticomplete at all. What I would like is something that can take my current, bad code and style transfer it into proper, modern code. best case, take code as I write it naturally and confirm it to the style guide of my organization.

We're building tools like this at Grit: https://www.grit.io/

These kinds of models are particularly good at repetitive, boring work like refactoring legacy code and completing framework migrations. Unlike Copilot, we've specialized specifically in these areas and completing them end-to-end (instead of just sitting in the IDE, we open already-verified PRs).

May I ask what model you are using?

We use a few depending on the task (Codex, fine-tuned T5, Bert models, etc.). Constantly experimenting with different variations. Since we focus on solving narrower problems in more depth, it leaves more room for optimizing accuracy.

Have you been able to get your rate limit increased for code-davinci-002? It defaults to a very small amount.

I recommend trying davinci-003 as well.

Yeah, been messing with that a lot.

I've been pretty impressed with chat gpt generating working implementations of various algorithms in different languages. Crucially, it actually knows about algorithms. I was trying to get it to generate some algorithm for calculating concave hulls the other day and ended up learning a thing or two about various algorithms for that in this space. Almost but not quite worked for my use case. It seems limited in the amount of code it can generate in one go. But otherwise, I was pretty impressed.

So, we're not that far off from basically pair programming with an AI that will do most of the boring/tedious work we currently do manually. Something like chat gpt integrated into an IDE could be useful right now.

Copilot or Codium

Yes we are incorporating into Graphistry as part of how we help sec/fraud/misinfo/crime/etc analyst teams investigate their data. Our platform does all sorts of GPU visual graph analytics & graph AI once data gets loaded in, and as part of our visual playbooks automation layer, this helps users make automations and fancier queries. Think Splunk, Spark, Neo4j, ... .

IMO tough question of who can do codegen as a scalable standalone startup, but that's ok. Pretty darn easy & useful for many productivity platforms like ours where it's just a super nice feature as part of delivering a broader magical experience.

Related: we are hiring a k8s/pydata person, ideally who has need a user & builder of investigation platforms, as we are working w co's like Nvidia to bring this kind of thing to some pretty major enterprise & gov teams. See gdoc linked on our careers page.

There's replit. Constantly announcing new features around such models. They'd introduced "ghostwriter" a while back and yesterday or so they announced ghostwriter chat.

Yes, we are building something that is somewhat like ILP/IFP (and other tried and tested but non scalable techniques) with the search space reduced by using modern ML language models. And indeed; the thing that works best in our system has not been done in the open yet. Of course we have no idea if it's viable for the masses; maybe if people see how well it works.

We built a semantic code search CLI tool (fully local and open source) using a similar model that I tuned https://github.com/sturdy-dev/semantic-code-search

Yup, as part of another thing. ML assisted everything is here to stay.

I've been messing around some. Flan-T5 generates surprisingly close stuff occasionally for simple prompts like #square x or #sum the elements in the list.

Reply on news.ycombinator.com