Comment by brabel | Hacker News Reader

Comment by brabel | Hacker News Reader

Parent Full thread

I am not a genius but have a couple of decades experience and finally started using LLMs in anger in the last few weeks. I have to admit that when my free quota from GitHub Copilot ran out (I had already run out of Jetbrains AI as well!! Our company will start paying for some service as the trials have been very successful), I had a slight bad feeling as my experience was very similar to OP: it's really useful to get me started, and I can finish it much more easily from what the AI gives me than if I started from scratch. Sometimes it just fills in boilerplate, other times it actually tells me which functions to call on an unfamiliar API. And it turns out it's really good at generating tests, so it makes my testing more comprehensive as it's so much faster to just write them out (and refine a bit usually by hand). The chat almost completely replaced my StackOverflow queries, which saves me much time and anxiety (God forbid I have to ask something on SO as that's a time sink: if I just quickly type out something I am just asking to be obliterated by the "helpful" SO moderators... with the AI, I just barely type anything at all, leave it with typos and all, the AI still gets me!).

Have you tried using Ollama? You can download and run an LLM locally on your machine.

You can also pick the right model for the right need and it's free.

I’m using ChatGPT4o to convert a C# project to C++. Any recommendation on what Ollama model I could use instead?

The one that does not convert C# at all and asks you to just optimize it in C# instead (and to use the appropriate build option) :D

I’m converting game logic from C# to UE5 C++. So far made great progress using ChatGPT4o and o1

Do you find these working out better for you than Claude 3.5 Sonnet? So far I've not been a fan of the ChatGPT models' output.

I find ChatGPT better with UE4/5 C++ but they are very close.

Biggest advantage is the o1 128k context. I can one shot an entire 1000 line class where normally I’d have to go function by function with 4o.

Yes. If the AI is not integrated with the IDE, it's not as helpful. If there were an IDE plugin that let you use a local model, perhaps that would be an option, but I haven't seen that (Github Copilot allows selecting different models, but I didn't check more carefully whether that also includes a local one, anyone knows?).

It’s doable as it’s what I use to experiment.

Ollama + CodeGPT IntelliJ plugin. It allows you to point at a local instance.

I also use Ollama for coding. I have a 32G M2 Mac, and the models I can run are very useful for coding and debugging, as well as data munging, etc. That said, sometimes I also use Claude Sonnet 3.5 and o1. (BTW, I just published an Ollama book yesterday, so I am a little biassed towards local models.)

Thanks for the book!

> (Github Copilot allows selecting different models, but I didn't check more carefully whether that also includes a local one, anyone knows?).

To my knowledge, it doesn't.

On Emacs there's gptel which integrates quiet nicely different LLM inside Emacs, including a local Ollama.

> gptel is a simple Large Language Model chat client for Emacs, with support for multiple models and backends. It works in the spirit of Emacs, available at any time and uniformly in any buffer.

https://github.com/karthink/gptel

This can use Ollama: https://www.continue.dev/

> If there were an IDE plugin that let you use a local model

TabbyML

I'm genuinely curious but what did you use StackOverflow for before? With a couple of decades in the industry I can't remember when the last time I "Google programmed" anything was. I always go directly to the documentation for whatever it is I'm working for, because where else would I find out how it actually works? It's not like I haven't "Google programmed" when I was younger, but it's just such a slow process based on trusting strangers on the internet that it never really made much sense once I started knowing what I was doing. I sort of view LLM's in a similar manner. Why would you go to them rather than the actual documentation? I realize this might sound arrogant or rude, and I really hope you believe me when I say that I don't mean it like this. The reason I'm curious is because we're really struggling getting junior developers to not look, everywhere, but the documentation first. Which means they often actually don't know how what they build works. Which can be an issue when they load every object of a list into memory isntead of using a generator...

As far as using LLMs in anger I would really advice anyone to use them. GitHub copilot hasn't been very useful for me personally, but I get a lot of value out of running my thought process by a LLM. I think better when I "think out loud" and that is obviously challenging when everyone is busy. Running my ideas by an LLM helps me process them in a similar (if not better) fashion, often it won't even really matter what the LLM conjures up because simply describing what I want to do often gives me new ideas, like "thinking out loud".

As far as coding goes. I find it extremely useful to have LLMs write cli scripts to auto-generate code. The code the LLM will produce is going to be absolute shite, but that doesn't matter if the output is perfectly fine. It's reduced my personal reliance on third party tools by quite a lot. Because why would I need a code generator for something (and in that process trust a bunch of 3rd party libraries) when I can have a LLM write a similar tool in half an hour?

> Why would you go to them rather than the actual documentation?

Not every documentation is made equal. For example: Android docs are royal shit. They cover some basic things, e.g. show a button, but good look finding esoteric Bluetooth information or package management, etc. Most of it is a mix of experimentation and historical knowledge (baggage).

> Not every documentation is made equal.

They are wildly different. I'm not sure the Android API reference is that bad, but that is mainly because I've spent a good amount years with the various .Net API references and the Android one is a much more shiny turd than those. I haven't had issues with Bluetooth myself, the Bluetooth SIG has some nice specification PDF's but I assume you're talking about the ones which couldn't be found? I mean this in a "they don't seem to exist" kind of way and not that, you specifically, couldn't find them.

I agree though. It's just that I've never really found internet answers to be very useful. I did actually search for information a few years back when I had to work with a solar inverter datalogger, but it turned out that having the ridicilously long German engineering manual scanned, OCR processed and translated was faster. Anyway, we all have our great white whales. I'm virtually incapable of understanding the SQLAlchemy documentation as an example, luckily I'll probably never have to use it again.

I believe you don't mean to be rude, but you just sound completely naive to me. To think that documentation includes everything is just, like, have you actually been coding anything at all that goes just slightly off the happy path? Example from yesterday: I have a modular JavaFX application (i.e. it uses Java JMS modules, not just Maven/Gradle modules). I introduced a call to `url()` in JavaFX CSS. That works when running using the classpath, but not when using the module path. I spent half an hour reading docs to see what they say about modular applications. They didn't mention anything at all. Specially because in my case, I was not just doing `getClass().getResource`... I was using the CSS directive to load a resource from the jar. This is exactly when I would likely go on SO and ask if anyone had seen this before. It used to be highly likely someone who's an expert on JavaFX would see and answer my question, sometimes even people who directly worked on JavaFX!

StackOverflow was not really meant for juniors, as juniors usually can indeed find answers on documentation, normally. It was, like ExpertsExchange before it, a place for veterans to exchange tribal knowledge like this. If you think only juniors use SO, you seem to have arrived at the scene just yesterday and just don't know what you're talking about.

Reply on news.ycombinator.com