Hacker News Reader

Comment by peab | Hacker News Reader

gemini flash is probably the best model for visual tasks right now. they also make it really easy to ingest videos

Crazy it's still the only video understanding endpoint. It's what I use it for and no other model even offers a competitor.

You probably can't build such a model without unlimited access to YouTube and Google has been tightening the screws on that over the years pretty systematically.

to be fair, all it's doing is sampling the frames and maybe doing transcription, if I'm not mistaken. So you can do it with the other models too, you just need to sample the frames yourself and do the transcript yourself

it does this at a variable rate of frames which you can set - not sure if it is transcribing or natively understanding audio, but I think it's the latter since it is much faster than most transcription models I am aware of

Regardless you are right - I can roll my own.... but why

Yes, was going to say I use it exclusively for video and audio. The ability to give it a YouTube link through the API and ask questions about it is awesome

Also best at OpenSCAD, seemingly for the same reason, at least in terms of "iterate on a design, comparing visual output to target".

Are you manually rendering previews of its OpenSCAD output to create images for it to review, or do you have a workflow that automates that?

Personally both, i use openscad with opencode, and paste images but it is pretty often the modell decides by itself it wants to see a render and uses the render shell commands to get a image to look at.

I've built some tools using AI. A custom GUI that has "copy context" and "copy image" buttons, to make prompting easy, and the camera position is persisted to disk on each change so that the agent can run a command to get a screenshot of it.

I'm thinking about inlining an AI chat window directly - I guess I might fire off a prompt to do that right now.

+1 gemini models where really the only ones fullt grasping spatial reasoning even compared to opus (at least when i last cared to check it)

I've found Sol excellent at OpenSCAD.

But I don't do "compare visual output to target" as much as 3D reasoning type tasks (eg: "Build a G1 curve where the -X face meets the +Z face" etc)

"G1 curve"? Nice, cool to have new vocabulary for the spell book. Thanks.

Yes! Discovered that on a random YouTube video on industrial design.

Ah, multimodal is a great point. I'll need to try that some time.

It is also very good and cheap for computer use.

Reply on news.ycombinator.com