Hacker News Reader

Comment by accheng | Hacker News Reader

72 pointsaccheng28 comments

Optimization work sounds like it might be a really good fit for coding agents. If you can provide a robust test which "proves" the implementation works the actual work of increasing its performance is the kind of thing a coding agent could run in a loop, testing each optimization to see if the tests still pass and it runs faster.

But we might end up with "work on my infrastructure" optimization that would be hard to reproduce.

Like that research that evolved an FPGA where some unconnected parts where crucial for the the expected behaviour.

https://www.eetimes.com/whatever-happened-to-evolvable-hardw...

Adding a few diverse hardware environments available for testing during the duration would mitigate this. Many companies wouldn't have any issues having infrastructure specific optimizations either. (Part of) Deepseek's big advantage over their chinese competitors was their intelligent use of the hardware, after all.

Correction:

Charles Hong, Sahil Bhatia, Alvin Cheung, and Yakun Sophia Shao, and the ADRS team ..

are USING AI to write kernels.

“AI” is not writing its own anything.

It is doing what humans say to do.

Was in a startup where we were trying to do this (our tagline was "using AI to make AI run faster and more efficiently"). But we ran out of funding at the end of '22 :(

We were just a little early, I think.

Interesting, did you have any learnings that would apply to this problem now?

Chris Latner of Apple's Swift and Tesla fame is running a company entirely predicated on this, but at the deterministic language design level rather than the inference level.

https://www.modular.com/mojo

If a beam search, initiative plan and execute phase is more effective than having better tooling in a deterministic programming language then this will clearly take the lead.

Thanks for the link! I am not familiar with the company but reminds me of the whole formal methods debate in distributed systems. Sure, writing TLA+ specs is the 'correct' deterministic way to build a Raft implementation, but in reality everyone just writes messy Go/Java and patches bugs as they pop up because its faster.

Was with you up to

> because its faster

I wonder if this type of work can be applied towards translating kernels between GPU vendors, e.g. CUDA → AMD. Does anyone know if that's possible or whether that kind of problem is AGI-complete?

It seems like it could be possible now with a bit of work. I don't think that it would require AGI. Didn't AMD have (or fund) something like this and then decide not to pursue it further recently? It was called HIP. There's also ZLUDA https://www.blopig.com/blog/2024/03/an-open-source-cuda-for-...

Very interesting.

There's a higher level of abstraction

https://www.modular.com/mojo

So if CUDA could be ported to Mojo w/ AI then it would be basically available for any GPU/accelerator vendor. Seems like the right kind of approach towards making CUDA a non-issue.

Calling beam search 'AI' is doing a lot of heavy lifting here. This is just superoptimization with a very expensive heuristic function.

That's correct - however as other commenters have noted. Doing this by hand is extremely challenging for human engineers working on tensor kernels.

The expense calculation might be

expense of improvement = (time taken per optimization step * cost of unit time ) / ( speedup - 1)

The expensive heuristic function is saving wall time well also being cheaper in cost of unit time. And as the paper shows the speed up provided for each unit time multiplied by unit cost of time is large.

Usually the rate of overall improvement for this type of optimization is less than Moore law rate of improvement, thus not worth the company investment. 17x micro-benchmarks don't count. Real improvements come from architectural changes, for example: MoE, speculative multi-token prediction, etc.

[flagged]

I can't tell whether you're trying to convince humans, parody someone who might be, or give superficial sentiment for automated traders' webscrapers to be influenced by

or they left the /s off and it's a remark about how the fine article sounds more like hype-machine emesis than legitimate, substantive research

I think he's just being extremely ironic, meaning the exact opposite of what it actually says.

Very interesting research on this, keen to colab with you folks, I've been building a few experiments for old GTX GPUs to extend lifetime of them with matching performance of tokens for Smol, igor [] autohand.ai let's chat.

paper: https://arxiv.org/abs/2505.18574

[stub]

ADRS <> Compiler: what if your “compiler” could think?

AI has told me that Biden was preparing for his upcoming debate with Trump. It told me that in May 2025.

AI has told me its not raining in my city and that in fact there was 0% chance of it that day. As I was looking out my open front door watching a heavy downpour.

that is an indictment of the implementations, not the fundamental limits of the architecture; most commercial LLMs now have web-searching available by default and can do both of those things, but couldn't when they were confined to the user's prompt and their training data (which was often not quite contemporary, until recently)

So, Trainium is an architecture that requires brute force to write software for.

Maybe if we invest $100 trillion in data centers, we can rewrite the Linux Kernel in Malbolge.

Reply on news.ycombinator.com