HNHacker News
TopNewBestAskShowJobs

rao-v

1,253 karma · joined October 26, 2024

v@inferencing.net
submissionscomments
rao-v··on Every Fucking Website (2020)
Genuine curiosity - is the pop up vaguely factual or sort of randomly generated?
rao-v··on Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
Native dflash support on day 1 helps a lot! High quality speculative decoding speeds up a lot of agentic work.
rao-v··on Twenty Years of Pandoc
I have at times mused that writing the internal state of pandoc to disk (yes 13th standard etc) would be the most interoperable file format
rao-v··on A Trampoline
Wait it only applies in 14 states? Why do people act like it’s true of the entire country?
rao-v··on How is the Bun rewrite in Rust going?
I don't quite understand the focus on the token cost of this rewrite. Obviously, we should examine if this rewrite is good, effective, good for the product etc., but the token cost seems ... not important?

The marketing value of this to Anthropic (if Anthropic even cares, this might just be the Bun team selling past the close) is to show that such a rewrite is possible and delivers engineering value. If exactly this project is $800K today, it'll be $200K and then $80K soon, so it's not so important to the story that it's cheap, just that a big "cool" rewrite is possible, and delivers velocity to the buisness.

rao-v··on Running a 28.9M parameter LLM on an $8 microcontroller
This is a really neat use of the per-layer embedding trick. It's also worth noting that there viable TTS models that are ~20-30M param, so it might mean you can have a ESP32 with no network access read stuff out to you in near real time!
rao-v··on Everyone should know SIMD
Food inflation has been moderately high but egg prices were a mix of weird short term events and umm price fixing (look up the recent case).

They are back to pre 2022 levels now! https://fred.stlouisfed.org/series/APU0000708111

rao-v··on Everyone should know SIMD
Good to know!
rao-v··on Everyone should know SIMD
A genuinely funny and good point!
rao-v··on Everyone should know SIMD
It seems like such a tempting gap though. The sort of thing you’d think in 2015 would be an obvious capability of 2026 languages!
rao-v··on Ghost Cut – Or why Cut and Paste is broken everywhere
It’s not unreasonable to suggest that text editors need a nice atomic move (but personally I don’t see the need for it)
rao-v··on Everyone should know SIMD
It distresses me that we don’t have a language that can do a best effort parallelization of arbitrary loop like code across SIMD, multiple threads, multiple cores and GPU with a small directive.

I don’t need it to be optimal, just … handy as an option!

The last time I brought this up here, folks offered a bunch of options that don’t quite do this, and the best candidate was this 15 year old compiler project that is Intel specific!

https://ispc.github.io/

Could some programming language nerd build this?

(While you are at it give me a clear idiomatic way to pay the cost to switch from array of structs to struct of arrays)

rao-v··on Claude Fable produced a counterexample to the Jacobian Conjecture
I mean you’d probably just generate random low descriptive length f(x,y,z), check for a const determinant then poke for invertibility.

Be fun to ask Fable to write a search program to find more counter examples using only early grad theory to guide the search.

rao-v··on Claude Fable produced a counterexample to the Jacobian Conjecture
This is so unreasonable! As @__alpoge__ himself notes this is classic crank graveyard territory and yet the counter example is something a grad student in 1997 could have found w a ~3 day computer search. Wild!
rao-v··on Judge a book by its first pages
The world needs more delightful websites that do one clever thing well like this. Turns out I need to read Percival Everett!
rao-v··on Governments, companies, nonprofits should invest in free, open source AI [pdf]
Pretty sure the labs subsidized the well known benchmarks w free tokens for evals
rao-v··on Claude Code: Anatomy of a Misfeature
Codex legit does a great job of this with it's auto-review feature. Does a good job of understanding what permissions I'm implicitly giving with the request, and where it needs to actually ask me to grant permission.
rao-v··on Solod: Go can be a better C
Arn't goroutines the killer feature of go? Don't see how you'd get them with this approach.
rao-v··on Governments, companies, nonprofits should invest in free, open source AI [pdf]
We really need to band together to fund / sponsor targeted inducement prizes (a la Nobel laureate Michael Kremer) for open models.

Every 6-12 months, give out $200K to the first model to hit a min threshold on a set of ~5-10 hard benchmarks (+ perhaps one secret benchmark) using a total of 16GB / 32GB / 64GB / 128GB of VRAM (at a min context length of 200K), then move the threshold up. Quantization etc. is dealers choice, it just needs to nail the benchmark on a reference machine by using exactly that much VRAM (no mapping to RAM / disk etc.)

You could crowdsource the funding, and cross subsidize by adding targeted prizes focused on corporate needs (the classic one is PDF processing benchmarks), and say that 25% of each corporate prize funding also flows into the general prize pool.

For a lot of these open-source model companies, it's less about the $s (though $200K is nothing to sneeze at), it's the clear recognition that helps their model efforts stand out, gain usage etc.

rao-v··on TS-2026-009: Insecure argument handling in Tailscale SSH permitted root access
Comically you can sign Tailnet lock from iOS, but it’s an insane workflow.

You need to generate a QR code then scan it from the signing mobile device, which opens a secret menu option to sign (fine just brings up a confirmation dialog).

Incredibly annoying but perhaps more secure vs the threat of randomly tapping at prompts

rao-v··on Mesh LLM: distributed AI computing on iroh
This really should be in the blogpost. It’s both useful info and basic courtesy to be explicit about which underlying inferencing engine you are using
rao-v··on An update on residential proxies and the scraper situation
I’m skeptical that the problem they are trying to solve is truly unreasonable bandwidth demands.

Sometimes it feels like what people want is to only serve websites and content to good normal users but not evil bad “scrapers” (because maybe maybe your content will be monetized in some nebulous way) but … you put your content up publicly on the web! That should be part of reasonable use!

EDIT: Lwn.net is perhaps not a fair target of my ire.

“There is also a desire to not impede the operation of legitimate search engines, the Internet Archive, and other such groups. Some sites may add explicit allowlists to, for example, give the dominant search engine access to the site. Such measures have the effect of further entrenching a monopoly that already serves us poorly and should be avoided. We have, thus far, succeeded in that.”

Is reasonable! Many others are not

rao-v··on Opinionated and easy Pi.dev configuration
I'm exploring the pi based options now and I like that Oh-my-pi actually adds new value over extensions (the advisor and stream interruption hooks are pretty clever).

I just wish it had a way for me to downlimit tool access (I love midsized local models, and I'd love to enable only 30% of the power for some usecases).

rao-v··on TLS certificates for internal services done right
You can change it to something a tiny bit nicer a few times!
rao-v··on Launch HN: Context.dev (YC S26) – API to get structured data from any website
This seems like a responsibly designed service, but I find it a little odd and baffling that we need such intermediaries for the average hobbyist / small project to reliably access sets of content published on the internet.

Wasn’t this the point of the web?

rao-v··on TLS certificates for internal services done right
At that point why not just use the .ts.net addresses Tailscale provides for free?
rao-v··on My thoughts on the Bun Rust rewrite
I do wonder to what degree this weird play originated from Anthropic, versus from an overeager founder selling past the close.

I can imagine Anthropic wanting to acquire Bun without the gimmicks.

rao-v··on Separating signal from noise in coding evaluations
In a real job, you would be allowed to see the test case that failed and tweak your code (or more likely the poorly written test).

If you let a modern LLM do even the first, they’d crush this specific benchmark.

What is interesting is understanding how LLMs are able to beat 70+% on this benchmark or getting some of the poorly framed questions right? Are they implicitly learning the test writers style? Are the solutions leaking into their training set?

Perhaps reassuring is that even Fable stalls out at ~72% (on the hidden set which OpenAI did not run this analysis on), so perhaps training on the bench is not happening in anything but the most indirect ways.

I care a lot because small open models can never learn idiosyncrasies like this, so I really want good ways to judge models fairly.

EDIT: Humm OpenAI is muddying the water a bit. Only 20%ish of problems are broken in ways that are unfair to the agent, 4-10% are broken in favorable ways, so the benchmark ceiling is probably closer to 80-85%

rao-v··on Mistral's Robostral Navigate: a state of the art robotics navigation model
Could you please open source this or a 4B version? I’ve been messing around w hooking up vllms to cheap robots and skipping the whole ROS stack and this would be an absolute delight to play with
rao-v··on Scheme Is a Hoot
“Spritely has been going strong for many years. I suppose Hoot was created to make Goblins [0] available to a broader public via wasm, as a reference implementation of CapTP specification”

Has extreme Curtains for Zoosha (https://amp.knowyourmeme.com/memes/curtains-for-zoosha) energy

← PreviousPage 3 of 10Next →