Hacker News Reader

farisallafion Show HN: I trained a language model that thinks the capital of Japan is Paris

thanks a lot for the support!!

yeah for the next training run im planning on testing that out extensively before putting into training

farisallafion Show HN: I trained a language model that thinks the capital of Japan is Paris

haha yeah I wrote this fully by hand

the only LLM's input into this was as a peer reviewer

I guess I have spent a bit too much time with the models these days

(for the record I mostly use Fable 5)

farisallafion Show HN: I trained a language model that thinks the capital of Japan is Paris

Hey! Thanks so much for the thoughtful reply...

totally agree with your points on benchmarking and research

also while the website ui was supported by an LLM, all the actual writing was by me(a human) whether it looks somewhat AI generated(well I guess I'm a little bias considering I wrote it) I think I've been speding too much time with the models these days

in all genuinely appreciate the thoughtful insights on my researcH!

farisallafion Show HN: I trained a language model that thinks the capital of Japan is Paris

Author here. This is hr-diffuse-1-nano: bidirectional Mamba-2 + LLaDA-style masked diffusion at 288M params, cross- arch distilled from SmolLM-135M, trained on 1xh100 for ~$500.

The honest headline results: 14% infill recovery where autoregressive models score ~0 (they can't condition on text after the blank), 7.5% repetition-loop rate vs 37.5% for the teacher, and a genuinely negative result I think is the most useful part: six different self-correction methods all failed at this scale, while a 300k-param external critic head detects errors far above chance. Small models don't doubt; they rationalize.

Weights are open: https://huggingface.co/devnull37/hr-diffuse-1-nano. Happy to answer anything about the architecture, the failed runs.