Scene_Cast2

IIRC the original author of the Lottery Ticket Hypothesis now disavows that idea.

One intuitive way of looking at it is like so - let's say that you have a gaussian-looking plot. You want to fit a gaussian. You have a stupid simple model where you can slide your gaussian left and right.

If your initial starting point happens to be roughly within range, great, your optimizer will take care of it for you and slide it into the correct place. If you're too far, too bad, no meaningful gradient.

Instead, neural nets give you the option to spawn a gaussian anywhere you please. In this case, no sliding is necessary, but it comes at a heavy parametrization cost.

Yes it would. Or, rather, labeling (not extra tokens).

Really neat findings.

I've personally had a line of thought where you bake in the role into the token. Basically have an embedding (same dim as token dim) for each role, add it to each token. This adds an unambiguous, unspoofable tag.

I ran this with a tiny Shakespeare model (not representative) and had a freeform embedding for each speaker. I ended up with a neat similarity map between every character. (I don't think the map was very informative for several reasons, but that's outside the scope of a small HN comment)

I think this also stems from ML being more like biology or alchemy and less like math or programming (where you can get down to the first principles, abstractions are rock solid, and non-determinism is limited in scope).

Krazam already has a video covering this exact idea.

On my (admittedly weird) setup, GPT-5.5 Pro times out.

The reading is off because the thermistor resistance also depends on applied voltage, not just temperature. LLMs couldn't get this even after feeding them multimeter voltage readings, not just ADC readings. They went into guessing much more esoteric things like ADC switched-capacitor input current, burnout-detect current sources or IDACs left enabled, board leakage, leaky cap, etc.

I'm personally heavily testing LLMs on electrical engineering problems. I'm finding that it's not meaningfully better at figuring out what's up than the other models.

To give you an idea - here's a very abridged summary of one sample question (originally a full paragraph): I have a voltage divider with a precision resistor and a thermistor, my voltage reading is off by 17%, where's that coming from. None of the models I tested (including Opus 4.8 and Fable 5) could figure it out.

The dev boards are already up for sale. I'm personally looking forward to the modules being stocked on LCSC, no idea when though.

This is the PCPartPicker chart that I monitor: https://pcpartpicker.com/trends/price/memory/#ram.ddr5.5600.... - $900 for 2x32GB, used to be $200 a year ago.

Just beware that for some people (myself included), it causes stomach issues (quite intense in my case). There are mitigation strategies (slowly build up the dose, use more water, take with food, split up the doses across meals, and consider using the less studied HCL variant).

I was sort of hoping that they were bootstrapped or at least non-VC funded. I'm wary of them introducing consumer-unfriendly revenue-generating schemes.

Exactly the same thoughts here (I've been looking into FCC part 15 myself too). And IIRC nRF has some pre-cert stuff to avoid going through the full gauntlet.

I'm guessing he's using the fact that dev boards are excepted (as opposed to final products). Somewhat unfortunate though, as these do end up in a lot of people's boards.

I get my USB-C connectors at around $0.08 at low volumes (LCSC).

I remember Last.fm's value proposition was 1) discovery and 2) community. (1) is (mostly, for most people) covered by "feed" algorithms of Spotify and YouTube.

I wonder how they're going to position themselves now.

I'm surprised that Perplexity isn't mentioned in the article or on HN. It has replaced Google for all but the most trivial queries. It runs circles around Google for finding anything niche or underspecified.

I use it through OpenRouter - I love how the pricing is per search and isn't a subscription.

There is one very clear example that I ran across due to the reasons outlined in the article. If you have a wavelet and you're trying to slide it around to make it fit, that will fail spectacularly. There are lots of problems that boil down to basically the above.

The neural net answer is being able to spawn a wavelet at any position, as opposed to tweaking the position of an existing one.

Aiming for $350 at launch.

This project doesn't have an online presence yet, just shoot a message to the email in my bio.

If anyone is interested, I'm actually in the process of launching a similar board. Much lower noise (by about 5x, custom AFE), tuned for load cells, 4 simultaneous-sampling channels.

Something that deeply frustrates me, as someone who did R&D on model architectures, is how similar the modern LLM model architectures are to GPT2.

(This is a bit disingenuous, as lots/most of work is spent on the scaling and training side of things.)

What I'm surprised by is that cars are chock-full of ornate, unique parts (cupholders are a good example).

I would have imagined that car infotainment controls would be a small fraction of the BOM, so I've been wondering if it's not really a cost thing. Sort of like small phones or 3D TVs from the early 2000's.

I found that having a positive pressure setup with an intake filter practically eliminated the dust buildup on my fans.

Why can't LLMs be conscious for their short 1-million-token input lifetime? (To be clear, I don't think they're conscious, but for different reasons.)

Oh interesting, I've heard of EMerge but haven't given it a try yet. Sounds like it's solid enough to be useful?

I recently had to disable their Chrome extension because it made the browser grind to a halt (spammed mojo IPC messages to the main thread according to a profiler). I wasn't the only one affected, going by the recent extension reviews. I wonder if it's related.

Chunk up your 128D embedding into 16 8D pieces, then quantize using the optimal packing points. Performs worse than k-means in my experience.

Hasn't this been the case for a while? I vaguely remember using the optimal e8 packing to try to get denser product quantization for vector embeddings.

I've found that having LLMs work with mermaid diagrams makes describing and modifying circuits less annoying.

Out of curiosity, how many tokens are people using? I checked my openrouter activity - I used about 550 million tokens in the last month, 320M with Gemini and 240M with Opus. This cost me $600 in the past 30 days. $200 on Gemini, $400 on Opus.

Have you tried the latest (3.1 pro) Gemini? In my experience, it's notably better for a similar type of problems than Opus 4.6. However, I don't really use OpenAI products to compare.