150 karma · joined December 11, 2024
This is why in the long run I believe we all should aspire to do LLM inference locally. But unfortunately we just are not anywhere close to par with the SoTA cloud models available. Something like DGX spark would be a decent step in this direction, but this platform appears to mostly be for prototyping / training models meant to eventually be run on data center nvidia hardware.
Personally I think I will probably spec out an M5 max/ultra Mac Studio once that’s a thing, and start trying to do this more seriously. The tools are getting better every day and “this is the worst it’ll ever be” is much more applicable to locally run models.
Crime is down. You have been manipulated.
“Using AI for code has reminded me of this sensation… switching in and out of “driving” feels more exhausting than being 100 % one or the other.” You’re not imagining it. There’s a fair bit of cognitive-science literature on task-set inertia: every time you hand work off (human→AI or AI→human) you pay ~100–150 ms to reconstruct the mental model, plus an exponentially-longer “resumption lag” if the state is ambiguous.¹ Do that dozens of times per hour and you’ve effectively added a stealth meeting to your day. A few things that helped me when pairing with an LLM: • Chunk bigger. Treat the AI like a junior dev on 30-minute sprint tickets, not a rubber duck you ping every two lines. • Use “state headers.” I prepend a tiny recap in comments — // you own: parse(), I own: validate() — so I can scan and re-hydrate context instantly. • Declare no-AI zones. Sounds counter-intuitive, but reserving, say, test-writing for uninterrupted solo focus keeps me in flow longer overall. …have you tried anything successfully to address it? We were annoyed enough to build something. At Recontext (YC S24) we sit between your editor and whatever LLM you’re using; every AI request is automatically tagged with the diff, dependency graph, and TODO items so when you jump back in, you get a one-glance briefing instead of spelunking through scrollback. Early users report ~40 % fewer context switches during a coding session. If anyone wants to kick the tires, we’re handing out private beta invites — email is in profile. ⸻ ¹ See Monsell, “Task switching,” Trends in Cognitive Sciences 2003 — the “switch cost” math is sobering.