For those running local models on macOS with Apple Silicon, how does Magnitude differ from what Apple's own first-party Core AI now does during its "specialization" procedure, wherein it performs some kind of "model optimization and conversion" (vis-a-vis AOT compilation) into a Core AI "compiled model" file that is optimized for Apple Silicon (leveraging custom Metal 4 kernels, or so Apple says), per WWDC labs from this year that discuss this? [0]
This just makes me think about optimizing model weights to run as 'close to the metal as possible' (i.e. within or 'just above' a UEFI boot environment, like the NightRun project), so that there isn't any OS-level overhead either. If we're optimizing, let's optimize! Curious though, maybe the OS doesn't impose much of a burden here? Open to hearing what other tinkerers think...
Are you saying that 6.1 is based on a new pre-train base model? My intuition would lead me to believe that Astra (GPT-6) was the new pre-trained base model, and 6.1 was a post-trained fine-tune that came out a short time after, but curious to hear if I'm wrong about that..
Not to mention you could at that point burn/etch the weights into silicon directly, and have models as 'ROM cartridges' that could perform at thousands of tokens/second, enabling entirely new use-cases.
Super cool, thanks for making this! I've been there with you on the perfectionism. Glad that 'good enough' won this time, and we're all the better for it. Keep going!
The chessbot analogy is compelling but it does smuggle in assumptions about its generalizability. I'm partial to how so far the history of AI progress has shown that digital neural networks are capable of generalizing more and more, which would allow one to draw a stronger parallel between chessbot domination to AGI domination. Regardless, I have the presence of mind to admit that there's still a gap there in terms of how humans might be doing something 'extra' that doesn't make these systems analogous, and hence the whole argument breaks down. Despite that, there are numerous other ways to envisage harms that, if not leading to outright extinction, still lead to really crappy futures (for both humans and AI) that warrant caution and coordination in developing this technology, neither of which we've seen too much of so far. (The actual history has been "No body can be trusted with this powerful technology except me!" "No, me!" "No, ME!" etc) It makes one really want to hope that decentralized bottoms-up processes of 'intelligence diffusion' can help blunt negative effects, but I'm skeptical of that, taking the long view on what intelligence has done to various groups on our planet so far.
Whenever I raise these same concerns of skill atrophy and de-centering of humans from critical functions to friends in the tech industry (that I also work in), I'm almost always met with some version of:
> "But how is this any different from when Socrates argued that written texts and books would harm human memory and true wisdom? Clearly he was wrong - books are a net a positive, and now this argument is just engaging in the same kind of fallacious thinking. Collective intelligence will inevitably evolve into more advanced forms, and we have to learn to let go of this 'human importance' aspect."
I genuinely wonder what they think about their own bodily functions, since they don't seem so concerned with being 'human'.
One could argue that corporations as "slow AI" (per the Charles Stross interpretation, where these organizations are the original 'paper clip maximizers' that optimize a given objective reward function) have already been doing this with the proliferation of microplastics and other environmental harms, of which we are just now seeing show up in global fertility data. Digital neural networks as AI are just another form of an 'unhuman entity' operating at planetary scale, capable of even more rapid destruction.
Reminds me of what happened once most people and households transitioned to individually-carried cellphones as opposed to the 'house phone' (i.e. landline). You used to have to call 'the house' before you got routed to someone else within it, and that would often be cause for catching up with a person incidentally, and now that's just, totally gone! A great example of the trade-offs to navigate when improving systems for efficiency.
Reminds me of this post from earlier in the year (from a VC, Michael Dempsey) titled "VC-Backed Startups are Low Status"[0], which I'm inclined to agree with, having been in the industry for over 15 years now and witnessing how the vibe shifted from "techies are good" to "techies are bad" over this period.
I graduated into a tech workforce that was a celebrated part of society (i.e. "high status") to one that is decidedly not (and I'd say it's grown to deserve this disrepute).
The first main wave of the vibe going negative (at least at the heart of the 'imperial core' in the SF Bay Area) was around ~2013/14 (the 'tech co bus protests'), then again around ~2018 ("don't call SF General 'Zuck General'"), and now it's kicked into a much higher gear during this current AI wave. It spans big tech co's to startups to everything in between. Anil's post speaks to this too, when he says:
> Politicians and media still look at VC as if it works like it did 10 or 20 years ago, and cheer them on ... when their primary goal is concentrating power and wealth
It's not just VC per se, but the 'managerial class' within tech rotted into mostly career-climbing types that were a far cry from impassioned creative technologists aiming to 'do good' with tech. It became the same status-bound competition you'd find on Wall Street and elsewhere (which the Dempsey post describes well).
I have a mentality and overall life orientation that is aligned with Anil and celebrates the open web, public interest technology, and so forth, and many of my peers in tech (often from elite universities and backgrounds) look at me as a strange creature. I'll bring up the need to increase awareness about Public AI and boosting AI literacy among citizens, and I hear, "Wow, you like, really care about like, people. That's so interesting." It's unbelievable, I wish I was kidding.
Thanks for writing this Anil. I wish for better days.
For all the people here alleging that AI companies will train on your data even when you as a user explicitly opt out of training and their terms say they will respect that, etc - do you also believe that within a few years of them having harvested your data, you could perform "knowledge probing" on their models by prompting various questions that determine if they can near-verbatim reproduce your unique data, and then have enough other people do the same that you can then just launch a class-action lawsuit? Because if not... I've got a startup idea for you.
Newport does tend to be more level-headed, but honestly, the people that have gotten AI 'right' so far (in the true technical sense of how scaling laws held, etc) had their strong convictions set back in the mid-2010s, when Newport would've been far more skeptical. At least Newport's shown that he's capable of updating, but in general, he hasn't been someone I'd value for their predictive power.
Doesn't it say something about our society that (a) it's known (at least among smart people, however 'smart' is defined) that AI is going to take away jobs, and (b) the way its treated is that people gather to online discussion forums to worship someone saying the thing that will replace their jobs isn't going to do it (but in actuality, eventually will)? In other words, who's actually doing anything to help the people out who will lose their jobs??
Not sure the larger point you're trying to make (maybe we're on the same page?), but it's a fact that AI has come from heavy long-term investments by the academic and public sectors, yet private companies are accruing most of the gains
From my hazy recollection from law school, the US appeals courts are called "circuits" because back in the day, they weren't actual buildings but rather judges that would go around town-to-town within their jurisdiction in a horse-drawn carriage 'in a circuit' (that corresponded to a certain geographic area). The name just stuck even after they were permanently enshrined into physical structures (each corresponding to a given region).
Spend a lot of time thinking about why individuals shouldn't trust large AI cos, now it's interesting getting to think about how the AI companies themselves don't trust each other. Capitalism is viciously cold and uncaring.
Agree with rewriting with Gemini, but I'd characterize its output as very "neutral" and "encyclopedic", rather than "human" (contra Claude which is as of recent models "trying-too-hard-to-be-human"-sounding).
I had a student in my class propose an "AI for emergency response" that leveraged local models. Not a far stretch from what we see here with CoMaps, plus a small Gemma or similar low-parameter model running on-device to help with anything required at the 'last-mile' (beyond just mapping/navigation) during an earthquake, fire, flood, etc.
My experience as a practitioner and educator in the space lead me to think it might be an issue of how AI cannot be easily perceived at a 'classical level' by most humans. In other words, people are 'far from the metal' when using consumer AI tools, and that leads them to develop the wrong understanding about it. When I provide a demo of e.g. local AI, say in LM Studio showing the console of it rapidly flashing through thousands of words in just a few seconds, and my machine heats up and the fans spin, the 'theatrics' of it, the very real-time feedback from the system, make people correctly update about what the tech is capable of, how it works, etc (despite what they may have heard online cranks say to the contrary). But I am only one person, and there is only so much of that I can do on my own that will 'scale' in time... (and this is to say nothing about severe deficits in peoples' understanding of how weights are not verbatim representations of data, how pre-training vs post-training works, the models as amnesiacs (and hence 'one-way single-purpose conversations'), how context/memory works, context rot, etc - and hence all the 2nd and 3rd order effects that can arise from such a paradigm, e.g. unintended consequences from agent swarms, etc)
What do you recommend people who are technically inclined enough to participate meaningfully here on HN, but do not work at the labs and cannot assist in that capacity, do to help the broader public understand this technology better and mitigate potential risks (by e.g. ‘up-leveling everybody’ through AI literacy etc and other sorts of collective defensive efforts)?
These discussions always gloss over the reliability of delivery (i.e. is your email making it to the intended inbox or getting squashed into spam, or even shot down before making it to the recipient), especially when custom domains are used (which I suspect is often the case with Fastmail). As much as I want openness, independence, portability, etc of my personal digital stack, sometimes "just getting the work done" takes precedence, and email is one of those areas if not THE area where that matters most online.
I found myself so strongly endorsing almost every remark and impression in this post - are we the same person?? AI truly is dual-use: it can lead to immense speed ups on 'research rabbit-holes' that used to take a lot of manual work and Google-fu, or it could also just waste a ton of time generating too much output to make meaningful use out of (I almost always now invoke a text expansion snippet like "What's the 2-paragraph-block TLDR of the previous response?").
It's always looming in the mind of an overachiever-type person such as myself that 'AI could search and synthesize better', and so it has a very seductive siren song for almost all tasks.
The current moment with AI does very much feel like the early days of atomic energy where people had all kinds of now-crazy-seeming use-cases for the atom, and this showed up again with at-home microwaves, as seen in that "Microwave Cooking for One" book from the 1980s (see: https://www.lesswrong.com/posts/8m6AM5qtPMjgTkEeD/my-journey...).
I do think with new technologies it's important to try many different forms and applications of it before finding their true practical utility, and this is no different. Maybe we're collectively (at least within the HN crowd) finally feeling the "cool-down of the mania" and starting to shift to more realistic uses of AI (at least until there might be another big capabilities jump...)
But at the same time, I cannot deny that AI has also made me become a better person in other areas. I'd always had poor 'handiness' or 'around the house' type skills, and the number of things I've gotten comfortable doing because of AI assistance, like hanging art on the walls, taking on some basic at-home plumbing and fix-it tasks, trying new things in the kitchen with cooking and food prep and brewing coffee etc, is frankly much higher than pre-2022. And that's to say nothing of all the help it's provided getting more fit in the gym. Maybe my upbringing lacked these things that others never had an issue with, but I am grateful to this technology for helping 'level the playing field' in this regard. Just like I know how it's done the same for non-techie people who are now a little more well-versed in the matters that I'd been naturally more fluent in.
So, like most tech, we need to be mindful of our use, especially us HN-types that are drawn to information processing systems, optimization, efficiency, etc. It's really a double-edged sword, and I've never felt that more than lately. And this is doubly true for anyone with certain neurodiverse information habits (all the open tabs, 'research trails' with poor follow-up, etc). I often joke that I'll know we've reached AGI when my open browser tab count hovers below ~5!
Great work! Love the harnessing of Apple's local inference. Wondering if you'd be willing to support the same conversational export, summarization, and search/retrieval flows but for the consumer platforms (ChatGPT and Claude) that have local conversations stored? (e.g. I've seen that local Claude Cowork conversations are stored in some kind of JSON-based schema that might be similar to Claude Code, but I'm really not too sure).
Yes, I thought about that, and which is why I'm just running this as a pilot for this quarter to see how it goes and I'm not particularly attached to it if feedback/results are mixed. But that said, there are real constraints on my ability to supervise students with in-person examination, but that is more a problem with how my society funds and cares about education. We are forced to take approaches that 'work at scale', hence experiments like this. But truly, reception was positive when this was introduced, because students too want to be held to a good standard and don't want others to get credit for not doing the work toward achieving the learning outcomes like the most honest ones.