I met Paul Atkins, around this time last year. One of his biggest problems (like anyone running a federal department) was that he's not allowed to hire or increase his budget, yet the demand for work that needs SEC attention is not going down. And that's after his dept, and all others, were required to perform layoffs under DOGE.
The other way to do this, which I literally did n that era, was to just go buy an extra stick of ram out of pocket (Ram used to not be expensive!) and just guerrilla upgrade my work pc myself, IT none the wiser, and just a lot easier and faster than all the BS time and bureaucracy it would have taken otherwise.
I remember even trying this with a cheap HDD->SSD upgrade as well, but by that point they had implemented drive encryption so that foiled that effort.
Literally the best possible kind of psy-op. Same with translating Sesame Street and distributing it to 100s of countries. Same with countries like Thailand exporting Thai food. It's a psy op based on what if we took nice things from our country and subsidized distributing them to the rest of the world, our country would grow in prestige and soft power. And these strategies are usually pretty effective, not to mention usually net-positive for all sides ... compared with so many other horibble harms US and other countries foreign offices have done around the world out of self interest.
This thing has 50% more memory than GTX PRO 6000, which fetch 18k-25k+ each right now on newegg, and generally sold out. If nvdia can still sell out 96GB PCI cards (on a 2 year old architecture) at those prices, how is an AMD equivalent with better specs dead on arrival?
Okay this tidbit is interesting to me "5–6 tok/s M2 -> 31–35 tok/s on an M5 Pro". So where will be in just another gen or two?
my impression right now is that M5 gen is on the cusp of practicality for local inference.
If techniques like OPs here, start to make the RAM situation more amenable, by the time we get to M6 or M7 (or AMD's equiv next gen APUs on TSMC N2 nodes), local AI could be ready to go much more mainstream.
Well that's a relief, glad I am a turtle then and so are most of my friends, and that our whole world economy and food system runs on turtles. So we should be fine.
"Latent space representation" I have been waiting for this moment in the evolution of AI. Well, waiting with some trepidation. It seems inevitable that frontier AI's will, at some point, leave behind human-comprehensible representations of language. Purely for functional reasons, it's going to start making sense for AI agents to communicate amongst themselves in much more efficient ways than borrowing the languages of flesh-bag humans as an interface medium.
I Imagine next that programming languages, interfaces and API design starts going this direction next. Being written, expressed and optimized as blobs of high dimensional vector space. As humans we might still be able to understand some abstractions of what our AI's are talking about to each other, but maybe not more so then we understand how different regions of our own brain communicate with each other.
this strategy worked to keep Ukraine alive, by enabling them to throw literally anything and everything they could obtain into the fight. And the system enabled rapid experimentation and evolution of what works. Also they didn't have enough of anything to equip all units equally or fully, so a market-like system of was also a way to triage short supply.
However the logistics costs of fragmentation are very real (relevant to the supply chain theme of this story). And now that Ukraine is producing the better part of 10 millions(!) of drones per year, they are shifting towards more standardized drone models to simplify logistics, achieve more economies of scale and also now to have the capacity to keep units equipped more evenly and reliably.
An unfortunate series of events that this thing ended up with these specs at 1,049.00. It was supposed to be cheap and cheerful. At first Steam took an opportunistic deal to buy up a bunch of near-obsolete-already chips from AMD to build a low-cost box around. Then years of delays and an explosion in DRAM and SSD prices and here we are.
4 year old chip design on an equivalently old process node, not that unlike nvidia selling 2-3 year old chips as the spark. Thanks to AI boom, consumer market really just getting the warmed-over leftovers here from AMD and NVDIA.
I do think local models are huge pending market opportunity for Apple. An M5 Ultra Mac Studio (if that exists) could be decent local AI machine, though so expensive as to stay niche. But by the M6/M7 generations and a recovery in DRAM affordability, the future could be interesting moment for them to deliver a compelling local AI platform that 'just works'. But I do think that a mini-pc that is easy to configure, can be always plugged-in, always on, higher power envelope than a laptop, but not obnoxiously loud and hot, is the right form-factor
Not to mention quitting in droves because very many don't want to take these cases or otherwise to stand in court and explain why current admin is not bound by existing laws, court orders, the US constitution in general, or internationally recognized human rights etc.
This has already happened. Home PC market is practically dead already due to memory, ssd and graphics card price inflation. Makers of components like PC cases and power supplies etc. are seeing demand down 30-40% year over year and this is going to put many suppliers out of business. NVDIA has stopped even listing gaming revenue on their earnings reports. Both NVDIA and AMD are not seriously interested in supplying the consumer GPU market anymore either.
The only hope left is really Apple, but even apple has conspicuously delayed the launch of M5-gen mac minis and mac studio. Mostly because even Apple can't source enough DRAM to fully supply all their product lines.
>> find a reliable place to sell it without getting ripped off by scammers.
This is a real problem and why I've just about given up on ebay or fb marketplace, esp for computers. If you are in Canada though sellit9.com is a great solution to having to deal with sketchy buyers.
@Ademeure Where do you think the market will be by the time, say year from now, when Apple has rolled out it's M6 generation? Do you think one more process node and architecture revision will be enough yet to tip the balance that local LLM starts to go mainstream?
What this article fails to mention is that there are also a record number of empty tankers routed to the US refineries right now, with the intention of shifting still-relatively cheap US oil products to overseas markets where the prices are already much higher and shortages have already hit. The effects of the Iran war on the US economy will really start to kick in over the next several months.
Nobody is using LLMs to make lending decisions. They are using LLMs to read, extract and audit the supporting documents that go into normal well-tested, compliant and rules-based underwriting systems. And firms A/B test against humans doing the same work. The outcomes your are looking for are metrics like delivering faster results back to customers, with fewer mistakes and less fraud, more compliant, than a comparable human-only process.
Thanks for flagging this, I was literally seeing the same thing with protechts.net in my activity tab this morning as I was trying to understand why firefox was aggressively draining my battery.
The conceptual problem is that we keep wanting to compare AI behavior to that of traditional computers. The proper comparison is comparing AI, and how we trust or delegate to it, to the concept of delegating to other humans or even to domestic animal. Employees can be trained and given very specific skills and guidelines but still have agency and non-deterministic behavior. A seeing eye dog, a pack mule or chariot horse will often, but not necessarily always do what you ask of them. We've only been delegating to deterministic programmable machines for very short part of human history. But ad human societies, we've been collectively delegating a lot of useful activities to non-perfectly-dependable agents (ie each other) for a very long time. As as humans we've gotten done more that a few notable things in the last several millennia with this method. However, humans as delegates or as delegators have also done a lot of horrific things at scale to, both by accident or by design. And meanwhile (gestures broadly around everywhere) maybe humans actually aren't doing such an optimal job of running and governing everything important in the world?
When compared to how human make a mess of things like in the real world, how high does the bar really need to be for trusting AI agents. Even far shy from perfect, AI could still be a step function improvement over trusting ourselves.
The real 'hype' was that the oh-snap realization that Open AI would absolutely release a competitive model to Mythos within weeks of Anthropic announcing there's, and that Sam would not gate access to it. So the panic was that the cyber world had only a projected 2 weeks to harden all these new zero days before Sam would inevitably create open season for blackhats to discover and exploit a deluge of zero-days.
Yep this was the moment to finally remember to cancel my cursor subscription. I find it unconscionable to do business with someone who would do business with Elon.
Like with all new tech trends, it takes them a hot minute to catch up, but it's highly likely they will (eventually) release some killer platforms for local AI. The shared memory, high bandwidth and power-efficiency of their M chips is a near-ideal architecture. If/when they finally push out the M5-ultra, that could be round one (albeit still not at the best price/performace vs comparable cloud api tokens). A real mass-market killer device for local LLMs is still going to require some remediation of the global DRAM shortages, and maybe the M6/M7 generation.
The insight here is, that in current warfare, quantity is the quality that matters. And with quantity, cost of replacement needs to be low, platforms expendable, cheap to maintain and resupply. It, and it's support infrastructure, need to not easily be detected and targeted by drones while on the ground. F35 is not these things. It's powerful but brittle, and like many US platforms, too much value packed into too few platforms. Not enough sustain in prolonged modern conflict. A one-punch military.