watch it somehow use 20x more tokens tho
425 karma · joined November 20, 2024
The onboarding experience of answering a WhatsApp message asking simply "what have you been putting off?" was pretty slick. Now I just want to see a version I can run on my own hardware so I don't have to worry about some sketchy company hoarding my data.
I know OpenClaw and Hermes are supposedly answers to this, but I find that they've each become so bloated with features that they're far more difficult to use and bug prone than I'm happy with. I'd prefer something more opinionated that limits the surface area for bugs as well as the number of decisions I need to make as an operator.
1. AI Agents (which are at the center of Omarchy) arguably make just about anyone a "developer" in the conventional sense, and potentially more willing to use a dev-forward desktop 2. Converting the types of devs who've been stuck on MacOS for years because they're neurotic about UI/UX "polish" could be a great way to smooth out Linux's many rough edges
IMO the biggest hurdle with building "a Linux desktop for everyone else" is QA. Discovering and fixing the random little bugs that only surface from millions of slightly different hardware/software configurations is a massive undertaking that I'm not sure can be easily brute-forced.
Alternatively, I see a lot of value in building an exciting desktop that attracts a critical mass of folks who are comfortable diagnosing bugs, submitting clear diagnostic reports, or even fixing said bugs themselves, and willing to do that work (or delegate it to an AI agent) for free.
Even if Omarchy doesn't end up being the desktop "everyone else" actually uses, it could be a great first step towards building out the kind of community linux would need to ensure such a desktop meets the requisite user-friendliness standards.
Honestly I'm probably more likely to build my own OpenClaw-mini with a custom client app. Shoehorning client connectivity into an existing messaging app borderline feels more complicated than just having an LLM build a minimal client app from scratch with full awareness of the backend capabilities.
It's not trivial to do, and I never managed to get bot-to-bot communication working. Even with my janky setup, the experience is honestly pretty great. Grok Bot simplifies the setup for this about as far as I imagine is possible, and frankly it's a pretty slick experience.
I fully expect this paradigm to catch on quickly.
Recently picked up a 2013 Mac Pro (the trashcan one) on eBay for $150, and the official 6-bit quant of 2.6B runs at 25-30 tok/s under llama.cpp on one of the D500 GPUs. After installing Ubuntu LTS I have myself a lovely (if somewhat inefficient) little homelab server.
https://artificialanalysis.ai/methodology/intelligence-bench...
Edit to provide AA's article explaining it:
https://artificialanalysis.ai/articles/artificial-analysis-i...
Whether authoring such a feature is itself a criminal act is an outrageous question, to which the answer should be an obvious, and emphatic "no".
I have high hopes on this topic, given token efficiency seemed to be the primary (only?) goal of the K2.7 Code release.
Excited to see the signals that come out of the big eval/benchmark sites.
A text-first terminal thin client strikes me as a compelling use-case for a pair of smart glasses, depending on the execution. Only thing I'd really want is to be able to connect a keyboard.
Genuinely asking.
> If I am fully self-hosting the entire bluesky app, I need to spend ten thousands of dollars a month, but it'll keep working.
That strikes me as a point in favor of AtProto for decentralization (unless it's possible for others to cheaply/simply run their own backups of the entire mastodon.social instance & I'm just unaware of that fact).
> With bluesky it's different.
The key AtProto differentiator I find most compelling (which it seems you omitted from your list) is that Bluesky can disappear tomorrow and, if I'm running my own PDS, I can simply log in to another different platform using my same username & account and immediately see my entire message history w/o skipping a beat.
From the standpoint of an individual user who just wants to own their own data w/o needing to worry about hosting entire apps or manage scaling, the fact that AtProto allows me to host a PDS and do just that while still using any of big-name apps w/o needing to create new logins and fracture my online presence feels like a far more pragmatic trade-off, IMO.
I guess it comes down whether you care more about data decentralization, or platform decentralization. Personally, I care far more about the former, and to that end the AtProto model feels much easier to set up & far more seamless in practice.
Everything I'm seeing about hosting costs in the current day and age is that the full AtProto stack (PDS, Relay, AppView) is roughly in par with hosting an ActivityPub instance of equivalent size (if not a little cheaper).
And with AtProto, folks get to pick and choose what slice of the stack they wanna host, and opt-in to more of it gradually as they see fit. With ActivityPub, you are either opting in to hosting an entire instance, or fully reliant on someone else.
I'm open to the idea I'm misunderstanding some aspect of ActivityPub, given I've not really explored the hosting side of all that deeply.
With ActivityPub, because running an instance requires hosting the data, the application, and dealing with all the subsequent scaling challenges, you kinda have to choose between being taking on active ops responsibilities or tying yourself to someone else's instance (which will probably be one of the bigger, more centralized ones).
If you decide you don't like an instance you picked and decide to move (unless things have changed) you're kinda stuck needing to start fresh.
With AtProto, it's trivial to jump ship to a different application platform and continue using your same identity. Exporting your data from a platform and self-hosting is a bit of a UX challenge, but at least it's possible.
As an example, I recently started using Tangled for the first time and was able to login using my existing bsky-backed domain (h14h.com). No need to create a new account or pick a new username -- it was as if I were already there. Then getting set up w/ self-hosting my git repos on a VPS was an afternoon of work at most, and it's just some backend service chugging away that I almost never have to think about.
The worst that will ever happen is I see a banner message in tangled.org saying something like "your repo is out of date and may be compatible with the latest version of Tangled", which I can solve by simply rebuilding & redeploying a docker image w/ the latest versions.
Granted, AtProto is definitely harder to wrap your head around architecturally. But actually interfacing it with a user is much simpler, IMO.
Don't have any evals indicating how it compares on upper-bound quality, but for a well-defined task it seems like GLM 5.2 on "High" is remarkably token efficient. Looking forward to seeing where it lands on the AA index.
Grok 4.3 was made on Colossus 1? It's not frontier quality, but it's definitely not nothing. Also, Cursor brings more expertise to help them improve their utilization further.
Additionally, I struggle to see how renting out excess capacity is anything but good. It brings in a ton of cash at a price premium, and ensures they have headroom to gradually phase out rental capacity as their internal demand increases.
> You think they have more compute than Google?
Fair point. No way they have more raw compute, but I do think it'd be fair to say they have more "excess compute capacity" than Google, but even that is pretty speculative.
> You think they have the most cashflow?
Again fair point. I was overestimating how much cash comes from a $2T IPO before actually looking at the numbers. My revised take is that SpaceX/xAI are now in-line with the other labs on cash liquidity, rather than leading (where pre-IPO they were way behind)
> You think they have higher quality training data than Anthropic, Google, and OpenAI?
Yes -- I'm standing by this one. Cursor has a multi-year head start on agentic coding data collection, and a GUI UX that likely provides richer user sentiment & quality signals than something like Claude Code.
Obviously OpenAI & Anthropic have far larger proprietary datasets for chat histories, and Google is an undisputed leader in data hoarding. But when it comes to agentic coding specifically (AI's most compelling use-case, IMO), I think Cursor's data is a HUGE deal. This is backed up by how good Composer 2.5 was given it was essentially Kimi K2.5 + Cursor data.
Additionally, I also suspect its possible to mine Twitter for user engagement & sentiment analysis to create surprisingly useful datasets.
> Training on what? They rented both of their data centers away!!!
I thought they only rented Colossus 1, but you're right they're also renting out a portion of Colossus 2. That said, they still have 2/3 of their Colossus 2 facility available for internal, which is likely enough to build something competitive. They also have 90-day termination agreements in place once they forecast a need for more internal capacity.
All that said, while I appreciate your pushback & I was overly hyperbolic on a few key points in my last comment, I stand by my core theses: The Cursor acquisition (assuming it goes through) combined with the SpaceX IPO puts xAI/Grok right in-line with the other big labs, at least in terms of positioning.
Whether they're able to execute remains to be seen, but I would not at all surprised to see a frontier-quality Grok 5 or Composer 3 release before the end of the year.
Per AA, while K2.7 Code is roughly on par w/ K2.6 in terms of intelligence, it uses half the output tokens to get there.
It is simply not feasible train an LLM to be as good as these frontier models are without a TON of high-quality examples of what "good" looks like. Every time a Cursor user (who didn't opt out of analytics) does/doesn't hit a "retry" button, or rejects/accepts an LLMs output, it allows Cursor to log record of a specific LLMs output and a binary signal of that output's quality.
Given they've been at this since 2022, and for most of that time sat comfortably at #1 in market share among comparable AI coding tools (only recently getting topped by Claude code), Cursor likely has the largest, highest-quality, SWE-specific dataset in the industry by a sizable margin.
Grok being so late to the party could only train on twitter data in combination with whatever they could source publicly or purchase privately, and likely hasn't had anywhere near the usage they'd need to build up their own competitive dataset from scratch anytime soon.
If you believe (as SpaceX seems to) that the AI's total addressable market is over $26T, and acquiring proprietary, high-quality training data is the difference between capturing ~1-2% of that market and ~10-20%, then $60B starts to look like a bargain.
Given xAI's Grok is way behind ChatGPT & Claude on coding capabilities, whereas Cursor was able to get in spitting distance of them w/ Composer 2.5 by simply running post-training on Kimi K2.5, I'm not sure Elon could dream up a more perfect strategic fit.
Cursor likely has the largest, highest quality dataset of any private firm for training new coding models, which would compete SpaceX's trifecta of becoming a viable competitor in the AI race:
1. Access to compute (they have so much that they're renting capacity to Anthropic & Google)
2. Liquidity for R&D+M&A (largest IPO in history)
3. High quality training data (this Cursor acquisition)
> Isn't it kinda bizarre that Elon is pivoting SpaceX to something else?
In a vacuum, absolutely yes. But in the bizarre context of the AI economics, chaotically scrambling to bring everything you need to compete in-house makes perfect sense.
Arguably, when compared with either OpenAI, Anthropic, or Google, SpaceX/xAI now own the most compute, are the most financially liquid, and (assuming the Cursor acquisition goes through) have the largest corpus of high quality training data.
We may very well be a couple of months away from a Grok release that goes toe-to-toe w/ other American frontier models, IMHO.
So when you look at this as a $60B play to capture an additional 10-20% of an estimated $26T total addressable market, it makes a lot more sense. Now, whether that projected TAM is even remotely close to reality (or even just enough to make Cursor worth $60B) is another question entirely.
[1]: https://www.satellitetoday.com/finance/2026/05/20/spacexs-ip...
* edited to add source for IPO numbers & tweak grammar/formatting
I have shortcuts set up to count the hours I log in my work Google calendar and copy them to my clipboard to help me prepare invoices.
So while I've already been sold on what Shortcuts can do, getting the general public to see the possibilities is probably gonna be a challenge.
The interface for creating them manually has been so bad for so long, it feels clear to me that LLM-driven shortcut orchestration was always the endgame. Apple built up their ecosystem of composable "tools", and then trained an LLM on how to call them.
The result, IMO, is the first OpenClaw/Hermes competitor that's feasible for use by the general public.
Everyone with a paid Claude or ChatGPT that they're struggling to use to the fullest is going to have very little reason not to swap over to an upgraded iCloud+ plan (if they don't already have one). I suspect we're going to see mass cancellation of $20/mo plans very soon.
OpenAI's timing for removing their temporary increased usage limits is looking pretty unfortunate...
Getting ~1000 TPS on near-frontier intelligence is a step change, and enables whole new use-cases for applications. Seeing limited compute resources beget selective access makes me worry for the future of competition.
The thing you DO hear a lot, though, is that you don't need to worry about bugs nearly as much as you do in other languages. But that's not because Elixir is "magic", rather, it comes from Elixir's runtime (Erlang/BEAM) providing best-in-class fault tolerance primitives like lightweight process isolation and supervision trees.
In practice that means the blast radius of bugs is generally tiny and any resulting crashed processes are often recoverable. The phrase you often hear is "let it crash", since the effort that goes into exhaustive defensive programming is usually more costly than the bugs you'd be trying to prevent.
If the first thing (for example) my mom sees upon installing the app is a dropdown model picker that contains things like "Qwen3.6-35b-a3b-mlx" she will 100% be bouncing off of it.
IMO the best version of this is a custom app/harness with a couple of pre-selected (and ideally fine-tuned) open models that immediately start downloading after checking the system's hardware specs. This would likely be a turn-off to most devs, but is absolutely essential if building an app for general consumers.
But honestly, obsoleting a huge number of otherwise great Apple Silicon machines is something Apple would moment consider a major "pro" of building a compelling local AI stack.
With how much speculation around the difficult time Apple has had getting people to upgrade from M1, I'm sure they'd jump at such an opportunity.