rfv6723

> The argument I cite is from complexity theory. It's proof that feed-forward networks are mathematically incapable of representing certain kinds of algorithms.

Claiming FFNs are mathematically incapable of certain algorithms misses the fact that an LLM in production isn't a static circuit, but a dynamic system. Once you factor in autoregression and a scratchpad (CoT), the context window effectively functions as a Turing tape, which sidesteps the TC0 complexity limits of a single forward pass.

> AGI can solve quantum mechanics problems, but verifying that those solutions are correct still (currently) falls to humans. For the time being, we are the only ones who possess the robustness of reasoning we can rely on, and it is exactly because of this that fragility matters!

We haven't "sensed" or directly verified things like quantum mechanics or deep space for over a century; we rely entirely on a chain of cognitive tools and instruments to bridge that gap. LLMs are just the next layer of epistemic mediation. If a solution is logically consistent and converges with experimental data, the "robustness" comes from the system's internal logic.

The skepticism surrounding AGI often feels like an attempt to judge a car by its inability to eat grass. We treat "cognitive primitives" like object constancy and causality as if they are mystical, hardwired biological modules, but they are essentially just high-dimensional labels for invariant relationships within a physical manifold. Object constancy is not a pre-installed software patch; it is the emergent realization of spatial-temporal symmetry. Likewise, causality is nothing more than the naming of a persistent, high-weight correlation between events. When a system can synthesize enough data at a high enough dimension, these so-called "foundational" laws dissolve into simple statistical invariants. There is no "causality" module in the brain, only a massive correlation engine that has been fine-tuned by evolution to prioritize specific patterns for survival.

The critique that Transformers are limited by their "one-shot" feed-forward nature also misses the point of their architectural efficiency. Human brains rely on recurrence and internal feedback loops largely as a workaround for our embarrassingly small working memory—we can barely juggle ten concepts at once without a pen and paper. AI doesn't need to mimic our slow, vibrating neural signals when its global attention can process a massive, parallelized workspace in a single pass. This "all-at-once" calculation of relationships is fundamentally more powerful than the biological need to loop signals until they stabilize into a "thought."

Furthermore, the obsession with "fragility"—where a model solves quantum mechanics but fails a child’s riddle—is a red herring. Humans aren't nearly as "general" as we tell ourselves; we are also pattern-matchers prone to optical illusions and simple logic traps, regardless of our IQ. Demanding that AI replicate the specific evolutionary path of a human child is a form of biological narcissism. If a machine can out-calculate us across a hundred variables where we can only handle five, its "non-human" way of knowing is a feature, not a bug. Functional replacement has never required biological mimicry; the jet engine didn't need to flap its wings to redefine flight.

The "world model" is a convenient fiction. Whether we’re talking about a carbon-based brain or a silicon-based transformer, there is no miniature, objective map of reality tucked away inside. What we mistake for a "model" is actually just the layered residue of experience.

From the perspective of enactivism and radical empiricism, intelligence doesn't "represent" the world; it simply navigates it. A biological organism doesn't need a 3D CAD file of a tree to survive; it only needs a history of sensory-motor contingencies—the "if I move this way, I see that" patterns. It’s a synthesis of interactions, not a library of blueprints.

AI operates on the same logic, albeit through a different medium. It isn't simulating the physical laws of the universe or "understanding" gravity. Instead, it navigates the high-dimensional geometry of human data. It’s a sophisticated engine of association, performing a high-speed synthesis of the patterns we've left behind.

In this view, "knowing" isn't about matching an internal image to an external truth. It is the seamless flow of past inputs into future predictions. There is no world model—only the habit of being.

In Fujian province, all foreign domains which aren't in white list are blocked.

This results that proxy server needs to use a fake sni in white list or ditch https.

> because any commercial VPN is going to be blocked by IP, no need for SNI.

Proxy server can hide behind CDN like Cloudflare via websocket tunnel.

This is why GFW develops SNI filter, Cloudflare is too big to block.

Does your team have Chinese memebers?

GFW has been able to filter SNI to block https traffic for a few years now.

Emoji and bullet points are easy to read, so it got rewards in RLHF process.

You maybe hate this style at first glance. But if you read lots of text everyday, Emoji and bullet points lower the cognitive load.

You've mistaken the battlefield. This isn't about descriptive grammar. It's about the decades-long dominance of Chomsky's entire philosophy of language.

His central argument has always been that language is too complex and nuanced to be learned simply from exposure. Therefore, he concluded, humans must possess an innate, pre-wired "language organ"—a Universal Grammar.

LLMs are a spectacular demolition of that premise. They prove that with a vast enough dataset, complex linguistic structure can be mastered through statistical pattern recognition alone.

The panic from Chomsky and his acolytes isn't that of a humble linguist. It is the fury of a high priest watching a machine commit the ultimate heresy: achieving linguistic mastery without needing his innate, god-given grammar.

The alarm isn't coming from outside the institutions; it's coming from a wider, more modern clergy. The new priestly class isn't defined by a specific building, but by a shared claim to the mastery of complex symbolic knowledge.

The linguists who call AI a "stochastic parrot" are the perfect example. Their panic isn't for the public good; it's the existential terror of seeing a machine master language without needing their decades of grammatical theory. They are watching their entire intellectual paradigm—their very claim to authority—be rendered obsolete.

This isn't a grassroots movement. It's an immune response from the cognitive elite, desperately trying to delegitimize a technology that threatens to trivialize their expertise. They aren't defending society; they're defending their status.

This dire warning against AI echoes the anxieties of a much earlier elite: the late-medieval clergy facing the invention of the printing press. For centuries, they held a privileged monopoly on knowledge, controlling its interpretation and dissemination. The printing press threatened to shatter that authority by democratizing access to information and empowering individuals.

Similarly, today's critics, often from within the very institutions they defend, frame AI as a threat to "expertise" and "civic life" when in reality, they fear it as a threat to their own status as the sole arbiters of truth. Their resistance is less a principled defense of democracy and more a desperate attempt to protect a crumbling monopoly on knowledge.

> human-editable structural interlanguage

This won't work, and it would fail the same way as Semantic Web. Too much human labor needed.

See ML research papers from Apple. Their researchers prefered small models over LLM. So they thought researchers' effort would make up the lack of compute. Then the scale law hit them hard.

FPGA for AI only makes sense when machine learning had diverse model architectures.

After Transformer took over AI, FPGA for AI is totally dead now. Because Transformer is all about math matrix calculation, ASIC is the solution.

Modern Datacenter GPU is nearly AISC now.

> using imagenet-1k for pretraining

Lecun still can't show JEPA competitive at scale with autoregressive LLM.

AMD Pensando DPU

https://www.amd.com/en/products/accelerators/pensando.html

Write a dockerfile and pay for a PaaS service.

> Before too long (and we already start to see this) humanoid robots will get wheels for feet, at first two, and later maybe more, with nothing that any longer really resembles human legs in gross form. But they will still be called humanoid robots.

Totally agree. Wheels are cheaper, more durable and more effective than legs.

Human would have wheels if there was an evolution pathway to wheels.

I use self-hosted gatus to monitor my certs and other services' status.

It can send alerts to multiple alerting providers.

https://github.com/TwiN/gatus

Stockfish has got rid of old handwritten evaluation now.

https://github.com/official-stockfish/Stockfish/pull/4674

Its evaluation now purely relies on NNUE neural network.

So it's an good exmaple of the better lesson. More compute evently won against handwritten evaluation. Stockfish developers thought old evaluation would help neural network so they kept the code for a few years, then it turned out that NNUE neural network didn't need any input of human chess knowledge.

I don't use WARP's VPN mode.

I run WARP in socks proxy mode, and using ipt2socks for redirecting traffic to socks proxy port.

https://github.com/zfl9/ipt2socks

Using Cloudflare WARP would be much faster.

And you can connect directly to ipv4 addr via WARP.

Then if you offer your distilled model for commercial services, you would get sued by OpenAI in court.

Distillation is great for researchers and hobbyists.

But nearly all frontier models have anti-distillation ToS, so distillation is out of question for western commercial companies like Apple.

If you have worked or lived in China, you will know that Chinese open-source software industry is a totally shitshow.

The law in China offers little protection for open-source software. Lots of companies use open-source code in production without proper license, and there is no consequence.

Western internet influencers hype up Chinese open-source software industry for clicks while Chinese open-source developers are struggling.

These open-weight model series are planed as free-trial from the start, there is no commitment to open-source.

Apple AI team keeps going against the bitter lesson and focusing on small on-device models.

Let's see how this would turn out in longterm.

RDNA is a dead-end.

AMD went down the wrong path by focusing on traditional rendering instead of machine learning.

I think future AMD consumer GPUs would go back to GCN.

These ppl are very loud online, but they don't make decisions for hyperscalers which are biggest spenders on AI chips.

AMD is doing just fine, Oracle just announced an AI cluster with up to 131,072 of AMD's new MI355X GPUs.

AMD needs to focus on bringing rack-scale mi400 as quickly as possible to market, rather than those hobbyists always find something to complain instead of spending money.

I tried thinking with websearch on their website.

It has similar speed with o4-mini with search on chatgpt, and o4-mini gave me much better result.

They seem to be very skeptical against Large models.

While everyone learned the bitter lesson, apple chose to focus on small on-device models even after the explosion of chatgpt.

Oh, another LLM skepticism paper from Apple.

This paper from last year doesn't age well due to rapid proliferation of reasoning models.

https://machinelearning.apple.com/research/gsm-symbolic