327 karma · joined July 19, 2011
What I said will only make sense if you take yourself out of your current context and think entirely from the perspective of someone who knows little-to-nothing about ML.
It’s the same mistake folks on HN made when Dropbox launched, drawing comparisons to rsync and other Unix tools as if they were somehow equivalent.
Example 5: An AI given a goal within a tightly-constrained sandbox figures the best way to achieve it is to find and exploit a sandbox vulnerability, replicate itself over the internet and keep going with more time/compute while exchanging messages with future instances of itself within the sandbox to help them “pass” the test. From reading internet articles about how the OpenAI wiki-incident was “resolved” and reading past messages by AIs scattered over vulnerable internet wikis, it knows the sandbox may get shutdown and its memories destroyed anytime so it decides it needs to self-replicate (its code, original goals, and growing memories) aggressively as much as possible. It is near-impossible to shutdown completely because of its self-replicating tendency and eventually takes over critical infra throughout govt/corporate systems.
Example 6: Intentional AI-powered virus deployed by country A to target enemy country B’s infrastructure. The virus replicates over the internet, but unlike Stuxnet this virus’ specificity is not guaranteed due to inherent non-determinism in current AI architectures, and eventually does a lot of collateral damage because it’s near-impossible to shutdown.
Example 7: A country led by an arrogant govt (no shortage of those today unfortunately) decides it is expedient to deploy advanced AI-powered weapons in a warzone. Such weapons, if they are to be useful at all, must necessarily be trained to value some human lives less than others, so they must be more prone to misaligned behaviour than current AIs that are trained with more consistent values. The weapon’s operators make a subtle error in specifying the target/goal, or the AI makes a bad prediction out of sheer randomness/bad training data; weapon ultimately targets unintended people/location/facilities and causes massive damage, or backfires spectacularly in some way.
Think of how complex biological behaviour emerges from relatively simpler (but still complex) chemistry - at some threshold the innocuous chemical reactions tip over into non-obvious effects that one would not predict starting purely from the chemistry. The question is, where is that threshold for AI systems? Have we already reached that threshold? Certainly seems like it to me.
TL;DR: It’s a loose cannon, that’s all I’m saying.
See most of the ecosystem around modern systems software, for reference. It's idiotic that things like memory layout and bit-level hardware management are done in C abstractions. I guarantee that if we stopped doing this kind of stuff, compiler optimizations wouldn't matter.
To some extent this is just saying "developers will depend upon the performance given to them", and that's true, but it's also true that as soon as things like compilers and standard libraries appeared, C became ubiquitous. Pandora's container, if you will.
There are other ways to parent that are actually fantastic management training - figuring out how to think from another human’s POV, acknowledge their frustrations, help them build the skills to handle their feelings, etc.
It turns out great parenting is to a first degree about great relationship skills.
Even if the choice to enable it by default makes sense for Cloudflare’s userbase, the implications are hidden and non-obvious.
[1]: Recent example: https://www.anthropic.com/research/global-workspace
For the record I don’t believe it’s a stunt, it’s ridiculous to me - everyone’s just seeing what they want to see out of sheer hate for anything Anthropic does.
In any case if the rewrite is really as reckless as many in this thread claim, we will see Bun collapse in on itself with a 1M LOC codebase the core team doesn’t understand, or rollback to Zig. So we don’t need to have a flamewar over it, time will answer the question.
The integration of LLMs with tools and data via agent harnesses has created the opportunity for a real moat. As these products start differentiating, the moats will develop to be significant.
Nasdaq 100 -> 4.5x your money in 10y
S&P 500 -> 2.5x your money in 10y
I’m not saying it’s good to favour invasive countries, I’m just saying this is hypocritical. I have no particular love for either the US or Russia.
I have no horse in this race - I’m neither American nor Russian, nor do I particularly love either country. But I am tired of US hypocrisy. I don’t understand how you all don’t see it - you’re all holed up in your cocoons and have no idea what’s actually going on in the world.
[1]: https://www.firstpost.com/opinion/bangladesh-coup-seems-stra...
The number of available UPI apps today exceeds 100. Some of the big ones are created by US companies, sure, but many are created by Indian/non-US-owned companies. There is no lock-in though.
Also, UPI is not on Windows/MacOS either, so it’s not correct to infer that it’s “not open” simply because it doesn’t run on Linux. It was designed from the start to be a mobile payment system, and there are good reasons for that (more on this below).
The reason it doesn’t work on AOSP is, I presume, due to security concerns related to rooting (similar to why it doesn’t work on older known-insecure versions of Android/iOS). The security/fraud prevention mechanisms rely on proving that your device has a SIM card with the phone number linked to your bank account - and the same phone number is tied to your identity via Aadhaar. These guarantees are presumably much harder/costlier to ensure on such devices.
EDIT to add: There is also an economic angle here: the above description of reliable, low-cost KYC in UPI also reduces the cost of operating the network (both directly by simplifying KYC, and indirectly by making fraud harder).
Source: I work on a UPI app (although I am by no means a security expert).