1,717 karma · joined March 29, 2021
If someone wants to be heard, they have like 5 hours after work, an infinite amount of internet forums to spend political battles on, and they can't put their head down at work to just... work?
They can't defend their own backyard, refuses to pay for their 2%, and free-rides off of US protection.
They tried to court China as their partner, only to now face a flood of Chinese car imports that outcompete their once prided industries.
They seem to only care about national security when it comes to fleecing US companies, rather than say defending against Russia, or strategizing against China.
Mistral isn't exactly some decade-long state funded national champion. Its a 1 year old startup whose core staff all come from American tech companies. With 0 revenue and minimal market share, there's like no conceivable metric in traditional antitrust theory to block this. Yet the EU prevents it from taking foreign investment all the same.
Look at TSMC in comparison. It IS the national champion of Taiwan, yet its majority foreign owned, doesn't stop it from underpinning the Taiwan economy and spawning massive ecosystems in its local areas.. Taiwan's government understands its place in the international economy, so prefers to cooperate and trust its international partners.
US has new upstarts like Tesla and Nvidia, which at their prime, add like 1 IKEA per fortnight to their market cap.
You understand Apple spent like billions for this electric car industry that never produced anything? Why would they want to starve the AI project now?
You understand how many people in Microsoft and Google are working on AI adjacent projects? Even if they aren't AI researchers, there's tons of work just to put the models in useful applications.
You underestimate the overlap between GenAI and EVs. Apple is not merely a software company, their AI strategy will not be merely send data to datacenter to run LLMs.
Apple has to run local LLMs heavily amongst its devices, for responsiveness, cost, and privacy reasons. This will entail a holistic effort from batteries to the neural inference chips to the AI models to the user design. Much more complicated than the datacenter only approach.
The car division would have also had people working on the battery, neural chips, UI design, etc. I would estimate 30% at least get moved to AI.
Now AI progress may outpace their ability to understand it, but they'll still love it and be dependent on it.
Unlike adults, they don't face the threat of job displacement from AI.
Nvidia could always just half their prices one day, and wipe out every non-state-funded competitor. But Nvidia prefers to collect their extreme margins and funnel it into even more R&D in AI.
You attribute this to a 'bias of underestimating other people's jobs' I attribute this to a 'bias of overestimate our own jobs'.
Most programmer jobs do not involve facebook clones that support 500 million users. Most programmer jobs are boring crud apps for random companies in traditional industries. Just think of what the outsourcing mills in India produce, and how many people they employ, then think again about how many jobs generative AI replace.
Also, Hollywood rapidly organised to try to slow down AI. They could see, very evidently, how extremely rapidly AI was advancing, and how lethal the threat is to them.
Its a lie people want to believe.
Try telling people about the pace of AI advancement, scaling laws, how AI art managed to surpass 90% of human artists in 1 year. And it'll either result in visceral disgust, or complete indifference.
People do not care. Even if the flood has consumed the street, and is rising 1 meter per minute, they do not care. Unless the flood hits their front door, and wets their feet, they prefer to live their normal lives assuming everything tomorrow will be the same as today.
Hence all the AI messaging has become all about 'copilots', because that's the only message people want to listen to. There's plenty of extremely frank discussions by AI company CEOs, AI company staff, or even AI enthusiast forums, but people are utterly uninterested, and prefer to scroll tiktok and watch their TV.
Only when they actually encounter a bot a work that does 95% of their work, will they suddenly wake up, and organise, and demand a response. Hollywood managed to organize at an incredible speed to demand a slowdown to AI, but the water was at their necks already.
Now, its understandable why most people don't want to read about AI advancements. They have their lives, chores, children, weekend trips. They feel entitled to a life in statis, where revolutionary change doesn't happen unless they are the ones demanding it. Too bad reality doesn't work that way.
In the end, its individual choice, to go all in, to decisively steer your career path for a post-AI future, or pretend it doesn't exist, and enjoy their remaining years in their jobs in bliss. I will watch with fascination and horror what happens to the latter group.
Detailed aspects versus GPT-4 turbo 1. World knowledge, slightly inferior. GPT-4 turbo was able to detail a protagonist's childhood year by year for a Japanese novel with near 100% accuracy (That a human reader would get the chronology wrong). Gemini ultra much more easily confused.
2. Creativity, Gemini ultra wins. Its writing style has far more flair than GPT-4 turbo, it also occasionally made some stunning analysis that I never thought of and made perfect sense. GPT-4 turbo is more like a textbook repeater, it doesn't make many mistakes, but also rarely surprises you with anything original.
3. Accuracy, GPT-4 turbo still makes fewer mistakes. Including in subtle logic (Like having a hypothetical battle between two characters in the same universe, considering the strengths and weaknesses of their powers, etc).
So this is definitely Google's first real-deal LLM. Its not better than current GPT-4 turbo, but its getting there. OpenAI must be feeling the fire to release GPT-5 before the end of the year.
Telsa had to deploy their capital very efficiently in a capital intensive industry. Say their bet on the Shanghai factory, despite obvious IP leak risks and spawning deadly competitors like BYD, was their only choice to reach scale at a low cost.
It took like 2 extra months for Azure OpenAI to get GPT-4 turbo. There's a noticeable time delay between OpenAI deploying their latest model and when Microsoft manages to shove it in Azure.
These small SQL LLMs are indeed worthless, since LLM performance on SQL queries is quite limited right now, every % of accuracy matters.
1 man 1 vote does not lead to the wisest or most long term of decision making, however, without it, the natural tendency is for the politician class and the wealthy to coalesce into one, with money reinforcing political power, and vice versa, sucking away any vitality in society.
In the US, as well as in many countries, its been shown again and again that money cannot simply buy the vote of every person. And there mass voting already demonstrates its power.
This principle extends to every facet of society. Company executives are checked by the board, the board checked by the shareholders, the shareholders checked by each other (the US allows shorting stocks for a reason).
I therefore find it questionable that a bunch of unchecked software engineers will lead to very good outcomes. No oversight, no pressure = freedom to pursue creativity and outcomes? Or merely idle hands? When people were proposing working multiple remote jobs during COVID, was that people being smart, or unethical behaviour being tolerated and justified?
Now the pendulum swinging too far back into the other direction would also be bad. See what happens to Boeing when engineers get completely subjugated by MBA management. But software people should not treat the exhortation to work a full 40 hour week as being whipped by slaves.
1. Consumes a minor amount of electricity (Data centers is only 2% of US electricity use, and currently AI is maybe only 5-10% of that). Its trivial compared to say metal smelting.
2. Consume water for cooling.
That's it, there is 0 direct pollution generated from AI, and even the water use is very minor compared to say farming, and can be improved via more water efficient cooling techs.
The main concern is the scaling speed. As LLMs scale up 10x, 100x, 1000x, those previously very minor electricity costs can quickly become grid impacting in a decade.
You are not afraid of corrupt bureaucrats, but of idealistic innovators trying to change the world?
By the way, there is no way to tame safety via checklists or 'policies'. Boeing sure had checklists and layers of policies trying to control risk, can't save them when they outsource everything to workers who don't give a crap about their job.
But people in the AI world care, they absolutely do care about what they build. Pride and passion in their work beats any checklists, and I'm confident self driving cars will be a massive safety boon for all.
However, the advanced, LLM based systems with tons of world knowledge (and therefore can easily tell if a pedestrian is abnormal) take tons of processing power and latency, so cannot be deployed in the current AI driving systems. So it'll take 5-10 years for this to be solved.
Also hope you know that Germany isn't the center of the world. Its not even the center in terms of cars anymore. German regulators don't get to dictate the progress of self driving.
1. Nvidia GPUs are dominant at training
2. Inference is easier than training, so other cards will become competitive in inference performance.
3. As AI applications start to proliferate, inference costs will start to dominate training costs.
4. Hence Nvidia's dominance will not last.
I think the most problematic assumption is 3. Every AI company we see thus far is locked in an arms race to improve model performance. Getting overtaken by another company's model is very harmful for business performance (See Midjourney's reddit activity after DALLE-3), while a SOTA release instantly results in large revenue leaps.
We also haven't reached the stage where most large companies can fine-tune their own models, given the sheer complexity of engineering involved. But this will be solved with a better ecosystem, and will then trigger a boom in training demand that does scale with the number of users.
Will this hold in the future? Not indefinitely, but I don't see this ending in say 5 years. We are far from AGI, so scaling laws + market competition mean training runs will grow just as fast as inference costs.
Also, 4 is very questionable. Nvidia's cards are not inherently disadvantaged in inference, they may not be specialized ASICs, but are good enough for the job with an extremely mature ecosystem. The only reason why other cards can be competitive against Nvidia's cards in inference, is because of Nvidia's 70% margins.
Therefore, all Nvidia needs, to defend against attackers, is to lower their margins. They'll still be extremely profitable, their competitors not so much. This is already showing in the A100 H100 bifurcation. H100s are used for training, while the now old A100s used for inference. Inference card providers will need to compete against a permanently large stock of retired Nvidia training cards.
Apple is still utterly dominant in the phone business after nearly 2 decades. They capture the majority of the profits despite
1. Not manufacturing their own hardware
2. The majority of the market share by units sold is say Chinese/Korean
If inference is easy, while training is hard. It could just lead to Nvidia capturing all the prestigious and easy profits from training, while the inference market is a brutal low margin business with 10 competitors. This will lead to the Apple situation.
Like, you had to have actively developed mods for them, for free, for years, and be famous in the community, then they'll hire you (If you want).
This works because the working conditions there are far far better than your average game company. And probably much more fun than say a bank.
AI inference is so costly at scale, one can easily see data centers start using 4% of total electricity, and in the next decade 8%. That will start to have severe effects on the power grid, basically require planning many years in advance to setup new power plants and such.
"Moore's law and incremental improvements' are irrelevant in the face of scaling laws. Since we aren't at AGI yet, every extra bit of compute will be dumped back into scaling the models and improving performance.
There's no need to use such angry and charged words when a single word suffices: "Hypergamy".
Female education is the absolute decisive and consistent factor across all societies in suppressing marriage and fertility rates, for good and bad. It both takes away prime fertility years, as well as raises female expectations (Because females always prefer a higher status/earning man than themselves).
So the more females earn, the fewer males they can accept emotionally. Hence the general collapse of marriage/fertility rates when this happens.
This phenomenon is very good for developing countries, preventing catostrophic malthusian booms. Even Muslim countries like Bangladesh and Saudi Arabia now have normal and sustainable fertility rates. For highly developed industrial economies, like South Korea, China, or most of the west, its terrible in the other direction.
The more low status males withdraw from the market, the harder woman have to compete over the high-status males with no fallback, and the more anguished they'll be at the end (Because in the end the male can only pick one to marry).
AI will absolutely first cause male withdrawals from the sexual market. As men are more visually stimulated (Male porn only requires a minimum of story), AI generated art is advancing far ahead of human artist capabilities, so will soon be able to generate an infinite variety of 10/10 girlfriends in any pose or expression or environment you want.
But woman will start to withdraw too. Female 'porn' is erotica, which have to portray the man's status, and the general story, for females to gain any enjoyment, which is why they are generally always in some sort of novel format. AI is quite far from being able to write engaging fiction, but only say 5 years away. In East Asia, there is already "Otome games" where the female roleplays in a human written visual novel (And pays money each month to buy their virtual boyfriend stuff). So the progression to AI, just makes the stories far better written and genuinely reactive to the female consumer. This is also incredibly attractive to many females, compared to chasing a real life high status man who can be incredibly callous and brutal given the choices they have.
For-hire CEOs often are also paid in equity, not just stock options. But even with just call options (Which have no downside risk), there is the risk of the professional reputation of the CEO, if you just do middling for a few years and leave, still plenty of job opportunitiess. If you crash and burn the company down... Very tough to find another good exec position again.
Now normal software is priced to squeeze as much money as you can, enterprises can afford more, so are charged more. Individuals are highly price sensitive, so has to be very cheap.
GenAI is quite different in that its not 0 marginal cost, the marginal costs are probably at least 50% of the price. So the price difference between enterprise and individual plans will be far smaller than usual, due to the common cost base.