cavisne

One additional factor is you need to beg for GPU capacity from cloud providers, so they are investing in them, and also giving them preferential capacity over companies they don’t invest in.

My impression is they still have a fairly efficient culture, the main problem with joining a bubble company would be if they are overhiring/losing focus.

Just sell shares when they vest to reduce your risk.

Using the underlying cost, it is not feasible to create a general purpose parallel processor for cheaper than Nvidia. Especially considering the economies of scale. So you basically need to make a bet on a less general purpose accelerator, as Nvidia loses its economies of scale when you do that.

When you include Nvidia's huge margins, this bet is less risky, but it's still only one that the largest companies can make.

Software alone cannot do it, you need to make a bet on hardware.

I wouldn’t think too deeply about it. It’s almost certainly just a prompt “if humans are in the picture make them from diverse backgrounds”.

Out of curiosity what’s stopping them cloning existing processes and technology like they do for everything else?

You don’t have to go to that level of detail but it is still relevant. For example FlashAttention is built on optimizing for this level of detail.

Right, and they don't even need to lower their training margins, since training needs a fancy interconnect they can just ship the same chip at different prices based on interconnect (and are already doing so with the 40xx vs H100).

It's a good question. I think fundamentally its because no wants to/can compete with Nvidia when making a general purpose parallel processor. They all want to make something a bit more specialized, so they need to guess what functionality is needed and not needed.

This is a really tricky guess, case in point that AMD's latest chip cant compete on training because they could not get Flash Attention 2 working on the backward pass because of their hardware architecture. [1]

Attempts to abstract at a higher layer have failed so far because that lower layer is really valuable, again Flash Attention is a good example.

[1] https://www.semianalysis.com/p/amd-mi300-performance-faster-...

Google do have an inference advantage with TPUs.

Everyone else needs to pay nvidia margins.

Training is murkier as it’s more about the total performance and scalability of the system.

I think this fits the intent of EU regulators. The DMA is essentially a tariff on American tech companies. A European company could now fork android and have a locked down App Store with large fees, without complying with DMA (as it only applies to certain companies the EU does not like).

Starbucks coffee has a Prop 65 label. It's likely that any domestic toaster would also put one on. The reason for this is the onus is on the company to prove it doesn't have any risky chemicals. The USA is the land of the frivolous lawsuit so its just not worth it.

I remember (maybe its still like this!) there used to be a page you could go to and see all your alexa voice clips since forever. It's amazing amazon has gone so long not caring about privacy without backlash.

EDIT: Yep its still there! https://www.amazon.com/gp/help/customer/display.html?nodeId=...

Need to watch the video still but my assumption was since it’s illegal/frowned upon to advertise gambling, especially to minors. This is just an earth way to put ad… I mean streams of gambling in front of a large audience.

No they run on online casino money, the founders own Stake.com

This paper from Microsoft [1] is the coolest thing I've seen in this space. Basically workload (deep learning in this case) level optimization to allow jobs to be resized and preempted.

[1] https://arxiv.org/pdf/2202.07848.pdf

Steve Coast (OSM founder) on Minds behind Maps [1].

Erik Prince on Shawn Ryan show - the host is a bit cringe but Erik has very unique perspective.

My dream podcast for 2024 would be Erik Prince w/ Dwarkesh (someone a bit smarter to push back and get more insight).

[1] https://youtu.be/uB9NxbnE1i0?si=C8gU9tkmh37kStBd

If the size of the internet is really a bottleneck it seems Google is in quite a strong position.

Assuming they have effectively a log of the internet, rather than counting the current state of the internet as usable data we should be thinking about the list of diffs that make up the internet.

Maybe this ends up like Millenium Management where a key differentiator is having access to deleted datasets.

This idea was actually in the launch keynote! Along with a bunch of other cool ideas that never shipped.

Maybe someday someone will explain what went wrong.

My guess is that competitors were quicker to market than they expected, and had unique advantages (Microsoft owns windows and Xbox, nvidia owns … nvidia).

I think this would be easily detectable anyway as whatever IP’s they are using would be viewing videos that basically no one ever watches.

Nvidia has pretty epic margins in the datacenter space, so some of it will be that. Epic because their software advantage in AI lets them charge a premium, so why bother competing for the absolute cheapest $/FLOPS contract.

More generally LLM training is weird because its a supercomputing workload being executed by Silicon Valley devs. So they want to use open source frameworks, find answers on stack overflow, use cloud providers etc. And it's new enough that random PHD's can invent stuff like Flash Attention without being employed by Nvidia.

Normal supercomputer users dont care about any of that.

Even with AMD working on hard on specific benchmarks for marketing purposes they could not get close to peak hardware performance on their brand new chips.

https://www.semianalysis.com/p/amd-mi300-performance-faster-...

Wow that twitter thread was great, actually diving into the output of these scam organizations.

PPP loan data is another great source for finding these scam charities, although most of them dont even have any output to critique.

"BI also reported in October 2022 that software engineer salaries at the company (Meta) can reach up to $308,148, not including stock options"

Excluding stock options is underselling the compensation at FANG by a lot, as they vest monthly and are immediately liquid.

My hunch is that there is some fundamental misunderstanding of what CUDA is and why it works. Its been like 6 years now of well funded attempts to displace CUDA, and there has been very little progress. In fact I'd argue its gone backwards as things like FlashAttention have further locked in the benefits of CUDA & Nvidia chips.

I looked a bit more into this program and its all a lot more complicated

1) The money was granted in 2020 based on an inaccurate map (leading starlink & others to get funding for covering places like Target's and major airports)

2) This Starlink-FCC debate has been a protracted process, since then the program has essentially failed (a third of the money has gone to companies that didnt deliver)

3) Since then significantly more funding has been given to states for broadband, making this FCC program relatively small

It seems like the FCC is clawing back as much money as possible to try and recover from the initial auction design.

https://communitynets.org/content/worries-mount-rural-digita...

Oversubscription where?

ISPs are not buying anywhere near that much transit bandwidth.

Starlink is basically a WISP with an actually scalable business model, just the towers (and soon a lot of the backhaul) are up in space.

WISPs rely on a local enthusiastic person to make it work.

The part of Meta research that worked on LLaMa happened to be based in the Paris office. Then some of the leads left and started Mistral.

Complex/simple is not really the right way to think about training these models, I'd say its more arcane. Every mistake is expensive because it takes a ton of GPU time and/or human fine tuning time. Take a look at the logbooks of some of the open source/research training runs.

So these engineers have some value as they've seen these mistakes (paid for by Meta's budget).

The timing is potentially clever. Apple has committed to supporting RCS next year, and will face regulatory pressure in places like Europe.

Even if short lived they could onboard a lot of Android users and then use RCS once it’s supported.

It would require a hardware update for older devices I believe, ie any that don’t have TPMs