why_only_15

If you believe this article, I would be happy to bet up to $10,000 that OpenAI will not collapse in the next 24 months. We could operationalize this by saying that OpenAI will employ at least as many people (or pay at least as much in salaries) in 24 months as it does now. The author of the article refused to take this bet when it was offered to him several times https://x.com/JeffLadish/status/1817999232105627722.

It's not two years after release, but it is two years after it was trained

how often is significant numbers of bit errors a problem, or when does that come up?

they also address something similar in the body: "Here are two audio samples at 14 kbps with heavy 30 percent receiver-side packet loss."

Apple has designed their own GPUs since they stopped using PowerVR with A11

what is "mostly irrelevant in 2024" quoting from?

The law has strict standards for fairness, for good reason. It excludes things like rumor, for good reason. But when you as a private person are deciding how to act, you make the best decision available to you with the information you have. The information we have is fairly compelling: multiple women have deeply regretted their interactions with him, enough to risk lawsuits to publish their regret publicly. Why interact with him unless absolutely necessary?

Where are you referring to? In the US railroads were never nationalized (long-term) and private roads were never much of a thing, and the US has a pretty good (freight) railroad system [1].

[1]: https://en.wikipedia.org/wiki/Rail_freight_transport#Statist...

HBM is often, as I recall, $30/GB for comparison

Why did Texas not start building out batteries for ancillary services until recently? Did/do they not need as much ancillary services as PJM? Also -- do you know what happened to ancillary service pricing as batteries became cheaper? Did grids become much more stable in practice?

This isn't really how batteries make money in the US right now, as I understand it. ~80% of battery revenue in ERCOT comes from what are called "ancillary services" [0] which is payment from the grid for being able to very quickly increase or decrease the amount of generation to ensure stability.

[0]: https://cimview.com/monthly-battery-revenue-in-ercot/

In the US, PV solar capacity factor is about 0.24 (partly because lots of people live near very sunny regions like California and Texas). The National Renewable Energy Laboratory claims that in some parts of the US utility-scale solar capacity factors are 0.33 [0]. Wind is about 0.36, Hydro also 0.36, Nuclear about 0.93 [1]. If you apply these capacity factors to the nameplate capacities (ignoring batteries), you get 58.5% solar, 19.7% wind, 15.1% natural gas, and 6.6% nuclear.

[0]: https://atb.nrel.gov/electricity/2021/utility-scale_pv

[1]: https://www.eia.gov/electricity/monthly/epm_table_grapher.ph...

I appreciate that his acknowledgements here were to his daughter ("for detailed feedback on the generated fractals") and wife ("for providing feedback on a draft of this post")

I assume a government program for batteries, but not sure.

> As of 2021, about 2 percent of houses of worship in the United States have solar systems, according to Lawrence Berkeley National Laboratory, which the University of California manages for the U.S. Department of Energy. That’s disproportionately high; houses of worship make up only 0.6 percent of all non-residential buildings.

What they meant here was that 2% of solar systems in the United States are on HoW and 1.9% of HoW have solar systems (page 5 of https://live-etabiblio.pantheonsite.io/sites/default/files/h... )

no, there are thousands of Blaze build files that each have their own binaries, tests, libraries, etc.

Do you have any thoughts on adding tensor core support? Tensor cores are most of what modern GPUs are used for but are devilishly hard to program well (they totally break most of CUDA's abstractions).

It's interesting that they say skill is about finding good trades -- my perception from playing a few times is that there's a lot of subtlety in how many of what you offer, and how the information that you have those cards changes others views about which suit is the goal suit. A successful strategy I've pursued playing with other friends who have never played before is just selling all my cards for >=$5 and making a small, consistent profit.

It is incorrect to say that the moat is the software. The moat is primarily the compute hardware, which is still incredibly good for the price, plus really good networking equipment. CUDA is not a significant moat for massive LLM training runs, as you can see from Anthropic moving from CUDA to Trainium (and so presumably rewriting all of their kernels to Trainium).

Compute shaders are not capable of using modern GPU features like tensor cores or many of the other features needed to feed tensor cores data fast enough (e.g. TMA/cp.async.shared)

I'd be curious to learn more about how it's compute bound and what specifically is compute bound. On modern H100s you need ~600 fp8 operations per byte loaded from memory in order to be compute bound, and that's with full 128-byte loads each time. Even integer/fp32 vector operations need quite a few operations to be compute bound (~20 for vector fp32).

Human explainability/interpretability is terrible -- it takes years to understand a person well enough to model their actions well in novel situations, and there's tons of literature that backs up that people consistently invent bullshit explanations when asked why they did things. Humans can only be held to account by telling them to change, which we can do with neural networks too.

You can keep around the KV cache from previous generations which lowers the cost of prompts significantly.

gpt3.5 turbo is a new model, not Curie. As others have stated, it probably uses Mixture of Experts which lowers inference cost.

Oh huh I didn't know about that!

NVLink is the most notable (>600GB/s GPU-to-GPU communication) whereas the best case scenario on 30x0/40x0 GPUs is peer-to-peer PCIe (~30GB/s) and average case is routing PCIe through the host (~15GB/s)

The hardware in your flat definitely cannot do 1 billion queries per second -- that requires a massive, global system, which can probably also support a shared system with ~100,000x less queries.

I don't think that's true?

Bigtable: https://cloud.google.com/bigtable/docs/replication-overview

> When using replication, reads and writes to the same cluster are consistent, and between different clusters, reads and writes are eventually consistent. If an instance does not use replication, Bigtable provides strong consistency, because all reads and writes are sent to the same cluster.

DynamoDB: https://docs.aws.amazon.com/amazondynamodb/latest/developerg...

> Both tables and LSIs provide two read consistency options: eventually consistent (default) and strongly consistent reads > Eventually consistent reads are half the cost of strongly consistent reads

Google internally announced a while back that Bigtable (which powers Spanner etc.) hit 1B queries/second -- there definitely exist systems with far larger scale (though admittedly this is with lower atomicity requirements and probably includes reads etc.).

Google Cloud Bigtable and DynamoDB both appear to have ACID -- I don't see why mainframes would be better for this than cloud.

Bitcoin is slow because of the many servers not in spite of it. Because of the design of the network, all servers need to receive every transaction and servers need to be able to be pretty small, which limits the transaction rate.

There are (or at least used to be) a number of datacenters filled with crypto mining equipment.