HNHacker News
TopNewBestAskShowJobs

ajtulloch

1,055 karma · joined April 1, 2012

Andrew Tulloch. Member of Technical Staff at OpenAI. Formerly distinguished engineer at Facebook/Meta.

Find me at http://tullo.ch and github.com/ajtulloch.

Email is andrew@tullo.ch

submissionscomments
ajtulloch··on Rideshare operations are being suspended in California
https://en.wikipedia.org/wiki/Capital_strike
ajtulloch··on 'Coup 53' tells the story of 1953 campaign by MI6 and CIA to oust Iran's leader
It's not explicitly mentioned in the article, but the documentary has been released for streaming today at https://coup53.com/.
ajtulloch··on Modeling a Wealth Tax
This is exactly the same level of mathematical sophistication as in Paul's original post.
ajtulloch··on Modeling a Wealth Tax
This is an incredibly stupid and mendacious line of reasoning. This is just the action of an incredibly rich guy afraid to give up a small sliver of his enormous wealth, and grasping at straws to do so.

A simple way to see how silly this argument is - replace "0.5% wealth tax" with "2% VC fund management fee", and see how that changes the conclusion!

ajtulloch··on How the Black Sea shaped the ancient Mediterranean world
For folks interested in this, Fernand Braudel's magnum opus "The Mediterranean and the Mediterranean World in the Age of Philip II" is an incredible work of history, studying the Mediterranean and the societies around it.
ajtulloch··on Named Tensors
crudely,

    linear transformations : matrices :: multilinear transformations : tensors.
ajtulloch··on CUDA 11.0
You might find https://github.com/TheSnakePit/mamba useful, especially if you are slowed down by package resolution.
ajtulloch··on A highly efficient, real-time text-to-speech system deployed on CPUs
Absolutely, it's super impressive work (as is your later work with Marat :) ).
ajtulloch··on A visual explanation for regularization of linear models
See e.g. https://en.m.wikipedia.org/wiki/Lagrange_multiplier and https://en.m.wikipedia.org/wiki/Karush%E2%80%93Kuhn%E2%80%93... for the justification.

It’s essentially an example of duality in numerical optimization (https://en.m.wikipedia.org/wiki/Duality_(optimization)#Dual_...).

ajtulloch··on The new Clang _ExtInt feature provides exact bitwidth integer types
Modern AMD GPUs still support 24 bit integers, see https://rocm.github.io/GCN_Float16.html.
ajtulloch··on Jax – Composable transformations of Python and NumPy programs
See https://en.m.wikipedia.org/wiki/Subderivative
ajtulloch··on How fast can you allocate a large block of memory in C++?
It's not `operator()`, it's value initialization. If you're particularly masochistic you can read more about it at https://en.cppreference.com/w/cpp/language/value_initializat....
ajtulloch··on Thousands of Google’s cafeteria workers have unionized
The framing of this is so incredibly (almost humorously) ideological. Look at what you've implicitly set up:

* the State, contracts, corporations, right-to-work law, at-will employment law, etc - all good, natural parts of the market.

* employee unions - bad, unnatural, rent-seeking, deviations from the market.

It's untenable.

ajtulloch··on Shutdown of US coal power facilities saved over 26k lives, study finds
FWIW I think the counterfactual is not asking what the impact is if Australia stopped emitting greenhouse gases directly, but what the impact would be if it stopped producing coal (for both domestic use and export).

Given Australia is a substantial producer and exporter of coal (responsible for around 7% of global production [1]), and that coal (+ natgas + oil) power generation are responsible for around 30% of global greenhouse gas production [2], and assuming that coal is responsible for between half and all of GHG emissions from power plants [3], you could ballpark that Australian-produced coal is responsible for an incremental 1-2% or so of global greehouse gas emissions (0.3 * 0.07 * [0.5~1]).

[1]: https://en.wikipedia.org/wiki/Coal_in_Australia [2]: https://www.c2es.org/content/international-emissions/ [3]: https://www.worldcoal.org/coal/uses-coal/coal-electricity

ajtulloch··on Cerebras’s giant chip will smash deep learning’s speed barrier
For the folks who are downvoting this comment, the author is absolutely a subject matter expert (and completely correct).
ajtulloch··on Information Theory with TensorFlow
This is very close to the thesis of Mackay’s Information Theory, Learning, and Inference textbook: https://www.inference.org.uk/itprnn/book.pdf
ajtulloch··on One-Percenters Close to Surpassing Wealth of U.S. Middle Class
Robert Paxton’s Anatomy of Facism (https://www.penguinrandomhouse.com/books/128540/the-anatomy-..., https://en.wikipedia.org/wiki/Robert_Paxton#Fascism for a 5 min overview) is a fairly compact historical overview of the rise of fascism in post-WWI Europe, and analyses the historical/cultural/material factors for why facists came to power in Italy and Germany but not in eg England. It are mostly focuses on the dynamic cited in the grandparent comment re:

> At that point the ultrawealthy will seek to form an alliance with the growing popularity of the far right and seek to annihilate the left.

This is the third stage of Paxton’s five stages, the first three being:

> 1. Intellectual exploration, where disillusionment with popular democracy manifests itself in discussions of lost national vigor

> 2. Rooting, where a fascist movement, aided by political deadlock and polarization, becomes a player on the national stage

> 3. Arrival to power, where conservatives seeking to control rising leftist opposition invite fascists to share power

ajtulloch··on How to Read “Gilgamesh”
For folks interested in a modern reinterpretation of the Iliad, the "War Nerd Iliad" by John Dolan (aka the War Nerd from the Exile) takes an interesting approach - see review in https://blog.lareviewofbooks.org/essays/war-nerd-iliad/.

That passage rendered there looks like (roughly):

> Kalkys, reassured, takes a nervous gulp and blurts: “The real reason that Apollo’s shooting us down is that Agamemnon insulted his priest! The old man came to ask for his daughter Chryseis back. He asked nice and politely, in the name of the god, which he’s entitled to do as a priest! Brought the proper ransom, and a wreath!”

> All the men grunt and nod. They knew it!

> Kalkys feels their approval and goes on, more loudly and accusingly: “But Agamemnon purposely insulted him! Threatened him! Laughed at the poor old man! Wouldn’t give his daughter back even though he asked politely!”

> Everyone nods and tells each other, “I told you that was it! That damn Agamemnon! Gonna get us all killed!”

> Kalkys concludes, “So we have to give her back! To her father, the priest! Or Apollo will kill all of us! And not ask any ransom for her or anything!” More nodding and grunting from the crowd. They knew it’d come to this.

> Kalkys pushes his luck now, brave like nerds are when the crowd’s egging them on: “And we have to send a sacrifice with her! Treasure, and gold, and calves and sheep, ones with no spots or scars! Perfect specimens, the kind Apollo likes!”

> More grunts and nods. Kalkys is drunk on public approval now, delighted with his own courage—and then he turns and sees Agamemnon and sits down very suddenly.

> Agamemnon stands up, with the hate pouring from him like heat from a rock on a hearth. The crowd goes quiet. It’s odd, how they fear Agamemnon. He isn’t all that tough in battle. And he’s nowhere near as big as Akilles, or as strong as Ajax, or clever like Odysseus. Doesn’t even fight in the front rank most of the time. But he is hands down the meanest man in the army, maybe the world. He never forgives, never forgets the tiniest slight to himself or his precious relatives.

> Him and his relatives! That’s what this whole army is doing here, avenging Agamemnon’s useless brother Menelaos, who married a woman way too beautiful for him—half goddess, in fact—and got dumped for a younger, hotter man. Who happened to be a Trojan prince.

> That’s why they’ve all been camping out here for nine slow, deadly years: Defending the nonexistent honor of Agamemnon’s blank of a brother, the fool Menelaos, Menelaos the cuckold.

> And now Kalkys has blurted out the one thing they’re all thinking: It’s Agamemnon’s fault we’re here! It’s his fault we’re dying under the magic virus arrows! And we didn’t even get any booty out of it!

> Agamemnon stands, sneering, letting the crowd vent a while, then drags out the silence so they can feel his hate. That’s where Agamemnon really shines—the best hater in a world where hate is much respected.

> When he’s made them all flinch away from his stare, he turns to poor old Kalkys: “You, Mister Science! You just love giving bad news, don’t you? You little coward, egghead. Did those entrails you claim to interpret ever once, even once, tell you anything good about me? Did those dead goat guts of yours ever tell, even one time, that I, your king, had made a good decision? No! No, because you only want to tell me what I’ve done wrong! Because you’re a cowardly little whiner!!”

ajtulloch··on AI competitions don’t produce useful models
https://arxiv.org/abs/1902.10811 is an useful counterpoint to this article's comments on ImageNet overfitting (esp e.g. §3.3, "Few Changes in the Relative Order").
ajtulloch··on The Secret History of Dune
For folks interested in the analysis of Dune (and Frank Herbert’s subsequent works), the Podside Picnic podcast (https://soundcloud.com/user-733327042) ran a “Dune History Month” in August which is worth checking out. The recent one on the Dune Encyclopedia (https://en.wikipedia.org/wiki/The_Dune_Encyclopedia) was fascinating.
ajtulloch··on Thoughts on Rust bloat
Systems that JIT large amounts of code (HHVM, etc) deal with this trade-off all the time. See e.g. https://qconsf.com/sf2012/dl/qcon-sanfran-2012/slides/KeithA... for an old discussion of some of the issues (e.g. inlined/specialized versions of memcpy were slower overall than a standalone-slower outlined version).
ajtulloch··on Jack Dorsey Is Gwyneth Paltrow for Silicon Valley
This is very close to the argument Alain de Botton makes in https://en.wikipedia.org/wiki/Religion_for_Atheists.
ajtulloch··on RaptorQ and performance optimization in Rust
Unaligned and aligned loads of AVX vectors have basically the same performance since Ivy Bridge IIRC.
ajtulloch··on We improved Tensorflow Serving performance by over 70%
FWIW, I believe that the current state of the art for batch-size 1, fp32 inference for ResNet-50 on Intel CPUs is AWS's work in https://arxiv.org/abs/1809.02697. After the low-hanging fruit outside of model execution are picked, this kind of work is probably quite relevant.
ajtulloch··on Growing a Compiler: Getting to Machine Learning from a General Purpose Compiler
This is an incredibly impressive line of work. Huge props to the Julia team.
ajtulloch··on The Swiss Army Knife of Hashmaps
One thing worth trying out as well are the folly::F14 hashmaps (https://github.com/facebook/folly/blob/master/folly/containe...), which has a roughly similar design. You can find some discussions on the performance tradeoffs by the SwissTable and F14 authors at https://groups.google.com/forum/#!msg/hashtable-benchmarks/-....
ajtulloch··on Ask HN: Ex-FAANG developers, where are you now and why?
I think the population you're observing here are people who first joined a FAANG before 2014 (since I imagine most people stayed for their initial four-year grant given FANNG stock performance). Facebook (not sure about the others) in 2014 had ~2k engineers IIRC, but Google IIRC had like 10-15k engineers around that time. The gap has closed in recent years, but looking at current engineering numbers isn't the right approach IMO.
ajtulloch··on Why Is Behavioral Economics So Popular?
e.g. https://en.wikipedia.org/wiki/Legalized_abortion_and_crime_e... or https://www.economist.com/finance-and-economics/2005/12/01/o...

> It was a good test to attempt. But Messrs Foote and Goetz have inspected the authors' computer code and found the controls missing. In other words, Messrs Donohue and Levitt did not run the test they thought they had—an “inadvertent but serious computer programming error”, according to Messrs Foote and Goetz.

> Fixing that error reduces the effect of abortion on arrests by about half, using the original data, and two-thirds using updated numbers. But there is more. In their flawed test, Messrs Donohue and Levitt seek to explain arrest totals (eg, the 465 Alabamans of 18 years of age arrested for violent crime in 1989), not arrest rates per head (ie, 6.6 arrests per 100,000). This is unsatisfactory, because a smaller cohort will obviously commit fewer crimes in total. Messrs Foote and Goetz, by contrast, look at arrest rates, using passable population estimates based on data from the Census Bureau, and discover that the impact of abortion on arrest rates disappears entirely. “I am simply not convinced that there is a link between abortion and crime,” Mr Foote says.

ajtulloch··on The Widely Divergent Life Expectancy in Every County in the US
Note that’s it’s also incredibly localized, even in stereotypically “wealthy” areas. For example, life expectancy varies by >10 years across neighborhoods in San Francisco County (https://www.cityofberkeley.info/uploadedFiles/Health_Human_S..., slide 32).
ajtulloch··on The Return of the Super-Elite
Are you unironically claiming that having an inequality of economic outcomes is the only way to motivate/incentivise human behaviour?
← PreviousPage 2 of 7Next →