Hacker News Reader

Hacker News

Comment by tdullien | Hacker News Reader

Full thread

tdullien

When channelling Oppenheimer, it is worth remembering that von Neumann quipped:

"Some people profess guilt to claim credit for sin."

View on HN

The version of the quote I've heard (and which sounds better to me) is this:

"Sometimes someone confesses a sin in order to take credit for it." -John von Neumann

Love the parallels people these days draw between OpenAI and Oppenheimer (ok, the Manhattan Project, but maybe thats part why OpenAI call themselves that, to alliterate)

Especially the part where Sama is trying to gather in one place the most talented, uh, anti-fas?

Maybe we will look back and see it as quite timely that Nolan's biopic on Oppenheimer arrived when it did.

Manhattan Project, that can be copied and duplicated for practically free. Interesting times.

That’s not actually true at all. The basic hardware required to run even a copy of GPT-3 is outside of the budgets of all but the wealthiest companies and individuals. The engineering talent required to wire it all up is also extremely scarce.

Training an original GPT-4 sized model would also cost on the order of hundreds of millions of dollars.

GPT3 is 800GB, which is about as large as the largest torrent files out there. GPT4 size is unknown, so I can't comment. While I haven't run the actual numbers (life is short), assuming a GPT3 torrent, my sense is that the cost of running it would be under $1M, possibly under $100k. Compared to (hundreds of?) $B for a state-level nuclear weapons program.

But yes, technically I was wrong. It is not 'practically free', it is 'within the budget of e.g. a race boat enthusiast'.

That would be the cost of running GPT3 as-is, not to train a new model or hook it up to any live information.

GPT4 has a much larger context window (16x larger), which suggests its file size would be at least 16x as large.

I reached for von Braun, channeled by Tom Lehrer: "Once the rockets are up, who cares where they come down? That's not my department!"

The same von Neumann that famously argued for (nuclear, apocalyptic) first strike at USSR.

Reply on news.ycombinator.com