chaos_emergent

this isn't at all realistic; a tiny fraction of free users are going to be willing to pay for your product.

Ethical objections in this thread are a red herring.

Our brains are not that different than other primates' brains. We do abhorrent things to other primates that __demonstrably__ have similar experiential qualia as humans, mostly for the benefit of our species. But somehow objectioners are losing their marbles over a human cell cluster that has no more than a few thousand constituent cells.

The computational ability of human brains is an unintentional happenstance of billions of years of random-walk evolution; there's no reason to believe that we can't make systems that are even more efficient with intentional design. If these organoids get us there, I'm all for them.

Explains why everyone I’ve talked to that uses Rippling (including our company) hates it.

The size that you can grow to while maintaining singular focus is getting bigger and bigger. Parker’s situation is probably different because he was able to raise a bunch of money early on after Zenefits bequeathed him with SV stardom. He had early signs of success because he was able to acquire customers through what seems like an incomparable feature completeness, at the cost of shit UX and buggy software (IME). For example: they literally didn’t file our tax documents despite telling us they did, resulting in thousands of dollars of fines.

I might be overly cynical based on our pretty awful experiences, but feel vindicated because we’ve moved to a smaller, leaner, more focused HR startup named Warp and haven’t had half the issues we did with Rippling.

what about a mechanism that 'locks' the trackball in place at rest, and then requires a little bit of force to 'unlock' the trackball to track?

I have a moonlander that I love, and I want to add a trackball or trackpad to it.

I saw the mod that Kevin Christensen wrote on ZSA's site: https://blog.zsa.io/diy-moonlander-trackball/

The missing piece is that the trackball doesn't click, and this seems to be the case in general with trackballs on the market.

Does anyone know why trackballs don't click? I'm imagining a casing that would allow the entire ball to drop by a few millimeters to trigger a click. One potential reason is that it's hard to click without moving the mouse on screen, but I'm unsure of how much of an issue that is in reality.

We're neither disembodied brains nor bodies in isolation of basic needs like shelter or desires for energy-intensive conveniences, so I challenge that we should try to assess energy consumption in the way you're proposing. The outcome and the inputs required to get to it are the two relevant variables IMO.

Being consistent, though, I should've included the amortized cost of training and the energy cost of the rest of the data center to the 4o analysis. I don't think that those costs would be more than a factor constant greater.

When we can create disembodied brains that don't have needs for shelter and desires for energy-intensive conveniences, I buy that we can do 10% of the energy accounting.

You can try it for yourself, my company is in my profile.

You should make it something random like “pineapple”

I agree! I think it’s “worth it”. A parallel: it costs an OOM more to drive places than, say, bike places, but it allows us to do the things we did before an OOM faster and expands the types of things we can do.

You can make the argument that it’s not “worth it” in terms of cost to our environment, but you have to acknowledge the upside of it simultaneously.

And the dynamics are different in that we’ve reached an efficiency asymptote with cars and are still in the middle (or perhaps at the start) of the S curve with AI systems.

I’d have to give away some of my secret system prompt sauce to tell you, but I’m sure that the summarizations I get are solid, we’ve done extensive evaluation of truthiness. I have tens of thousands of hours of calls summarized monthly from paying customers who rely on them to be accurate. We’ve only ever gotten hallucinations when we accidentally omitted the transcript, and even that was fixed by allowing the LLM to alert us when it didn’t see a transcript.

By that argument, anything that an AI system could do today that a human does right now is an incredibly massive waste of our resources :-)

It sure does beat needing to have someone to do it for you :-)

Yeah, an incredibly dumb analysis.

I did a back of the hand estimate based off of the following assumptions:

gpt-4o: 10 H100s = ~7000W power requirement, 30 seconds to generate a summary for an hour-long call

human: ~10.7 kWh/year energy requirement (avg per American), 20 minutes to generate the same summary

The numbers come out to 60 Wh for 4o and 400 Wh for a human

I’m confused by the evidence they use:

> to justify those costs, the technology must be able to solve complex problems, which it isn't designed to do

Planning and reasoning are the two greatest areas of research in AI right now, with an OOM more researchers devoted to it than there were to the first generation of generative AI architectures

> In our experience, even basic summarization tasks often yield illegible and nonsensical results

Summarization with current generation models is excellent. I can get a summarization of a several-hour-long-call with better recall than I could have had myself, for less than $2 in inference costs.

> even if costs decline, they would have to do so dramatically to make automating tasks with AI affordable

We’ve seen a literal 10x decrease in cost from gpt-4-32k to gpt-4o in a single year of AI development (3:1 cost blend). And that ignores that sonnet-3.5 is 50x cheaper than gpt-4-32k while getting better scores on pretty much all benchmarks?

> the human brain is 10,000x more effective per unit of power in performing cognitive tasks vs. generative AI

Patently false, we’re not untethered brains floating around and require shelter, food, and a ton of other energy intensive requirements to live, and an AI system can perform a task that it is designed to do easily 10-20x faster than a human could.

If anything this makes me more bullish about AI systems having a positive ROI; the criticisms they have are based on extraordinarily (if not nefariously) dumb assumptions.

specifically, it's 20x a GHG compared to C02, and I don't really know what you mean by it being part of carbon cycle.

What do you think that is?

actually I just went into the paper and skimmed it.

Their image tokenizer creates image encodings in a discrete space of dimension 8192; each image consists of 1024 discrete tokens. The 8192 vector size matches their language tokenizer (discrete and also size 8192). Then they just mash those tokens together, which leads to downstream problems in training: because the two token spaces have different entropies, stable training in later steps is difficult, and they have to do a bunch of regularizations to make sure that gradients don't start exploding.

Interesting stuff!

can someone give an intuition for the unified tokenizer that they mentioned? I understand the idea of using a GAN or an autoencoder to make a image tokenizer, but it's unclear how you combine that with text to have a single tokenizer over both modalities (if I'm understanding the paper correctly)

They're releasing a model to the public that likely costed tens of millions, for free, and you're complaining that they don't want to have to deal with bullshit PR for giving a teenager fucking around in his backyard the recipe for a pipe bomb.

orders of magnitude

When training a large model you're doing a single forward pass + backprop over multiple Infiniband-connected nodes for a single model instance, so if one node goes down it takes a logical unit of nodes down with it. For reference, GPT-4 was rumored to be around 1.7T, and doing some back-of-the-hand math[1], that's like 500-700 H100 GPUs per model instance, which means you need a multiple of that for any training parallelism whatsoever.

[1] back-of-the-hand-math: 1.7T * 4 bytes = 6.8 TB; 3-4x that for activation + gradients = 27.2 TB; 27.2TB / (80GB / H100) = 349 H100s; 1.5-2x conservative multiplier accounting for not fully using node resources + memory overhead in the machine = ~500-700 H100s.

truly insane numbers.

If you don’t think an AI could do that then I think you need more imagination and to follow the OOMs

This is the most insightful nugget into OAI's consumer product strategy I've read, thank you.

have you tried using gpt-4o? It's pretty incredible at recognizing handwriting.

Maybe instead of overloading the shorthand with two definitions, it's best to just use the actual words.

as a follow up curiosity, has anyone tried using LoRA on the entire model for pretraining to compare regular training model performance to LoRA?

The findings are that the best fine-tune performance comes from fine-tuning all weights, followed my MLPs, followed by attention heads, using LoRA. Authors assert that the performance difference is based on the target module of the NN.

Isn’t an equally valid argument that MLPs tend to constitute a greater number of weights in transformer networks than attention heads, and the performance difference can be traced to a greater number of weights having freedom to change? I’d be curious to know if randomly choosing a subset of matrices to train, regardless of where they are in the network, would provide analogous performance to LoRA on a specific module with comparable learnable weights.

Isn’t this fairly obvious after a two second glance at the abstract

> Every time i've actually tried something like this it has not outperformed statistical process control.

There are clear counterexamples to your experience, most notably in maintaining plasma stability in tokamak reactors: https://www.nature.com/articles/s41586-021-04301-9

"Why would you even try to predict the weather if you know it's going to be wrong?"

- most OCs on this thread