Hacker News Reader

Comment by WasimBhai | Hacker News Reader

I do not intend to take anything away from the technical achievement of the team. However, as Satya opined some weeks back, these benchmarks do not mean a lot if we do not see a comparable increase in productivity.

But then there are two questions. First, are the white collar workers specifically consultants, engineers responsible for increase in productivity? Or is the white collar workers at the very right tail e.g., scientists?

I think consultants and engineers are using these technologies a lot. I think biologists at least are using these models a lot.

But then where is the productivity increases?

View on HN

It's a complex proposition. I think Satya was talking about actual gdp growth right ? In theory lets say all knowledge work is now 50% faster due to A.I. Well then I would assume this should affect civil society as well - planning a bridge, a railway etc should happen faster and more efficiently (the actual building of thins won't, but a lot of time is spent on planning a red tape). Healthcare in general should become way more efficient with people getting better treatment; this should have a positive economic effect. It does seem to me like it should be able to speed things up in the real world but of course a lot will have to do with how well the models can reason / how often they make catastrophic mistakes + the will of the governments and people to start using them seriously.

But its more complex than that - if many people start losing their jobs we all take a hit on gdp because they can't consume as much anymore, so it could take perhaps a long time until gdp actually sees meaningful gains.

And one last thought - Satya likely hasn't spent much time thinking about gdp, it's just not his field. He's a smart guy for sure but this isn't what he does.

The problem is slightly different.

Unemployment hasn't really picked up, and is unlikely to do so, unless the central bank is incompetent. (They have been from time to time.)

However, some advances don't show up in GDP. Eg Wikipedia is a tremendous achievement. But nobody pays for it, so it doesn't show up in GDP statistics.

> Unemployment hasn't really picked up, and is unlikely to do so

That's an important assessment. I don't know if you're right. If the models are going to continue to get more capable I'm expecting unemployment to rise , I don't see how it won't (sure we are promised A.I to create tons of new jobs no one has imagined yet, I haven't seen a reliable clue for such jobs yet).

No, it won't (necessarily) be AI that's creating the new jobs. In general, when a new technology comes along and automates away some jobs, you can't expect the same technology to provide the new jobs.

To give an example from the recent past: 'hipster' baristas that make you a five dollar coffee are a fairly new job. At least at scale.

But I doubt you'll be able to find any technology that automated some other job but created barista jobs.

It's just that the market will find stuff for people to do for money, unless prevented to do so by incompetent central bank policy or (too) onerous labour market regulation.

(The labour market can take quite a lot of regulation, and still be able to get people jobs. Have a look at Germany today for an example.)

> It's just that the market will find stuff for people to do for money

Will it ? Let's take my example, I'm a 41 year old male with around 15 years experience in software development. Lets say 4 years from now myself and million others are losing our development jobs to A.I. What does the market have for my skills? I can try going into healthcare or teaching (though that's quite an extensive retraining + salary reduction), I can go into the trades (same) or get some other work that's hard to automate like caring for old people (very low salary). All of these options involve massive salary reduction, and that's in the positive scenario that I actually am able to retrain and survive such a shift mentally. It's quite likely many software devs won't be able to become plumbers and nurses and will become chronically unemployed.

Well, we have many examples where in the past technology (and to a lesser extent trade) have let to some sectors of the economy using fewer people than before.

The situation you describe isn't all that special.

Yes, losing your job (or your career) is not fun, and can be painful. Massive salary reduction can happen.

No, that hasn't lead to widespread unemployment in the past. At least not widespread enough to be visible in aggregate statistics, especially over the noise of the 'normal' business cycle. However, individuals can obviously have pretty long spells in unemployment, but that can also happen without a shift in technology.

> Yes, losing your job (or your career) is not fun, and can be painful. Massive salary reduction can happen.

I'm just trying to get the point across that unemployment might rise so gdp may fall, in fact I think it should be the baseline scenario and not thinking some new jobs we can't imagine yet will be created. It's so hard to imagine these new jobs because if the machines will out perform us cognitively it follows we will be able to get intelligent robots into the real world quite soon after. Then seriously what the heck is left? Fewer jobs, not more.

There is one "cure" I can think of for this and that's something closer to socialism, the market will have to step aside and the government will create massive amounts of new jobs. For example classes can be 5 pupils per teacher instead of 30 pupils per teacher. Nurses can attend to 3 patient beds instead of 8. But letting the market sort this out ? I don't think so.

> It's so hard to imagine these new jobs because if the machines will out perform us cognitively it follows we will be able to get intelligent robots into the real world quite soon after. Then seriously what the heck is left? Fewer jobs, not more.

So I admit that this is a serious possibility that we need to consider.

But for the argument to make sense, we can't just talk about the general 'Oh, new technology will make a bunch of jobs obsolete.' We have to specifically talk about what (might) make AI special in that it might be even more general than electricity.

You didn't mention these special factors in your original comments.

I am not sure whether AI will be different or not, or rather I don't know how different it will be.

So far I see it as a good sign that we have many relatively equally competitive models from different providers, and some of them have open weights and some of them even have completely open sources (including training algorithms). So at least it's unlikely for the technology to be monopolised by any one entity.

> There is one "cure" I can think of for this and that's something closer to socialism, the market will have to step aside and the government will create massive amounts of new jobs. For example classes can be 5 pupils per teacher instead of 30 pupils per teacher. Nurses can attend to 3 patient beds instead of 8. But letting the market sort this out ? I don't think so.

If you want to involve the government, I'd rather give everyone a basic income, than to give our pupils inferior teachers and our sick people inferior nurses. (After all, we are assuming that humans will be worse at these jobs than the AI.) Also, I'd rather have people enjoy whatever it is they want to do, instead of being forced into some government provided make-work programme.

I can feel this already with my own use of language models.

All the questions I had before language models, I have answered with language models.

That doesn't mean I have no more questions though. Answering those questions opened up 10X more questions I have now.

In general, everyone knows that answering scientific questions leads to new and more questions. It is the exact same process in the economy. There is a collectivist sentiment though in society and the economy that wants to pretend this isn't true. That the economic questions can be "solved", the spoils divided up and we live happily ever after in some kind of equilibrium.

As far as new jobs, they are here now but they surely sound as ridiculous to think about as being a professional youtuber in 2005. Or I think of the person making a geocities website in 1997 vs a front end developer. There is no date that a front end developer emerges from the html code monkey. It is a slow and organic process that is hard to game.

> As far as new jobs, they are here now but they surely sound as ridiculous to think about as being a professional youtuber in 2005

How many people can make an actual living out of Youtube? Surely they exist but to reliably live off it for decades (not just 1-2 years of temporary fame - which is also very hard to come by) I'd say fewer than one in ten thousand people will make it. I can't call "Youtuber" a career path with that kind of success rates anymore than I can call being an actor in Hollywood a career path.

As it stands currently I'd say this is difficult to measure.

They're not baked into workflows where the measurable output is attributed easily to the model use. Productivity in its current form is transformative in the sense that the use case and gain differs for the individual (who even provide different prompts). So some are keeping the gains for themselves, others are using it to improve quality rather than quantity.

It'll come in time, it's important to remember gpt 4 was released 2 years ago this month. The newer models are more reliable and could probably be introduced into workflows more frequently. Today I spoke to a company who are looking to use it to reduce cost in the next year.

That’s true, but productivity has many factors and takes a long time to get confidence on. Any productivity value that could be stated clearly would have similar downsides to a benchmark, and take far longer.

Benchmarks are useful as leading indicators. Early warning signs. If there’s no relation to the eventual productivity then hopefully that benchmark will disappear as it’s not useful.

In a fast moving space like this it’s reasonable to make use of leading indicators.

Also, why not compare to GPT-o3 in the benchmarks?

The models not really available.

they have access to o3, I do. Thousands of people do(tens of thousands at this point?). Come on. Compare to SOTA, when you're saying it's the best AI you have.

I honestly wasn't aware it's available to the few outside of pro.

Reply on news.ycombinator.com