tmnvdb

I think the way he sketches the the AI labs as "marketing geniuses" for not just releasing their models as auto-correct is a bit cynical, as well as implying in general that these labs are muddying the waters on purpose by not agreeing with <authors position> and by engaging in "hype" (believing in the technology).

Hmm, it seems that the author takes very clear (and sometimes cynical) positions on some controversial questions. For example, "They don't have the capacity to think through problems logically." is an hotly debated claim, and I think with the advent of reasoning models this has at least become something one should not state in entry level material, which would hopefully reflect common understanding rather than the authors personal opinion in an ongoing discussion.

There are more claims like this about what language models can't do "because they just predict the next token". This line of reasoning, while superficially plausible, holds a lot of assumptions that have been questioned. The heavy lifting here is done by the word "just" - if you can correctly predict the next token in every situation (including novel challenges), does that not require an excellent world model - somehow explicitly reflected in the weights? This is not a settled question but the last few years of LLM success have been completely on the side of those who think that token prediction is quite general.

The material also makes several comparisons to human intelligence, and while it is obvious that humans are different from language models we do not really understand the emergence of all the things that are claimed to be "impossible" for the machine to have in humans (consciousness, morality, etc), it just so happens we are all human so we all agree we have it. Furthermore, it is not clear to me that something can only be called 'intelligent' if it perfectly mimics humans in every way. This is maybe just human bias to our own experience and risks a "submarines can't swim" debate which is really about language.

Many of these philosophical objections have been questioned by people in the field and more importantly by the rapid progress of the models in tasks they were supposed to be incapable of performing according to philosophical objectors. The last few years, every time somebody claims models "can't do X" a new model is released and lo and behold, X is now easy and solved. (If you read a 6 month old paper of impossible benchmarks, expect 75% to be already solved). In fact, benchmark satuation is a problem now. In other words, the goalposts are having trouble keeping up, despite moving at high speed.

I don't think you are doing the general public any service by simply claiming that it is a lot of hype and marketing, these models are really advancing rapidly and nobody really knows where it will end. The philosophical objections seem to be rather weak and are in rapid retreat with every new model, on the other hand the argument in favor of further progress is just "we had progress so far by scaling, if we keep scaling surely we will have more progress" (induction). This is not a strong guarantee of further progress.

The claim that the labs are 'marketing geniuses" for realising language models as chat instead of autocomplete (which they "really" are according to the text - what does that mean?) also seems a bit silly given the obvious utility of the models is already much higher than 'autocomplete'. This seems to be another instance of the common bias that a model that "just" predicts the next token is not allowed to be as succesful as it clearly is in all kinds of tasks.

I don't think a lot of these opinions are particularly well founded and they probably should not be presented in entry level material as if they are facts.

Edit: just to add a positive note, I do think it is extremely useful to educate people on the reliability problem, which is surely going to lead to lots of problems in the wrong hands.

I'm amazed people are upvoting this piece which does not grapple with any of the real issues in a serious way. I guess some folks just really want AI to go away and are longing to hear that it is just all newfangled nonsense from the city slickers!

That assumes all good ideas naturally translate into viable business models, but markets don’t always work that way. Consumers don’t always have the information, leverage, or incentive to push for things like transparency—especially in industries where companies benefit from opacity.

Not everything that’s good in a broader sense aligns with what businesses are incentivized to do. The absence of a transparency-first company doesn’t prove there’s no demand—it just shows that the market structure doesn’t naturally reward it.

If you didn't need disruption, why did people choose to use the new service?

Only the answer is taken into account for scoring. The <thinking> part is not.

Conceptual reasoning machines rely on concrete, explicit and intelligble concepts and rules. People like this because it 'looks' like reasoning on the inside.

However, our brains, like language models, rely on implicit, distributed representations of concepts and rules.

So the intelligble representations of conceptual reasoning machines are maybe too strong a requirement for 'reasoning' unless you want to exclude humans too.

We have been in the 'magic scaling' era for a while now. While the basic architecture of language models is reasonably simple and well understood, the emergent effects of making models bigger are largely magic even to the researchers, only to be studied emperically after the fact.

"There are no LLMS that reason" is a claim about language, namely that the word 'reason' can only ever be applied to humans.

If he actually did it that would be far worse.

SMIC makes the Ascend 910c using its N+2 process—basically TSMC’s 7nm equivalent—and ASML’s TWINSCAN NXT:2000i DUV immersion lithography machines. But with yields stuck around 20% late last year, it's far from commercial-grade. GlobalFoundries and Intel have also wrestled with yield issues on this node, so it's no walk in the park. Expect the Chinese government to keep funding it for a few more years, but by then, the tech might be a bit dated—TSMC's been producing 7nm chips commercially since 2018.

That Article does not mention Google, and Google did not develop the Lavender system used by the Israeli.

The US mainland is not currently under threat but the US world system is.

I'm not a fan of Trump but I also feel he has not been so bad that I think that surrendering the world order to Russia and China is a rational action that minimizes suffering. That seems be an argument that is more about signalling that you really dislike Trump than about a rational consideration of all options available to us.

The risks needs to be weighed against the downside of not deploying a capable system against your enemies.

Lavender is not an autonomous weapon but if you want to seriously consider if Lavender is a good thing (I am undecided) you need to compare the effect of this operation with Lavender and the effect of doing the same operation without the Lavender system. Otherwise you run the risk of making arguments that in the end just boil down to 'weapons bad'.

The logic of this argument implies we should develop weapons with maximal collateral damage to deter their usage.

Google is a company that relies to a large extent on users to trust them with their data and for advertisers to want to be associated with them. Hence they have a stronger incentive to avoid being seen as an evil corp then some other companies. This is also important for recruitment, as many engineers do not want to (be seen to) work at a privacy invading evil corp, so it is important that google creates plausible deniability for those engineers as well.

This sentiment ignores the reality on the ground in favor of performative ideological purity - civilians are already getting blown up all the time by systems that do not even attempt to make any distinction between civilians and soldiers: artillery shells, mortars, landmines, rockets, etc.

I understand the sentiment but the logical conclusion of that argument is that the US should disarm and cease existing.

Indeed. Usually weapons are banned if the damage is high and indiscriminate while the military usefulness is low.

There is at this moment little evidence that autonomous weapons will cause more collateral damage than artillery shells and regular air strikes. The military usefulness on other other hand seems to be very high and increasing.

With or without autonomous weapons, war is always a sordid business with 'dead babies', this is not in itself a fact that tells us what weapons systems to develop.

Yes the case against death camps is not that they are a waste of time symbolic measure with no real world effects.

Good, this idea that all weapons are evil is an insane luxury belief.

You're right, the EU does have a lot of capital, but its not made available to European business.

Gleefully celebrating the defeat of my enemies is not complaining.

Developers are cheaper but not a lot, and regulatory costs are higher - the difference in the end in not huge. Plus: if you want to win a global market you need global talent at global wages. You need a lot of private capital and Europe does not have any.

Because America has the best private capital and startup ecosystem in the world it has a good chance of picking the big winners. There is no corresponding European ecosystem, only a bunch of small national ones. In fact, European investors are not betting on EU startups because they are unlikely to be able to scale to beat US and Chinese competitors due to lack of market and capital market scale.

They don't want a motivation letter to be written by an LLM (because it's specifically about the personal motivation of the human candidate) - as far as I can see this not reflect either positively or negatively on their level of optimism about AI impact in general.