2,735 karma · joined February 28, 2020
Of course that is terrible, but it is one of the worst examples you could have given to make your point.
The fact that we don't know why is a clue pointing at some area of math that we haven't discovered yet. The hope is always that it will uncover some hidden fertile valley that will lead to lots of new discoveries. But the proof of the conjecture itself, without understanding, is really not that valuable.
My point is that even if AI discovers many new truths, there's still plenty to do for the mathematical community, in dissecting it and building useful abstractions to understand it, abstractions that can be leveraged for further exploration and uncovering new questions.
What an intriguing thing to say :)
What are you referencing?
It may be true that regular agents trained for general purpose use do not behave this way, but they seem to be capable of learning such cheating behaviours when relentlessly being fine-tuned towards near-impossible objectives.
In this sense, it is not really fair to say that the agents found these solutions. It was the surrounding learning framework that achieved this, which is a much more powerful problem-solving mechanism. As users we do not have the capabilities or budgets to be able to tackle our own problems like that, we have to make due with the frozen behaviour the AI labs trained for us.
Individuals are unlikely to give up their control unless the alternative is clearly substantially better. And, as you say, there are other larger interests beyond individual preferences that need to be overcome too.
Regardless, as we have seen many times with other technologies, I think that convenience will be a much stronger driver of self-driving acceptance than the safety benefit.
People understandably prefer to keep control of their own safety, even if on average they are worse at it. Because individuals that do care about safety can actually do many things to lower the risks well below the statistics, and likely remain much better than self-driving.
There’s also the question of responsibility and liability. In society we have such pressures so that there’s an incentive to continually reduce risks. It is dangerous to shift the responsibility to fast-growth tech companies that will care much less about individual accidents than the drivers involved, and are more capable of absorbing liability without substantive remediation.
Safety is outsourced in many contexts (aviation, police, military…), but it has to be substantially better than what the individual can achieve on their own. For instance, due to the expertise required, or because you can achieve more through coordinated collective action. And there has to be very high trust in the companies involved, the regulatory systems around them, and the technologies employed.
> They're Made out of Meat
https://web.mit.edu/people/dpolicar/writing/prose/text/think...
For once it’s appropriately used.
If one attempts to have a single definition stretched over all the meanings, then you indeed get a definition that is way too broad that includes soups and sandwiches, which is ludicrous. But it's quite common for a single word to have multiple senses, let's just go with that.
- A wedge salad is right on the edge. It's a traditional salad except that the lettuce leaves are not separated. But if it was anything else in that form factor, like replacing the lettuce wedge with a potato, that's not a salad.
- A primarily cooked dish, particularly if it is cooked all together or is served warm, is definitely not a salad.
- An “untossed salad” is definitely not a salad.
- English speaking people add the term “salad” to a bunch of concoctions that are definitely not salads.
The time it takes to press a key is a reasonable target to aim at for API latency I suppose, but it is still an arbitrary target. Waiting to display until keyUp just adds more latency if your API is faster. Having it synced with keyUp doesn't make it feel more immediate to me.
That is “the people” isn’t it? It is the consequence of democratization and openness.
I am not sure if superstars were good representatives of what most people wanted. Often, I am sorry to say, it was the lowest common denominator. Now common people can actually enter the arena and represent their corner of the population.
But, almost by definition, if you are part of “the people”, you don’t get to stand out much.
For the time being, unless you truly have millions, the outcome from training will be very net negative, while focusing on building on top of existing AI will yield amazing things if you apply the same talent and effort.
When it does get cheaper, then it will be easier to acquire the skills and experience too, and the struggle you went through by trying to do it now will be somewhat wasted.
Besides, I am well versed in this field, and it is not rocket science. There are plenty of software engineering domains that are a lot more challenging, like high-end graphics, large-scale data engineering or kernel programming. People will learn to train LLMs when people want them to.
The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM company, and still then it's a struggle. Those few that do train, they spend most of their budget on compute and have relatively small teams.
Getting experience in this field requires having access to very expensive hardware to begin with. And the skills will be quite hard to convert into any real value for someone, leading to a decent income, unless you have a ton of funding from patient investors, or you have decent contacts in Bay Area networks to get hired at the right place.
With all due respect, paulg is in somewhat of a bubble, this is not congruent with the global situation.
Often they do sell it for parts, or they enshitify the hell out of it to squeeze revenue from loyal customers. Not necessarily because it's the optimal strategy, but because truly fixing a business is hard and these are decent shortcuts from their perspective.
But that's not a given, sometimes they do truly turn it around for the better.
But indeed, if Pangram is marketing itself as "near-zero false-positives" now, that entails that the are aggressively biasing it towards being conservative, likely leading to lots of false-negatives and low match scores on AI content.
But the analysis of the internal admin pages for generating articles, the API and the interview structure are pretty damming evidence that this is largely driven by AI.
Their revenue has always been sustained by the fact that their technology needs to be constantly replaced because it keeps getting better. The moment it stops getting better, the replacement rates plummet and so do their revenues. It's also really not that hard to compete with them when they get complacent.
I'm not counting Nvidia because they don't produce semiconductors themselves, they are a different kind of business.
They are indeed an example of those that cater to consumers and/or build popular products based on foundational tech from others, like Apple or Sony, which do tend to be quite profitable.
But the actual deep-tech semiconductor firms? They may be critical to the world economy, but they don't actually make that much money comparatively. In many cases they are not real monopolies, it's just that no one else wanted to continue investing in a shitty business model. Only the likes of TSMC and Samsung were okay in playing the low-margin game, but most US players left the board.
I believe AI has a lot of the same characteristics.