Chief scientist of major corporation can’t handle criticism of the work he hypes

“What are you doing?”, asked Minsky.

“I am training a randomly wired neural net to play Tic-Tac-Toe” Sussman replied.

“Why is the net wired randomly?”, asked Minsky.

“I do not want it to have any preconceptions of how to play”, Sussman said.

Minsky then shut his eyes.

“Why do you close your eyes?”, Sussman asked his teacher.

“So that the room will be empty.”

At that moment, Sussman was enlightened.

The biggest defeating problem for pure AI teams is that they don't understand the domain well enough to know if their data sets are representative. Humans are great at salience assessments, and can ignore tons of the examples and features they witness when using their experience. This affects dataset curation. When a naive ML system trains on this data, it won't appreciate the often implicit curation decisions that were made, and will thus be miscalibrated for the real world.

A domain expert can offer a lot of benefits. They could know how to feature engineer in a way that is resilient to these saliency issues. They can immediately recognize when a system is making stupid decisions on out of sample data. And if the ML model allows for introspection, then the domain expert can assess whether the model's representations look sensible.

I'm scenarios where datasets actually do accurately resemble the "real world", it is possible for ML to transcend human experts. Linguistics is a pretty good example of this.

Not throwing any stones here, because I've been guilty of the same sort of arrogance in other contexts. But I think the same thing happened a ton during Bubble 1.0 and the software-is-eating-the-world thing. And it's hardly limited to tech: https://xkcd.com/793/

For me, at least, where this came from was ignorance and naivete. Three things cured me. One was getting deeper mastery of particular things, and experiencing a fair bit of frustration when dealing with people who didn't understand those things or respect my expertise. The second was truly recognizing there were plenty of equally smart people who'd spent just as long on other things. And the third was working in close, cross-functional team contexts with those people, where mutual listening and respect were vital to the team doing our best work.

So here's hoping that the AI bigwigs learn that one way or another.

The human will still have to look at the x-ray to see if the AI missed something. 95% accuracy is not good enough, those 5% of cases are what most of their training is for, missing it can mean a lost human life. Maybe it can be used to speed up obvious diagnoses, but it cannot be used to filter and rule anything out. The amount of time a radiologist will spend looking at the x-ray will probably not be reduced, so I don't think there's money to be saved here.

A useful productivity tool could be to examine datasets after the radiologist found nothing, as a way to double-check their reading. This won't reduce costs but might marginally improve patient outcomes. Radiologists in first-world medical systems don't really miss a lot of stuff, though.

And of course for simple obvious non-life-affecting stuff like broken bones and dental x-rays, you don't need radiologists now either. Your son's x-ray was probably not looked at by a radiologist.

There was a paper a while ago about an effort at something like this. They got the AI going and noticed that the first thing it did was classify all the x-rays by race of the patient. Then they freaked out, gave up on their original project, and wrote their paper about how AI is inherently evil.

I always like to ask myself: "what are they trying to get me to do?". Whether it's a habit, voting pattern, product, etc. Then I ask myself: "is it useful? who does it benefit?"

Whether it's true or not, idrc. what's more important (at least to me) is what it does and who it benefits

(also in this case to avoid any ethical conundrums, something is useful if it makes $, beneficial to X party if it makes them $)

But this is an interpretation after the fact, it feels exactly like you describe it. Stupor.

Edit: typo

I'm in the field. Haven't experienced people who I could have described as bros. Can you point to one example?

That is just success bias. How many times did people try out perpetual motion machines? The rest of the article tries to make the converse that if something is inadequate today, it will be the replacement in the future.

When do you consider something to be not worth spending time on?

For example, I want to have a centralized authority which can override fraud or a mistake in a financial transaction. I want laws to apply, I want them to be written by elected humans.

I'm not even sold on the value of any kind of electronic voting in general elections, since trust in the process is so vital here that in my mind the horrible inefficiencies of pen and paper and a bunch of humans manually tallying up votes in a thousand school gyms until 11 pm is actually quite okay. I'll pay for that with my taxes, no problem. Now you add blockchain into the mix, and I don't know what problem it solves that does not have superior alternative solutions.

And so on. But I'm gonna stay open minded. Technology X might one day find the perfect problem to solve, or I might realize I was stupidly wrong about technology X for some time.

In order to have an actually good discussion, we need to look at the thing and go past the "well someone criticized the Internet too and now look at it".

So taking the idea of the blockchain. What makes the blockchain a "different thing"? The differential aspect is that it allows untrusted nodes to join in a distributed architecture. It's not the only database that exists and it's not the only distributed one either. So any claims of new features brought by blockchain should justify why they need the "untrusted distributed nodes" part. If they don't, we can assume that those new features don't really need the blockchain: either it's already been done, it's not an use case people are too interested in or it's not viable due to other reasons (economical, technical apart from storage, political...). In the case of blockchain claims, most don't actually justify the need for the untrusted distribution. For example, smart contracts: it's just a fancy word for "computer program" only that it runs on a distributed trustless architecture. But is that really needed? My bank already runs computer programs that execute loan payments, for an example of things people try to implement with smart contracts.

Compare that with personal computers or even AI. PCs allowed data manipulation, storage and calculations at a capacity that was not previously available. Of course the first computers wouldn't have enough power to do things that a wide array of people would find useful, but "low power" isn't a fundamental aspect of personal computers in the same way that "low bandwidth" isn't a fundamental aspect of blockchains.

So my general skepticism regarding blockchain is that it presents technological solutions to social problems (so it won't work). AI is different since it's a bit all over the place. In principle, it aims to solve problems that are obviously worth solving, but as long as it will fall short of its promises, the partial solutions we do get are kind of a mixed bag: if we need to keep a human in the loop or at the wheel, it's suddenly a lot less attractive. And the path to the AI being good enough to be trusted is nebulous at best. But we'll see. As for PCs, their utility has always been obvious and the roadmap clear.

Why? There's probably plenty of usage of ML where both the initial training set, its users and its outputs are internal to one company and hence well-controlled. Why should such a model be constructed with adversarial usage in mind, if such adversarial usage can be prevented by e.g. making it a fireable offense?

Sure you want quality here but there’s always going to be a human in the loop for this kind of work.

Any workflow with a human in the loop has this speed vs accuracy tradeoff.

While I’m not saying that speed trumps accuracy here, I don’t think you can dismiss without evidence that the tradeoff exists and speed might have benefits.

When is “improved structure prediction” useful or important?

Often a process is simplified or distilled down to a sound bite for the general population. Then we simply repeat without understanding the details.

Given that improved structures has sped up drug discovery, I can see where the mistake is made (X has improved Y, therefore X will improve Y)

If you want to steelman the GP's argument, you should compare the web app with e.g. some niche in pure math. There the trade off between novelty/interest and usefulness to people today is more clear.

I think the two are incomparable and both useful, but it's disingenuous to strawman the GP as saying web apps are more useful than relativity.

> None of it would be possible without academia though. Industry just applies academic research.

Meh, that vastly oversells academic research. Very little of academic research in computer science is actually used in the industry. It's not that the industry is ignorant, but rather that the majority of academic work is useless: They create artificial problems [1] and solve them in shoddy ways, with hand-picked benchmark results, and frequently without even publishing the source code.

It's probably not surprising, given that the typical incentive is to get a PhD. So you need a "problem" that can reliably be solved in 3-5 years and which allows you to produce 5-10 conference papers with your name on it.

[1] I'm not talking about theoretical fields – my comment is purely about supposedly practical research.

I worked at Google and there's just tons of stuff that never actually existed in academia and was created, launched, and then replaced by something better entirely within the company without any publications!

How often have you talked to someone at an institution about some egregious wrong, and they defend it as "that's our policy?"

Before 2016 or so, "AI Safety" meant the types of things I listed above: how to design AI systems that are safe to use in safety-critical settings.

Unfortunately, the term "AI Safety" has been overrun with effective altruist longtermism folks talking about something closer to https://en.wikipedia.org/wiki/Existential_risk_from_artifici... which is ill-defined and, frankly, usually completely vapid nonsense.

> so the question of how to mitigate its risks is going to be at least as squishy a concept as the underlying tech + applications.

No, no, no!!! AI safety methods -- methods that really increase confidence in real systems -- are not generic. ALL of the useful AI safety work I know of is deeply related to one or more of: the specific model architecture, the specific optimization algorithm, the specific data, or the specific application. Almost always all of the above.

That's just wrong. This very paper we're discussing has AI safety factors built-in: the AI is not supposed to be able to lie, and yet it exhibited deceptive behaviours anyway. That falls squarely under AI safety, and is a pretty useful observation to inform future attempts.

> The current state of so-called "AI" is our fire. It's impressive and useful (and there are real dangers associated with it) but it has no bearing on intelligence, let alone superintelligence.

That's conjecture. We don't know what general intelligence really is, so you simply cannot gauge how close we are. For all you know we could be one simple tweak away from current architectures, and that should be terrifying.

> As it stands, we don't have the understanding or the tools to do anything useful wrt safety from a super-intelligent AI.

Even if that were true, which it's not, we certainly won't develop that understanding or those tools if we don't start researching them!

> We do have the understanding and tools to do something useful about asteroids and climate change.

Suppose you had a time machine and went back to the 19th century to explain the dangers of asteroids wiping out all of humanity. They didn't have the launch capability or the detection abilities we do now, but if they were sufficiently convinced of this real threat, does it not seem plausible that they could be motivated to more heavily fund research into telescopes and rockets? Couldn't we plausibly have reached space sooner and be even better prepared in the present to meet that threat than we are now?

Instead of a time machine, couldn't we just use our big brains to predict that something might actually be a serious problem in the future and that maybe we should devote a few smart people to thinking about these problems now and how to mitigate them?

This all just strikes me as incredibly obvious, and we do it in literally every other domain, but somehow say "AI" and basic logic goes out the window.

The intelligence is what determines the shape of these networks so they can even learn a useful pattern. That’s still gonna be humans for the foreseeable future.

Your ability to read this sentence right now when we have never met and may not even be on the same continent was once in the domain of science fiction. Don't underestimate technological progress, and specifically, don't underestimate the surprising directions it could go.

Some fantastical AI predictions will happen, most probably will not, and some utterly terrifying ones no one foresaw will almost certainly happen. The unknown unknowns should worry you, and AI is full of them.

That's just incorrect. The Tunguska event was a nuclear-weapon scale asteroid. These are predicted to happen once every hundred years or so. If it happens over a populated city millions would die. If a person wrongly concludes this was a surprise nuclear attack, maybe everyone would die, to say nothing of the real risk that the asteroid itself could be big enough to wipe us all out.

There's a lot of uncertainty around climate change, but changing climate patterns will certainly change resource allocations (fresh water, arable land, etc.). This will lead to shortages in places that were once abundant, which could easily lead to wars in which millions die.

> Really stuff like asteroids and climate is a good counter-argument to caring about AI risk.

This. This boggles my mind. Long-tail risks exist, and burying your head in the sand and pretending they don't just places millions of lives at risk, and potentially the entire human race. You don't have to think these are top priorities, but to dismiss them as complete unimportant is frankly bonkers.

Who are you talking about exactly? Who are these alleged grifters and who are they grifting?

> There is zero actual scientific evidence to support their claims

What claims are you talking about, exactly? Let's get specific.

The fact that we don't know how close we are is itself dangerous. It's like doing gene editing on pathogens without a proper understanding of germ theory and biosafety. That's where we are with AI.

2. AI safety research attempts to mitigate some AI dangers.

3. Therefore we must fund AI safety research.

Yup, checks out.

If they had done an adequate and accurate job explaining what, if any, plausible value or potential Galactica had, warnings would not have to have to carry so much weight. Or their PR could have focused on their tricks and optimizations, and not characterised Galactica at all.

Instead they'll be known as the people who described a science flavored word salad generator as if it was a tool useful for "summarize academic literature, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more"

If they had, for example, positioned this as a tool for creating unscientific, but academic sounding anti-vax propaganda, people could have questioned their morals, but not the "fitness to purpose" of their tool.

You have the choice to avoid google accounts and limit the destruction a google AI system can do to you.

You don't have a choice to not be born black, and not be put in jail for longer just because you are black.

Why don't you care that millions of people will be hurt by these things, and care more that an app developer gets locked out of the app store? Apple hasn't put anyone in jail.

These AI ethicists are complaining about all of this, but of course they yell more loudly about sexism and racism, because, you know, those are fairly serious things that should be addressed first???

I don't think they need to be original, I think they need to bang their drum loudly. "Oh, that women's suffrage movement won't shut up about how they don't have a voice in policy that governs their life, can't they talk about something else for once" isn't an indictment of the people complaining, it's an indictment of the people not listening.

[0] https://www.propublica.org/article/machine-bias-risk-assessm...

[1] https://www.propublica.org/article/yieldstar-rent-increase-r...

[2] https://www.science.org/doi/10.1126/science.aax2342

Sad because I am an Ars Technica fan, but can empathize with GP's point about AI coverage being repetitive and often over-hyping results.