Hacker News Reader

That's actually something ML is incredibly useful at, when it comes to machines with sensors - failure prediction / anomaly detection, etc.

In the industry, (preventive) maintenance takes up a pretty huge chunk of resources. It's something techs need to do often, and it's often a laborious task, but it's obviously done to reduce downtime.

So the business insight, as they like to call it, is to reduce costs tied up to repairs and maintenance.

All critical applications have multiple levels of redundancy, so that a complete breakdown is very unlikely, but it's still a very expensive process if you're dealing with contractors. If you can get techs to swap out parts before the whole unit goes to sh!t, then that's often going to be a much cheaper alternative.

But, in the end, it comes down to the quality of data, and the models being built. A lot of industrial businesses hire ML / AI engineers for this task alone, but expect some magic black-box that will warn x days / hours / minutes ahead that a machine/part is about to break down, and it's time to get it fixed. And they unfortunately expect a near-perfect accuracy, because someone in sales assured them that this is the future, and the future is now.

Yep, you and the user you're replying to are both right in different ways. One thing's for sure - machines don't generate "insights" on their own.

Let's define an "insight" as "new meaningful knowledge", just for fun. We could talk about what comprises "new" and "meaningful" but it would be beside the point I'm making.

In a supervised learning problem, the range of possible outputs is already known, meaning the model output will never be categorically different from what was in the training data. The knowledge obtained is meaningful as long as the training labels are meaningful, but it can never be new.

Unsupervised learning doesn't have a notion of "training data" but that means an unsupervised model's output requires additional interpretation in order to be meaningful. It is possible to uncover new structures and identify anomalies in new ways, but this knowledge isn't meaningful until someone comes in and interprets it.

Applied to the specific example where sensor data is used to try to generate insights about machine functionality: Either you can only predict the types of failures you've already seen, or you can identify states you've never seen but you wouldn't know whether they mean the system is likely to fail soon or not.

It's the Roth/401(k) tradeoff. For model output to be useful, someone must pay an interpretation tax. The only choice is whether it is paid upon insight deposit or withdrawal.

> Either you can only predict the types of failures you've already seen, or you can identify states you've never seen but you wouldn't know whether they mean the system is likely to fail soon or not.

Yup, this is something I've seen from both sides. First you mention is basically the standard, while the last is part of the deep learning voodoo black magic that executives and sales love.

I've had people approach me with proposals like "What if we just churn [ALL OF] our data through this or that model, and let's see if it comes up with some patterns we've never seen or thought about"

And that's not just for industrial applications. It's everywhere.

What is concerning to me is that this mentality will surely induce more unrealistic expectations. Before you know it, business execs are starting to ask why we need business analysts at all, because surely those fancy deep neural networks can extract all kinds of features - "only need data scientists to figure out those things".

So yeah, that's my fear. That businesses will blindly start to discard domain knowledge, and just feed black-box models their data, and let the data scientists wrestle with the results.

It’s not only outputs/labels that provide “insights”.

Knowing how the outputs relate to the inputs is where most new insights could come from.

For example, what feature (input) is driving the “failure” of the machine (output/prediction)?

This is where ML explainability comes in.

This is demonstrably false; AlphaGo made significant new discoveries, for example.

Yeah this is where it would have helped if I had discussed what I meant by "new".

AlphaGo is a supervised learner that outputs optimal Go moves given opposing play. It yields new discoveries in the same sense that a model designed to predict mechanical failures from labeled sensor data would: I didn't know what the model was going to predict until it predicted it, and now I know.

But what the factory owners want is a machine that can take raw, unlabeled sensor data and predict mechanical failures from that. They want insights. "Why not just feed all our data into the model and just see what comes out?" they ask. "I don't see why we need to hire at all if we have this neural net."

The reason you need a human somewhere in the system if you want insights is because someone needed to program AlphaGo specifically to try to win at Go. At the factory, someone needs to tell the machine what a mechanical failure is, in terms of the data, before it can successfully predict them.

Then, neither "winning at Go" nor "mechanical failure" are states that the system hasn't already been programmed to recognize. That's what I mean when I say a supervised learner cannot generate "new" output.

If the business wanted to track the rate of failures and create predictive models about when things fail, or detect anomalous behaviour, that's what they would have set out with as the goal, and, perhaps, some ML model might have helped, but probably, it would've been too unreliable and any number of standard predictive models with well known characteristics would have been used instead.

That's not what they wanted.

What people are being sold is AI/ML as a magic bullet that will do something useful regardless of the situation, and it lets business people avoid making decisions about what they actually want, because AI/ML can be anything, so they just signup for it and expect to get 20 things they didn't know they wanted handed to them on a plate.

Turn out, it's not enough to just collect a bunch of data and wave your magic wand at it. It wasn't with web analytics 10 years ago, it's still not.

What you actually need is someone who has a bunch of tricks up their sleeve, and has done this before, and can suggest a bunch of Business Insights the business might need before they start building anything, people that actually decide what to do, and actions taken to investigate, and solve those problems.

I mean, to some degree you're right; perhaps ML models could be useful for tracking hardware failures, but that's not what the parent post is talking about. The previous post was talking about just collecting the data and expecting the predictive failure models to just jump out magically.

That doesn't happen; it needs a person to have the insight that the data could be used for such a thing, and that needs to happen before you go and randomly collect all the wrong frigging metrics.

...but hiring experts is expensive, and making decisions is hard. So ML/AI is sold like snake-oil to managers who want to avoid both of those things. :)

Projects rarely end up doing what was planned when they started. As long as ML is solving real problems in practice, upper management will keep treating it as magic fairy dust to sprinkle around aimlessly.

It's all about how you package things. ML connected to an audio sensor could predict failure modes that are dificult to detect otherwise. Now that might not be was was asked for, but a win is a win.

I think both your post and the one you are responding to are correct.

I’ve experienced what the OP was alluding to...namely, it helps tremendously to start out with an understanding of the problem you’re trying to solve, more so in supervised learning. It’s incredibly frustrating to ask managers what business problem they are trying to solve only to be met with “We don’t know, that’s what we want the software to tell us.”

On the other hand, if they say “we want to predict machine failures” or “reduce maintenance downtime” now we have a lens in which to view the data.

If AI could do the magic as those managers hoped, they would be out of a job

I work in this space, and you've called out the thing that drives me absolutely bonkers. No matter how accurately I can diagnose the state of the equipment, what everybody wants is to know how long until it explodes, so that they can take it down for maintenance ten minutes before that.

This is a good explanation of how things work and things haven't changed much in the past 5 years. What changes the most is the amount of data and computing power ("compute") that are used in training neural nets.

Architectures are also tweaked slightly although they continue to be largely based on the two architectures that started the current deep neural net boom, Long-Short Term Memory Networks (LSTMs) and Convolutional Neural Networks (CNNs), which are largely used for natural language processing and machine vision respectively.

Every once in a while there is a bit of excitement about results that come out of adding a new parameter which is then given a fancy name like "attention" and generates much breathless copy about how neural nets will soon become as intelligent as a dog/ your three year old/ your personal assitant etc. Usually though it's not clear whether these new parameters or architectural tweaks are really responsible for the success of the proposed techniques and when one looks at the field as a whole it becomes more and more apparent that the most successful work is backed by the most data and the most computing power.

To slightly er tweak your prediction, I think what'll happen is that, indeed, like you say, there will be libraries and APIs and so on (well, there already are) but all these will be controlled by large companies that can afford the ground work. So, I don't expect small outfits, research teams at universities or tiny startups, to make a big difference. Academia is fast losing the ability to produce work that beats the state-of-the-art anyway (that's from personal comms with other researches). So in terms of academia perhaps we should expect a shift away from neural nets, to something that's easier to do research on.

> but all these will be controlled by large companies that can afford the ground work. So, I don't expect small outfits, research teams at universities or tiny startups, to make a big difference. Academia is fast losing the ability to produce work that beats the state-of-the-art anyway (that's from personal comms with other researches).

That makes me particularly sad. In a utilitarian sense, the ideal situation would involve new approaches being democratised. Instead, we're ending up with walled gardens.

The current situation isn't dissimilar to what has happened to the Web Platform during past 5-10 years. And, similar predictions are made wrt VR and AR (both seem to attract different types of investors).

To look at a beautiful reframing of AI as a tool and not as a quasi-other agent, please enjoy this page: https://nooscope.ai/

The best way to think of today's AI capability is: automation.

You train it with a lots of examples of "given this input, this is the output i want", and hopefully it learns to get the "correct" (similar input => similar output) output for new inputs that you feed it. i.e. you've now automated the process of figuring the correct output for a given input.

There is also the "reinforcement learning" AI paradigm where the trained AI is choosing actions (from a given repertoire) in order to maximize an action outcome score based on some scoring function you provided. This is appropriate in a situation where you want the "AI" to do something more than just select the correct output, but again no magic - you're having to anticipate and score the potential outcomes.

In my opinion this type of tasks is the typical AI tasks that get most exposed to the general public. The AI/ML model is set to attempt a human task. The aspiration is to get as good as a human (or faster/better more precision etc). Classic statistical inference/ models usually don't perform very well in these scenarios(or do they?).

The typical AI methods train a model that utilises features that don't make much sense to humans (pixels/texture/word tokens etc). In contrast with traditional statistical modelling where each feature were given 'meaning' and their importance explored via investigating the observations (data) using properties of well studied mathematical models, the AI methods that often made the press don't put a lot of emphasis on these properties. The advancement is a push to utilise all available information/data to outstanding 'performance'. The explanation of the inner features within that math space is usually secondary (though I don't mean that authors of novel methods don't care about mathematical modelling).

I might be too naive here but that's how I feel after trying out many methodologies in my field.

The "AI" capabilities that are making headlines nowadays are mostly based on deep (multi-layer) neural networks with millions, or billions, or parameters. These nets do self-organize into a hierarchy of self-defined feature detectors followed by classifiers, but as you suggest for the most part they are best regarded as black boxes. You can do sensitivity analysis to determine what specific interval values are reacting to, and maybe glean some understanding that way, but by-nature the inner workings are not intended/expected to be meaningful - they are just a byproduct of the brute force output-error minimization process by which these nets are trained.

Neural nets are mostly dominant in perceptual domains such as image or speech recognition, where the raw inputs represent a uniform sampling of data values (pixels, audio samples) over space and/or time. For classical business problems where the inputs are much richer and more varied, and already have individual meaning, then decision tree techniques such as random forests may be more appropriate and do provide explainability.

>Neural nets are mostly dominant in perceptual domains such as image or speech recognition, where the raw inputs represent a uniform sampling of data values (pixels, audio samples) over space and/or time.

I mostly agree with that, however there have been advancement in scientific areas such as chemistry and biology (DNA/RNA) where the data are definitely meaningful and a lot times categorical. So the methodologies can be applied in wider areas, they just need a lot of domain knowledge and experience.

> the possible MNIST digits are 0-9

Except - and this rather ties into your point - those are not the only possible digits; your network also has to deal with (ie reject) other possible digits such as "P", "E", "3̸̶", or "[Forlorn Sigil of Amon-Gül redacted]"[0], which look like, but are not, decimal digits.

0: https://www.youtube.com/watch?v=ajGX7odA87k

That depends on how you train it.

If you only train it on examples of 0-9 then those are the only outputs it's going to give. If you fed a "P" into such a net then the outputs would be the degree of similarity of that "P" to each of the (0-9) digits it was trained on. You could of course threshold the output and ignore any prediction with confidence less than, e.g., 90%.

If you wanted the net to do a better job of rejecting non-digits, or at least some specific ones, then you could include a bunch of non-digit examples in your training data (so now your net has 11 outputs: 0-9 and "non-digit"), then hopefully - but not necessarily - it's highest confidence prediction will be "non-digit" when presented with a non-digit input.

> If you only train it on examples of 0-9 then those are the only outputs it's going to give.

Exactly.

> 11 outputs: 0-9 and "non-digit"

IIRC, this doesn't work (or not well) because the net tries to find similarities between the various members of the set "everything except these ten specific things", but you could just require low confidence for all digits on non-digit inputs as part of the gradient descent function.

The problem is more that - if people with decision-making authority trust the AI to not be insane and evil by default - failure modes like this have to occur to you before the AI starts misbehaving in production.

> currently ML/AI requires us to know what the possible answers can be before we even begin training the network

Isn't that what unsupervised learning is for?

I took it more to mean that, e.g., when solving for 1+1 it won't be allowed to answer "tomatoes".

There's no such thing as truly unsupervised learning - any machine learning algorithm is defined by some guidelines you've provided (or built-in) for what output it has to generate.

For example, you might think of data clustering as an unsupervised problem, but in reality you're still providing supervision in terms of what similarity measure to use and some control over how many clusters it should generate; and, at the end of the day, the output is always going to be a set of clusters, not, say, a stock tip or wry comment on the nature of your data!

So, uh, how do you talk to the C_Os about this and tell them that it's GIGO?

Asking for a friend.

You get a co-worker to do it.

Tell them ML is so smart it's almost as smart as a C_O.

So if they can solve the problem the ML probably can too ...

Things may have changed over the past 5 years or so.

Things may have changed over the past 5 weeks or so with GPT-3.

Until GPT-3 can write something meaningful, it's really just a showcase of the technology and a gimmick of a product.

Sure it's cool, but what problem is it solving? As far as I can tell the only useful function it has is polluting the internet with pseudo-intellectual comments to promote some agenda (likely political). So now that I think about it, it actually would be incredibly valuable for things like subverting democracy.

I'm still waiting for AI to figure out self-driving cars, natural language processing, and other "holy grail" problems that researchers have been chipping away at for decades but we are still a long way from having "solved" them.

Also in the case of self-driving cars, there is actually a very serious problem of it being difficult to guarantee safety. You can't tie every input to every output to demonstrate how it will behave (it is way, way, wayyyyyyyy too big for that), so you have to show statistically by driving so many miles that it's better. Which is still sound from an engineering perspective! But it leads to things like Teslas driving into trucks parked sideways across the road because the engineers simply never predicted it would ever happen. And now it's happened multiple times!

Until GPT-3 can write something meaningful

What is "meaningful"? Honest question. Isn't meaning assigned by a reader? If I'm reading poetry generated by GPT-3 and I like it just as much as poetry written by a human poet, does it make it meaningful? What if I finetune GPT-3 (or the bigger next gen version) on every scientific paper ever written, and as a result it generates a novel idea that turns out to be valid and useful, should I care if it happened by accident, or without "understanding"? Can the knowledge encoded in the model parameters be interpreted as some form of understanding? If no, why not? What's missing exactly? What is different from how human scientists operate?

Well you can randomly string words together and occasionally get lucky and make a meaningful argument, but that doesn't mean you've created a good method for constructing new ideas.

It seems to me the simplest way to decide whether or not something is meaningful (in this context) is whether or not the author (which is the algorithm GPT-3) can respond to criticisms against its own argument in a coherent way. In which case it has to get lucky twice, so it's that much less likely to happen randomly. If the author cannot respond to comments in a comprehensible manner, it's hard to defend the author.

can respond to criticisms against its own argument in a coherent way

I'm not sure about GPT-3, but let's imagine GPT-4 next year will be able to do this. It just does not strike me as a particularly high bar to clear. Let's go further, and assume GPT-5 in 2022 will pass the Turing Test (you personally will not be able to tell). What would you say then?

Please don't wildly speculate about technologies you clearly don't understand. "Does not strike me as a particularly high bar" means you're unfamiliar with how these systems work at a deep level (by which I mean, you personally cannot sit down at a terminal and build one). So rather than -ahem- making things up and asking "what then?" -- please ask for textbook recommendations on these topics if you'd like to know more.

I guess we'll just have to wait and see what happens next year :)

p.s. I built my first language model (LSTM based) back in 2014. Then I built a VAE based one. Then a GAN based. None of them were especially good, so I switched to music generation (this actually works pretty well). My most recent project is using sparse transformers for raw audio generation. Building novel DL models is literally in my job description.

Please don't wildly speculate about strangers you meet on HN.

Why just wait when you can bet?

In order to bet, we would have to agree on evaluation criteria. A task like "respond to criticisms against its own argument in a coherent way" is difficult to evaluate. Turing test is also pretty vague, and some people already declared it passed many years ago: https://www.bbc.com/news/technology-27762088

Yes, it would be difficult but not impossible to agree on clear criteria. We would also need an impartial third party to make a determination.

The Turing test hasn't been passed if I'm the judge. Supposing I were unable to tell the difference between any AI system and a human interlocutor defending the same argument, at any time before 2023, I'd admit my prediction was wholly incorrect.

I doubt very much I'll find a taker for this bet, however. The AI field has always been bigger on optimism than results, as we both know.

Yes, I agree TT has not been passed. But probably starting later this year we will be seeing more and more claims it has. At first it will be clear it's not. Then not so clear, and then the goalposts will be moved again, so that when GPT-5 is announced and it is clearly capable of keeping a conversation, the reaction on HN will be the same as the current reaction to GPT-3: "meh".

That seems like a pretty testable prediction. If I can't tell the difference between GPT-5 or whatever it is and an adult human native English speaker by the end of 2022, you'll win the bet.

How many questions will you need to determine if it's a bot? How many sessions, and what percentage of correct guesses will determine the outcome of the experiment?

If it wasn't a bot but a human, we'd typically have an unbounded conversation, except by the bounds of politeness, until I was satisfied one way or the other, so that seems a reasonable protocol here, perhaps with some reasonable upper bound on the time spent (an hour?) to avoid putting a potentially unbounded commitment on human participants. (Actually, "We've been chatting since four, I have somewhere else to be" is a pretty good signal of humanity... and should help you see why you're not going to win this bet.) I wouldn't expect it to take more than a few minutes, and I would expect to be wrong in (much) less than 1% of sessions.

The more time you spent chatting the better your chances are to guess correctly. I admit that in 2 years the models might not be good enough to fool you for hours. If you put enough thought into it, especially knowing what the bot was trained on, you could devise a set of tricky questions which would expose it.

However, I believe the models will be good enough to fool you during, say, a 20 question/response dialog. They will definitely be able to fool vast majority of unsuspecting humans. And they will definitely be able to keep track of conversation (remember what you said previously, and use it to construct responses to follow up questions).

How will the bot answer "when and where were you born, and how do you know?" How will it answer "what color, besides red, best communicates the flavor of a strawberry, and why?". How will it answer "What historical figure does my communication style make you think of most, and why?", or "Which of your family members comes to your mind first?" or "What do you think the context was in which the following poem was written?". I don't need to know what it was trained on to win this bet, and 20 open-ended questions is more than enough.

Between the vast majority of unsuspecting humans and me there is a considerable gap. Mind the gap!

You are kidding, right? All these questions you provided are extremely simple to answer, compared to many other things clever human interrogators might say during TT. I'm starting to doubt your NLP expertise.

The TT ready model I'm envisioning will be trained on many billions of chat sessions. It will contain dozens of preconstructed graphs and will dynamically construct dozens more (personality graph, common sense knowledge graph, domain specific knowledge graphs, causality graph, dialog state graph, emotional state graph, etc), it will have a bunch of emotion detectors, humor detectors, inconsistency detectors, lie detectors, praise detectors, etc. It will have the ability to query external sources (e.g. google search --> web page parsing --> updating relevant graph). All these modules will filter, cooperate, and vote, providing input to higher level decision making blocks. These blocks will use those inputs to condition and constrain response generation process. This is finally where a language model comes in, and this until recently has been the hardest part - generating a coherent, grammatically correct, interesting text, directly addressing a specific prompt. This part has been solved. Until GPT-2 last year we simply could not generate high quality text. Now we can, and GPT-3 is even better at that. Sure, there are plenty of non-trivial problems left to solve, but I don't view them on the same level of difficulty - some of them have already been solved in the process of IBM Watson development, so I'm optimistic. The hardest remaining challenge is probably constructing common sense graphs. [1] looks promising.

p.s. your questions are so naive I'm not sure if you're trolling me. A human might answer them like this (and a bot built 50 years ago could easily imitate that):

"when and where were you born, and how do you know?"

- [personality - redneck] I was born on a farm in Oklahoma. How do I know what?

"what color, besides red, best communicates the flavor of a strawberry, and why?"

- Red is the right color for strawberries.

"What historical figure does my communication style make you think of most, and why?"

- You talk like one of them big city hipsters.

"Which of your family members comes to your mind first?"

- My little bro Jimmy, we just went fishing together on Tuesday.

"What do you think the context was in which the following poem was written?"

- [depends on the poem] I don't get this poem. What is it about?

[1] https://arxiv.org/abs/1906.05317

You're betting that in the next roughly two-and-a-half years, the common sense problem will be solved well enough to fool me, despite not having been solved in the entire history of AI up until now. I'll take that bet. How confident are you?

To fool you for 20 questions, yes. I'm ~70% confident, so I'll bet you $100 :)

To clarify, the common sense problem is a hard one. It is similar to level 5 autonomy driving. That will take a while to solve. But what we are talking about here is kinda like Waymo cars which can drive themselves in ideal weather at slow speeds in Arizona. So in 2.5 years I think the best chatbots will be as far from having common sense as the current Waymo self driving cars are from level 5 autonomy. Which is to say they will be pretty good.

You're on for $100. I'm > 99% confident that I won't be fooled by any AI before 2023, and would have bet any amount that I could afford to set aside.

Ideal weather at slow speeds, with a professional human driver throwing road-condition curveballs at you and challenging your responses? I like my chances.

Sounds good! My email is in my profile, if I'm wrong I'll pay up! :)

Keep in mind that you would have to differentiate bot's responses from those of an old truck driver, a snobby philosophy student, a conspiracy theorist, a stoner, a bimbo following latest Kardashians news, etc. Lots of different personalities of real humans who can throw curveballs at you during a chat session. I hope you don't expect to only chat with bay area devs? :)

I'd be very disappointed if I only get bay area devs (since that's a miniscule fraction of the people I've had the pleasure of having interesting discussions with during my life). And indeed, "give me a few sentences about your background, schooling, and interests" is an excellent opening question ;)

The thing is, if we could write a specific, closed-end, prescriptive definition of "meaningful" or "understanding" or whatever, then we'd be able to program it. And we can't, so we have to settle for something else, usually how a thing fails to be what we (indeed subjectively) consider meaningful. Still, it's not arbitrary.

The way that something like GPT-3 tends to fail basically is that you 2-3 paragraphs where paragraph 3 will containment statements that subtly contradict the semantics of paragraph 1. Things like seeing things inside closed drawers and other cues that the thing has no fixed world-model. That's what gives an impression of "meaningless" or "no understanding". (admittedly, I've only played with GPT-2 but this is a description of how texts written by these models at first seem plausible and then implausible as you read more).

It's not some philosophical objection akin to "nothing but a human brain can think".

the thing has no fixed world-model

What is "world-model"? What makes you think GPT-3 does not have some kind of a world model? It's clearly not a very good one, but at the same time it does not mean it can't get better. A 3 year old also does not have a very good world model, what's the difference between his world model and one of GPT-3? Again, clearly there's a big difference, I'm just not sure we know enough about what's going on inside 175B model to make any dismissive statements about it. I'm also not sure what will happen if you train a much bigger model on much bigger data. Things might start to emerge at some scale.

We know much more about the world-model of GPT-3 (which is none, but it has a language model) than we do about the three year old (which we can't build or reproduce in silico) but we know plenty well enough to say that no, things are not going to magically "emerge" at some scale. You need something fundamentally different from endless cloze deletion exercises to learn what a drawer is.

> What if I finetune GPT-3 ...

> if you train a much bigger model on much bigger data...

Yep, no one is denying that the potential exists for something to be created that can do something useful.

However, the point being made is that GPT-3 doesn't do anything particularly useful.

The generated text from is not consistent, and apparently fine tuning can result in poor performance (ie. generates random crap). It scored better at a bunch of metrics, which is great, but of questionable practical value.

"world model" or not, it's currently interesting, amazing... probably not useful unless you're writing a spam bot.

What is unclear, is if bigger models will actually solve that or not.

Well, GPT-3 isn't any kind of general intelligence - it's explicitly architected as a language model - something that learns to pay attention to prior context to predict what word comes next. The only kind of world model it has is a statistical model of what word is most likely to come next based on the corpus it has been trained on.

You could argue that general intelligence is also based on prediction, and that a human's world model therefore isn't so different in nature, but there are some very significant differences ...

1) GPT-3's model is based only on a corpus of text (facts, lies, errors, etc) it was fed... there is no grounding in reality.

2) GPT-3 is only a passive model - it's not an agent that can act or in any way attempt to validate or augment it's world model.

3) GPT-3 is architecturally a language model .. it can get better with better or more data, but it's never going to be more than a language model.

The difference between a 3-year old's brain and GPT-3 is that the 3-year old's brain is not a one-trick pony ... it's a complex cognitive architecture, honed by millions of years evolution, capable of performing a whole range of tasks, not just language modelling.

The 3-year old's brain also has the massive advantage of being embedded in a 3-year old autonomous agent able to explore and interact with the world it's world model is representing... It you tell GPT-3 pigs can fly then as far as it is concerned pigs can fly, whereas the 3-year old can go seek out pigs and see that, in fact, they can't.

In communication, meaning is a collaboration between writer and reader. The writer does their best to convey something; the reader does their best to understand.

There's also the kind of meaning that scientists and researchers talk about when they extract knowledge from data. That's pretty different from communication; it's more a process of internal generation of notions and explanations that could later be conveyed in communication.

And then there's the meaning that is even more internal. E.g., reading tarot cards or tea leaves, people generate meaning out of nothing. And then there's Pareidolia: https://en.wikipedia.org/wiki/Pareidolia

If you're saying machine-generated text is meaning in the sense of that last category then sure, it's something we loosely call meaning. But it's fundamentally the same as other sorts of cleromancy [1], just more elaborate.

[1] https://en.wikipedia.org/wiki/Cleromancy

a process of internal generation of notions

How is this "knowledge extraction from data" process different?

Immanuel Kant would call the former "rendering a synthetic judgement" and David Hume termed the latter revealing the a priori knowledge. Plato would call both simply giving substance to the forms.

It seems none of this is something new. It's just that you no longer need a good education to learn about old ideas; you can come up with them on your own.

Different from what? Regular communication? Because the first involves trying to sync up two minds to have the same idea. The latter involves one mind, trying to generate an idea that ends up being useful. Useful in the George Box sense: "All models are wrong, some models are useful."

I meant how is knowledge extraction done by a language model different from knowledge extraction done by a human scientist?

Sorry, I don't understand the question.

GPT-3 does some form knowledge extraction, right?

You said above: "There's also the kind of meaning that scientists and researchers talk about when they extract knowledge from data."

So if we agree that in both cases some form of knowledge extraction is happening, I wonder how these two forms compare.

A search for "GPT-3 knowledge extraction" doesn't yield much, so I don't know what you're talking about. But as far as I'm concerned, knowledge exists in the heads of people, so I don't think we agree on that.

knowledge exists in the heads of people

How is this knowledge encoded in the heads of people? Perhaps through the choice of synapse strength and connectivity between neurons?

What is being encoded in GPT-3 weights as it reads through millions of pages of text? How is it different from tuning biological synapses?

Still don't know what I'm talking about? It's OK, neither do I :)

There's a ton of problems that a quality language model like GPT-3 can solve, beyond just spamming text generation - stuff like translation quality, automatic post-editing of text, classification, etc.

"Sure it's cool, but what problem is it solving?"

Isn't it good enough, or very nearly, to generate fake news? And if you can generate something that fools a large percentage of humans, even for a second or two, you can sell ads. It would be solving a problem for anyone who can profit from it. I thought the people who developed it stated that it was too dangerous to release widely? Dangerous = useful to bad people, no?

That was just PR fluff designed to play to the ideological biases of their Valley employee base. They released GPT-2 anyway some months later because other people were going to replicate it anyway, and guess what, the river of fake news we're flooded with daily is still not being generated by AI. It's being generated by journalists with an agenda, same as ever.

Don't get me wrong. You can absolutely generate news articles with these text generation models. But if you look at examples of the "fake news" generated by the monster GPT-3 model, it's not really any different to junk churned out by supposedly respectable news organisations i.e. grammatically correct but filled with logical contradictions, false statements that can be checked in 30 seconds with a search engine and so on. Most people already learned which news sources are trustworthy and which are pushing an agenda. If those outlets replace their journalists with GPT-3 it won't make any difference. The readers who don't trust them will still not trust them, and the readers who just want partisan cheerleading will continue to be satisfied.

The proportion of people who will spend 30 seconds with a search engine to check on something is practically infinitesimal. Even people who do it frequently don't do it most of the time. And it doesn't matter anyway, because it's too late - you only have to fool people for a second, or a fraction of a second.

Great, so focus on solving that problem generally instead of worrying about AI.

What I see is actually quite different. If the people who could detect fake news was practically infinitesimal then you'd see the vast majority of people having super high trust in the media. In fact most people don't trust the media, lots of polls showing that. Sure they may only fact check something occasionally (often by reading about a topic they happen to understand), but people aren't stupid. After they notice or hear about a few mistakes and observe they're always in the same ideological direction, they get the picture.

In ODS.ai Russian slack-chat a Replika developer has showed the increase in metrics that was provided by GPT-3 in comparison to GPT-2. Here you can see it on Telegram (also in Russian, but the pic is in English) without the access to ODS-slack. https://t.me/dlinnlp/931

>But it leads to things like Teslas driving into trucks parked sideways across the road because the engineers simply never predicted it would ever happen. And now it's happened multiple times!

This is a completely wrong reading of the situation. Tesla engineers clearly understood it is impossible to determine whether a stationary object is actually on the street or just an overhead sign with the equipment that is available on current Tesla cars and therefore they simply ignore all stationary objects. Because Autopilot is not a self driving technology this is not considered a problem. If a Tesla with enabled Autopilot ever runs into a stationary object it is clearly the driver's fault. It's extremely predictable and since it's easy to blame the driver there is no need to fix the problem by adding the required sensors to Tesla cars.

I don't understand why the goal is to "guarantee safety". It seems to be generally accepted that human error causes approximately 90% of motor vehicle accidents [0]. Surely then the goal of any autonomous or semi-autonomous transport system should be to merely reduce that percentage? If all motor vehicles were "self-driving" and the total number of annual deaths was reduced by one, wouldn't that be a good thing? I would rather reduce my chance of dying early than eliminate the possibility of being killed by a bot.

0. http://cyberlaw.stanford.edu/blog/2013/12/human-error-cause-...

Because humans are illogical creatures and will ignore objective facts if they believe control (or perceived control) is being taken away from them. The challenges of rolling out true autonomous vehicles are psychological as much as they are technical.