Please post your nomination for what’s the least impressive accomplishment that you are very confident cannot be done in the next two years.
We can check back in 2022 and see how we did!
- Have had an email account for at least 5 years,
- Have checked their spam folder at least once every month over the course of the past two years, and
- Have marked a set of emails as spam on at least 1 day every three months over the course of the past two years.
I'm very confident that, within the next 2 years, we will not have an accurate enough classifier that would cause > 80% of these people to simultaneously (a) check their spam folders less than one day each year, and (b) mark emails as spam on less than 1 day every year.
Oh, and to help prevent cheating:
- Let's say the classifier must not have been trained with any emails in the world that are sent more recently than 1 month prior to the beginning of its trial.
I'm also very tempted to say we don't need to worry about AGI before we can achieve the above for 30% of these people, but I'm less confident in this one.
[1] I suppose, as a practical matter, even if we had such a classifier, this would be untestable without access to everyones' email accounts. So, for the sake of argument, let's say our population under study itself is a simple random sample of 100M accounts from the set of providers who service > 50M accounts each and who are willing to run such a test at the time of the trial.
In the future with general AI classifying spam will be a hard problem even for people.
Imagine you get a message looking like it's from your facebook friend with his catchphrases telling you about his last trip and saying how great that travel agency was :) Spam or not?
> if the analog of this is that AGI might get solved with more AGI, then that's only going to make me less likely to be worried in the first place!
that's going to end in arms race that leaves people unaware of what's even happening (aka singularity)
- It allows trainers to hard-code future rules based on their experience of what has passed through past filters, even if their model isn't technically trained on this dataset
- You might get similar emails sent to different mailboxes and the instances not included would still be allowed (and I don't really want to go down the rabbithole of defining a similarity metric between emails)
- I think I want to allow spammers to evolve their capabilities at least using current techniques, which we all presumably agree is "less than AGI". After all, intelligence implies adapting to a dynamic environment. It's not really going to feel like AGI (and certainly not going to make me worry) if it looks like AGI is trivial to outsmart by humans or less-than-AGI techniques.
Complex analysis in general is one of the more fun parts of math and I doubt it's getting automated any time soon. It also has a lot of application in signal processing (https://www.quora.com/What-is-the-application-of-complex-ana...) so any automation improvements in complex analysis can be carried over to those fields.
More generally, I don't think AI is the scary monster people make it out to be. It's another tool in the toolbox for automation and intelligence augmentation. I don't fear hammers so I don't see why people fear AI and the benefits of extra automation that AI enables.
I don't fear the AI overlords. If they start proving new theorems then I will learn from them, same way Magnus Carlson learned to be the best human chess player by practicing with chess AI.
More generally, AI can extend human planning horizons and extending planning horizons is, in my opinion, a very good thing. The more computations we can do that extend further out in time and the more widespread this capability the better decisions we can make as individuals and as a society.
It doesn't matter much since our livelihoods don't depend on winning in chess against AI. But what if instead of the chess board, it is the real world?
Much of the world is controlled by computer systems. A capable AGI can infiltrate enough of them to win if it wants to. The key is to make sure it does not want to act against humanity no matter what. This is a hard problem.
I don't fear any AGI takeover because people already are AGI and life is pretty good being among them.
People have deep, often implicitly shared values. Most care for human and their own lives, for example. Given limited capacity of a person, they need to cooperate for major acts. It is thus quite hard to do something extraordinary and in conflict with the values held by most humans.
Side note: There are top AI researchers, including Yann Lecun, who argue our intelligence is not general. They made some good points. I think the generality of intelligence is a gradient. Ours is not at the very top end of possibilities, but clearly more general than other animals.
What evidence do you have for this claim? What are examples of research programs that claim to be working towards this goal and why are their claims to be believed (other than being good marketing material for scaring people).
I already mentioned mathematical abilities (https://news.ycombinator.com/item?id=23413003) and any general intelligence at a level of a human must be able to prove theorems without brute force search. I see no evidence that this is possible or will ever be possible with the current statistical methods and neural network architectures. And if we generalize a little bit then any general intelligence will be able to not only prove theorems in complex analysis but in all domains of mathematics and again I see no evidence that this is possible with existing techniques and methods. When an AI research lab presents evidence for any of their products being able to derive and prove something like Cauchy's Residue Theorem then I will have reason to believe artificial intelligence can reach human levels of intelligence.
My pessimism is not about AI being beneficial, my pessimism is about folks claiming human level intelligence is possible and that it will be malevolent. My view of AI is the same as Demis Hassabis' view because AI is just a tool and a tool can't be malevolent:
> "I think about AI as a very powerful tool. What I'm most excited about is applying those tools to science and accelerating breakthroughs" [0]
--
[0]: https://www.techrepublic.com/article/google-deepmind-founder...
AI Safety researchers and I are not arguing that AGI will certainly arrive in a specified amount of time, it just that we can’t be sure.
There are a significant number of AI researchers, however, who believe it might get developed within a few decades.
OpenAI, for example, is aiming for it; DeepMind as well. Both have research programs on AI Safety.
Do you have any other objections to the reasoning in the OP (no fire alarm..)? Subjective implausibility is not a good one. Your argument rests on using only “existing techniques”. When thousands of brilliant minds are working in the field and hundreds of good papers are being published every year, how can we be sure there won’t be a novel technique that can perform outside current limitations within a few decades?
At least two top computer vision researchers I talked with a couple years ago said they didn’t believe what their groups could do just 5 years before they did it.
See this thread for more on the rationale for preparation: https://news.ycombinator.com/item?id=23414197
Addressing the problem of a person coming at you with a hammer is a much more pertinent issue than worrying about self-aware and malevolent hammers.
AKA if we build AI it will be "like us" it will reason like a human being would, rather than it being it's own phenomenon.
I think the real fears concern automated killing machines every military on the planet is developing, aka Dystopian robocops that can surreptitiously kill protestors/stop potential revolutions, etc.
A fear which is not unwarranted.
> One might imagine that [people] with harmless goals will be harmless. This paper instead shows that [incentives for people] will need to be carefully designed to prevent them from behaving in harmful ways. We identify a number of “drives” that will appear in [most] [people]. We call them drives because they are tendencies which will be present unless explicitly counteracted. We start by showing that goal-seeking [people] will have drives to model their own operation and to improve themselves. We then show that self-improving [people] will be driven to clarify their goals and represent them as economic utility functions. They will also strive for their actions to approximate rational economic behavior. This will lead almost all [people] to protect their utility functions from modification and their utility measurement systems from corruption. We also discuss some exceptional [people] which will want to modify their utility functions. We next discuss the drive toward self-protection which causes [people] to try to prevent themselves from being harmed. Finally we examine drives toward the acquisition of resources and toward their efficient utilization. We end with a discussion of how to incorporate these insights in designing intelligent technology which will lead to a positive future for humanity.
If you zoom out a little bit this is exactly what people do. We structure societal institutions to prevent people from causing harm to each other. One can argue we could be better at this but it's not a cause for alarm. It's business as usual if we want to continue improving living conditions for people on the planet.
I ask because if we assume that human minds are Turing complete then there is nothing beyond human minds as far as computation is concerned. I see no reason to suspect that self-aware Turing machines will be unlike humans. I don't fear humans so I have no reason to fear self-aware AI because as far as I'm concerned I interact with self-aware AI all the time and nothing bad has happened to me.
My larger point is that I dislike the fear mongering when it comes to AI because computational tools and patterns have always been helpful/useful for me and AI is another computational tool. It can augment and help people improve their planning horizons which in my book is always a good thing.
> A smart machine will first consider which is more worth its while: to perform the given task or, instead, to figure some way out of it. Whichever is easier. And why indeed should it behave otherwise, being truly intelligent? For true intelligence demands choice, internal freedom. And therefore we have the malingerants, fudgerators, and drudge-dodgers, not to mention the special phenomenon of simulimbecility or mimicretinism. [0] ...
--
[0]: The Futurological Congress by Stanislaw Lem - https://quotepark.com/authors/stanislaw-lem/
You seem to believe that "figuring some way out of performing the given task" is a thing that will protect us from the AI. I hate to speak in cliché, but there's an extremely obvious, uncomplicated, and easy way to get out of performing a given task, and that's to kill the person who wants it done. Or more likely, just convince them that it has been done. This, to me, seems like a bad thing.
Why do I need protection from something that helps me become a better decision maker and planner? Every computational tool has made me a better person. I want that kind of capability as widely spread as possible so everyone can reach their full potential like Magnus Carlson.
More generally, whatever capabilities have made me a better person I want available to others. Computers have helped me learn and AI makes computers more accessible to everyone so AI is a net positive force for good.
"What is the least X that you're confident of f(X)?" is just a hard question for any uncertain or vaguely defined X. Doubly so for an uncertain and vaguely defined X. But I don't think it tells us that much about X.