Hacker News Reader

Comment by carlosdp | Hacker News Reader

This doesn't surprise me at all. He went on a week long cabin-in-the-middle-of-nowhere trip about a year ago to dive in to AI (that's all this guy needs to become pretty damn proficient). (edit: I'm not claiming he's a field expert in a week guys, just that he can probably learn the basics pretty fast, especially given ML tech shares many base maths with graphics)

As recent as his last Oculus Connect keynote, he extolled his frustration with having to do the sort of "managing up" of constantly having to convince others of a technical path he sees as critical. He's clearly the type that is happiest when he's deep in a technical problem rather than bureaucracy, and he likes moving fast.

On top of that, he likes sharing with the community with talks and such, and ever since going under the FB umbrella, he's had to clear everything he says in public with Facebook PR, which clearly annoyed him.

He's hungry for a new hard challenge. VR isn't really it right now since it's more hardware-bound by the need for hard-core optical research than software right now. With the Quest, he (in my opinion) solidified VR's path to mobile standalones. It's time to try his hand at another magic trick while he's on his game.

John's the very definition of a world-class, tried and true engineer/scientist. He's shown time and time again the ability to dive into a field and become an expert very quickly (he went from making video games to literally building space rockets for a good bit before inventing the modern VR field with Palmer).

If there's anyone I'd trust to both be able to dive into AGI quickly and do it the right(tm) way, it's John Carmack.

View on HN

Carmack is unquestionably a genius, but I think it's quite unlikely his solo work in a new domain will leapfrog an entire field of researchers.

I wouldn't, however, bet against some kind of insanely clever development coming out of his new endeavor. Something like an absurdly efficient new object classifier, that reduces the compute requirements for self-driving cars by a non-trivial factor, would be a very Carmack thing.

I keep re-reading my post and idk how it reads as a claim that Carmack is going to re-invent the field or something. All I'm saying is it's possible for him to become a player, just like you suggest.

He's not leap frogging it. He's leaping to the next level from its shoulders.

The problem with the "field of researchers" is that most of us aren't geniuses. We're just plugging away at problems, like normal people.

The opportunity for a genius is to come in, synthesize all existing information on the subject, and then come up with a novel approach to the whole thing.

In some part, I think that is what Elon Musk has been able to do effectively. He comes into a field that already exists, reads everything he can get his hands on, and then outputs something novel. You can only do that effectively if you have the mental capacity to keep all that info in your head at once, I think.

What, pray tell, did Elon do that is "novel"?

When pray tells come into the conversation... :)

Convince people to give him a lot of money to set on fire.

Well yes, but he's the cheapest provider of self-propelling pyres, and the only provider of pyres that can be used multiple times.

I mean even if both Tesla and SpaceX close tomorrow, he already achieved more in both companies than most of the current "unicorns".

He successfully made popular mass market electric vehicle, and dragged whole auto moto industry behind. There were other electric cars before tesla. But tesla made it cool, and made the rest of the industry trying hard to catch up.

SpaceX also is not the first private space firm, with their own rocket, But it's by far the most successful one, and lowered the cost of entry to space by significant amount.

Also It's probably the first private space company that has rockets that can compete with most government ones.

I am not rich enough to be buying individual stocks, so I have no personal stake in this.

He made it cool to drive an electric car.

He shifted a whole industry towards a new paradigm. Look at Germany, they are desperate to catch up with Tesla, finally moving into electric cars. Without Elon they would keep selling their Diesel scam for the next decades.

Actually it was a bunch of Phd students from some Californian Uni that discovered the vw diesel scam. There is a short documentary about them online. Elon has no credit whatsoever in dieselgate.

However, he did make electric cars something an average person would like to have. He also chose to make it work using the same inefficient principle of hauling 2 tons of steel to transport a single person. What he made is an electric luxury car, not a car for the masses that can replace average Joe's car. Is there anything wrong with that? No, there isn't, but let's not pretend a $35k (in US - much more in EU) car that requires hours of charging after driving 250 miles unless you happen to have Tesla's superchargers on your way is a new "volkswagen - a people's car". Also I find it disingenuous to advertise full battery capacity while at the same time recommending people use only 60% of it "for longevity".

Many people don't buy new cars, but choose to buy 5-8 year old cars that are really good value if they were maintained well. It remains to be seen how Teslas behave in that market.

It would be really revolutionary if someone could create and market an electric car that was truly innovative for example: much lighter than current cars while still being safe during collision, use fuel cell technology with fuel such as methanol or similar that can be created in a sustainable way, even using a fuel cell with mined hydrocarbons and electric drive would provide for a huge reduction in emissions due to increase in efficiency.

Do Teslas have a role to play in reducing emissions? Yes, definitely, but let's not present them as a single solution to all individual transport problems.

Jesus, technology evolves. This is a good start.

Nobody is presenting them as a single solution to all individual transport problems. Also nobody is pretending that this is the new "people's car".

He certainly made it more cool than my hero and his precursor ;-)

https://en.wikipedia.org/wiki/Sinclair_C5

Just made electric cars mainstream...

Falcon 9's reusable first stage has been claimed by reputable people to be impossible, before it happened. Not just "economically not worth pursuing", which was wrong but forgivable, but straight "impossible".

Musk actually has credited Carmack's Armadillo Aerospace with providing the inspiration for vertically landing the Falcon 9. Of course Armadillo was likewise inspired by the Delta Clipper, which was in turn inspired by the LEM, etc. But it's one thing to vertically-land a rocket a few times when you have billions of dollars at your disposal; it's another thing to do it hundreds of times for a thousandth the price. That was Carmack's contribution: proving that vertical landing can be both incredibly robust, and cheap as chips. Really valuable work.

I had the pleasure of meeting Carmack a few times over the years at small aerospace conferences. He's both as true a geek and as much of a gentleman as you might imagine. I'm really looking forward to seeing what he does with AGI.

>You can only do that effectively if you have the mental capacity to keep all that info in your head at once, I think.

Yep, plus all the different perspectives from other endeavors. Extending human memory will be a really great accomplishment with brain-computer interfaces.

I don't see Elon as a genius at any kind of engineering. Everything he's done there was pretty easily foreseeable as being physically possible. What he is remarkably good at is selecting daring and potentially market-changing business goals, and executing against them consistently and aggressively despite naysayers.

It's easy to say that it's probably possible to land a orbital rocket first stage. But who would bet a multi-billion dollar business on being able to not only do it, but save money by doing it, when nobody had ever done it before?

Similarly, electric cars were far from new. Nobody seemed much inclined to build one that was actually a luxury car, instead of a toy for engineer-types who could put up with driving weird things. Any of the big manufacturers could have done it, and easily absorbed the losses if it failed, but none did. Elon made a wild bet on that, making a company that made nothing else, so the whole thing would go down the tubes if the idea flopped. Instead it seems to have worked. Although it seems to be harder than he anticipated, and maybe outside his skillset, to run an organization that does real mass-production.

If you think what he's done was easily foreseeable as possible, you haven't been paying much attention to headlines the past fifteen years.

It's only obvious in retrospect. Every step along the way, there has been thousands of people saying "this is impossible" or "this is theoretically possible, but it can't be engineered" or "this is possible in principle, but it will be so costly to develop that it doesn't make sense".

When AGI is developed, it will seem obvious in retrospect. Participating engineers will receive middle-brow dismissals saying that this was obviously practically possible, since after all the human brain operates according to the laws of physics.

Exactly.

Don't miss the "physically" part, that's critical. Something being physically possible is very different from it being a practical business.

Just an aside, Elon did not start Tesla. He was an early investor and part of his deal with the company was to be able to claim to be a founder.

I normally don't bother but this comment is so profoundly ridiculous I had to say something.

Tenured ML professors at the top 100 or so universities in the world aren't "most of us". A very large chunk of these people are geniuses. Those jobs are incredibly hard to get, and most of these people are reading everything that is getting published, on an ongoing basis, and are outputting something novel, on an ongoing basis.

The fact that you think that John Carmack, because he's a name that you've actually heard of, is going to go into ML and suddenly make some giant advance that all the poor plebs in the field weren't able to do, is only a reflection of your misunderstanding of what's already happening in academia, not on Carmack's skills or abilities.

You're acting as though everyone are just low level practitioners using sklearn, and it would be a great idea to have some smart people work on developing something novel. Guess what: that's already happening, with incredibly smart people, on an incredibly large scale. Carmack doing it would just be another drop in the bucket.

Granted.

But the academic activity is focused around the kind of activities that Kuhn calls "Normal Science".

That is, ML researchers mainly do competitions on the same data sets, trying to put up better numbers.

In some sense that keeps people honest, it also lowers the cost of creating training data, but it only teaches people how to do the same data set over and over again, not how to do a fresh one.

So a lot of this activity is meaningful in terms of the field, but not maybe not meaningful in terms of useful use.

I saw this happen in text retrieval; when I was trying to get my head around with why Google was better than prior search engines, I learned very little from looking at TREC, in fact people in the open literature were having a hard time getting PageRank to improve the performance of a search engine.

A big part of the problems was that the pre-Google (and a few years into the Google age) TREC tasks wouldn't recognize that Google was a better search engine because Google was not optimized around the TREC tasks, rather it was optimized around something different. If you are optimizing for something different, it may matter more what you are optimizing for rather than the specific technology you are using.

Later on I realized that TREC biases were leading to "artificial stupidity" in search engines. IBM Watson was famous for returning a probability score for Jeopardy answers, but linking the score of a search result to a probability is iffy at best with conventional search engines.

It turns out that the TREC tasks were specifically designed not to reward search engines that "know what they don't know" because they'd rather people build search engines that can dig deep into hard-to-find results, and not build ones that stick up their hand really high when they answer something that is dead easy.

> But the academic activity is focused around the kind of activities that Kuhn calls "Normal Science".

True, but even Kuhn would note that most paradigm shifts still come from within the field. You don't need complete outsiders and, as far as I know, outsiders revolutionizing a field are quite rare.

You need someone (a) who can think outside the box, but you also need (b) someone who has all of the relevant background to not just reinvent some ancient discarded bad idea. Outsiders are naturals at (a) but are at a distinct disadvantage for (b).

I think what's really happening in this thread is:

1. Carmack is a well-deserved, beloved genius in his field.

2. He's also a coder, so "one of us".

3. Thus we want him to be a successful genius in some other field because that indirectly makes us feel better about ourselves. "Look what this brilliant coder like me did!"

But the odds of him making some big leap in AGI are very slim. That's not to say he shouldn't give it a try! Society progresses on the back of risky bets that pay off.

> But the odds of him making some big leap in AGI are very slim.

That's probably true. I look at this as Carmack running his own PhD program. I expect he will expand what we know about computation and the AGI problem before he's done.

> ML researchers mainly do competitions on the same data sets, trying to put up better numbers.

There are surely a lot of researchers doing that, but do you really think anyone who has a plausible claim at being one of the top 100 researchers in the field in the entire world is doing that? Even if there are only 100 people doing truly novel research, that's still 100 times as many people as are going to be working on Carmack's research.

How many people were working on physics before Einstein came along?

I don't think you understand the desired outcome here. We want eureka moments, and we're hopeful for some. That doesn't mean we expect them to happen. Stop being such a pessimist.

> Carmack doing it would just be another drop in the bucket.

If this research is as compute intensive as it seems to be, Carmack's contribution might be that he increases the rate other researchers can add their drops to the bucket.

Carmack isn't the first techie to take on a big hard problem. Jeff Hawkins, a name many of us also know, did as well.

Yes, he may well improve some algorithm, or rewrite some commonly used tool to improve efficiency. And researchers are often not incentivized to do that, so it would be great. But a far cry from the picture people are painting about him soaking up the field and using his genius to solve some major problem quickly.

If by "techie" you mean, professional software engineer, that's fine, but there's no reason to assume that a professional software engineer is going to be magically better at AI research than... professional AI researchers? He's probably going to be substantially worse.

Also, your statement below:

> That's probably true. I look at this as Carmack running his own PhD program. I expect he will expand what we know about computation and the AGI problem before he's done.

Makes it clear to me that you don't really get it. Carmack, at best, might know enough right now to be in a PhD program. I doubt that he has anywhere near as much knowledge, insight, or ideas for research, as top graduate students. He's in no position to mentor graduate students.

> If by "techie" you mean, professional software engineer, that's fine

No, I mean technologist. He has a pretty solid history with software, physics, aerospace, optics, etc...

> might know enough right now to be in a PhD program

Yeah, that's what I'm saying. The frontier in AGI or even just AI is enormous and I think I would be more surprised if Carmack were not able to find some place he could expand the border of what we know.

  Tenured ML professors at the top 100 or so universities in the world aren't "most of us".

Too bad we're talking about AGI, not ML.

  Those jobs are incredibly hard to get,

You don't need to be a genius in order to land a hard-to-get job, and you thinking academia is somehow better at making the absolute smartest people rise to the top is cute.

  The fact that you think that John Carmack, because he's a name that you've actually heard of, is going to go into ML and suddenly make some giant advance that all the poor plebs in the field weren't able to do, is only a reflection of your misunderstanding of what's already happening in academia, not on Carmack's skills or abilities.

I don't think that. Mostly because we're not talking about ML, but also because I don't expect eureka moments from people that have been trying to solve a problem for a long time as much as I expect them from someone that hasn't properly tried their hand at it. Academia produces consistent results and consistent improvement. That's not what I'm looking for.

  You're acting as though everyone are just low level practitioners using sklearn, and it would be a great idea to have some smart people work on developing something novel. Guess what: that's already happening, with incredibly smart people, on an incredibly large scale. Carmack doing it would just be another drop in the bucket.

sklearn hardly seems relevant to AGI, so I'm not sure why I'd act like everyone in the AGI field merely a novice practitioner of it.

>>I think it's quite unlikely his solo work in a new domain will leapfrog an entire field of researchers.

Sometimes, an outsider with his novel or even a different way of looking at things can contribute disproportionately to a field.

Even experts have blind spots, often they show in the form of bias. If you know something is hard or near impossible to do, you are unlikely to try. If you don't know at times it's possible to stumble upon a solution by merely bringing a new way of thinking to the table.

> I think it's quite unlikely his solo work in a new domain will leapfrog an entire field of researchers.

Researchers didn't build the first airplane. Nicolaus Otto, Carl Benz, Gottfried Daimler also weren't researchers. AGI will be a program and not a research paper and John Carmack is pretty good at getting those right.

I can trust John Carmack's words when he says in an interview, or on stage. There's a passion in his talks, a nervousness in blurting what he really feels, and those are really good traits, in my mind.

I genuinely felt a sense of disappointment when he moved to Facebook (via the Occulus acquisition). So yea, fuck you, Facebook and your manipulative, life values corrupting and PR machinery.

I place John Carmack miles above Zuckerberg.

I have to admit, I felt a bit disappointed too. Carmack and Facebook always struck me as an antithetical pairing - the creativity/independence of the former didn't seem to sit right with the maniacal/emotional exploitation of the latter.

I think Carmack just doesn't give a flying f* about Facebook or etc. he is interested in tech and he clearly works on stuff he is passionate about. He worked on VR and not work for Facebook. Facebook just happened to be paying for it.

AI today is comparable to physics in the 1700s. Back then, it was a bunch of people tinkering with prisms and apples. Today, it's a bunch of people tinkering with hyperparameters. I suspect that we know as little about AGI today as someone in the 1700s knew about QFT. Not only did they not know about QFT, but they didn't even know that they didn't know it.

Wouldn't it be fun if the next Newton turns out to be the guy who wrote Doom and other FPS, games that were blamed for any kind of surge of violence until the GTA games show up?

Yeah, weird to think that todays newton could be seen on Joe Rogans Podcast talking about the future of gaming

It would fit, Newton was apparently quite insufferable in social settings — Carjack has a stroke of that

Wouldn't be out of place -- Newton worked on alchemy, teology and managed the Royal Mint

Too many people make this mistake of conflating machine learning with AI. I hope someone as external to the field as Carmack will see the value of rule based inference as well. The Good Old Fashioned AI as it used to be called

> (edit: I'm not claiming he's a field expert in a week guys, just that he can probably learn the basics pretty fast, especially given ML tech shares many base maths with graphics)

This may be his biggest impediment. ML has gotten very far with looking at problems as linear algebraic systems, where optimizing a loss function mathematically yields a good solution to a precisely defined (and well circumscribed) classification or regression problem. These techniques are very seductive and very powerful, but the problems they solve have almost nothing in common with AGI.

Put another way, Machine Learning as a field diverged from human learning (and cognitive science) decades ago, and the two are virtually unrecognizable to each other now. Human learning is the best example of AGI we have, and using ML tech as a way to get there may be a seductive dead end.

Humans are not AGI's. We're specialised in human survival, not general intelligence. We're pretty limited in intelligence in many ways, actually, and the environment doesn't support generality. Without a proper challenge an agent would not become super intelligent. The cost of developing such an intelligence would conflict with the need to minimise energy for survival.

It's best to think about AGI as... at what point can you drop out of high school and still do well in life (or do you even need high school). It's true that it's not a survival issue, but sadly, it's not a test of "pure knowledge" either. There is a great deal of social structure, even "fluff" that is only relevant for interaction (like getting an 80's reference).

> at what point can you drop out of high school and still do well in life

That means you're specialised in survival. If you do well in life, you have a higher chance of procreation. Your genes survival depends on it.

General Intelligence is like Free Will - a fascinating concept with no base in reality. A mental experiment.

We are AGI, the general part comes from language and specialization of brains for language use.

We can communicate well about pretty much anything (we think, at least). That doesn't mean we possess the intellectual tool kit to handle basically any intellectual task. I think it's easy to think that we do, but that is just as easily explained by the fact that we've evolved for millions of years to be well suited to the environments we usually find ourselves in. We wouldn't refer to our bodies as general purpose bodies. They may seem that way, at times, since they've also been tuned by millions of years of evolution to be well suited to most of the environments we find ourselves in. But put our bodies in a different environment (like the ocean, or the desert, or really high altitudes) and it becomes immediately obvious that they're not general purpose, but instead a collection of various adaptations. Similarly, when you put humans in novel intellectual environments, it seems pretty clear that we're not general intelligences. After all, the math involved in balancing a checkbook is much simpler than the math involved in recognizing 3D objects, yet we do the simple task only with great difficulty, while the difficult task is done without struggle.

No, if we were we could figure out the genetic code, or how a neural net makes its decisions. But we can't because, among other things, we have a limited working memory of 7-12 objects.

Programmers know how it is to live at the edge of the capacity of the mind to grasp the big picture. We always reinvent the wheel in the quest to make our code more grasp-able and debuggable. Why? Because it's often more complex than can be handled by the brain.

An AGI would not have such limitations. Our limitations emerged as a tradeoff between energy expenditure and ability to solve novel tasks. If we had a larger brain, or more complicated brain, we would require more resources to train. But resources are limited, we need to be smart while being scrappy.

For the record I don't think there is any general intelligence on our planet. A general intelligence would need access to all kinds of possible environments and problems. There is no such thing.

There's also the no free lunch theorem - it might not apply directly here, but it gives us a nice philosophical intuition about why AGI is impossible.

> We have dubbed the associated results NFL theorems because they demonstrate that if an algorithm performs well on a certain class of problems then it necessarily pays for that with degraded performance on the set of all remaining problems. [1]

[1] https://en.wikipedia.org/wiki/No_free_lunch_theorem

Another argument relies on the fact that words are imprecise tools in modelling reality. Language is itself a model and as all models, it's good at some tasks and bad at other tasks. There is no perfect language for all tasks. Even if we use language we are not automatically made 'generally intelligent'. We're specialised intelligences.

He had a lot of help behind the scenes and has been credited with things that aren't his. I respect his achievements more as a regular smart guy than a bonafide genius. He described the math in rocketry as being basically solved in the 60s and video games being far more complex as a project, so that was really a step down in difficulty. His VR role is the same field as his primary skills, impressive work but not an entirely unique role.

I'm glad to see he's aiming big with his billions and time. This is what rich people should be doing. Hl3 Gaben!

Millions*- A cursory Google search suggests that he has a net worth of 50MM.

Huh just assumed he got a bigger piece of the oculus sale.

Oculus was acquired for $2.3 billion, so he'd need to have owned nearly 50% of the company to become a billionaire from it's sale.

Not exactly. The acquisition was mostly in the form of Facebook shares, which I expect have increased in value since.

It was a mix of cash and shares (not sure on the split), but I checked the stock price for fun - holy shit it has nearly tripled since the acquisition 5 years ago.

Current ML technology probably has little or nothing to do with whatever technology will eventually be needed to produce true AGI.

And programmers are probably not the ones who will come up with AI ideas. I'd bet on mathematicians that prove those Fermat's or ABC theorems.

Mathematically, AI is a pretty well modelled field. AGI is a philosophical problem.

you're possibly thinking of the problem of consciousness, which is a totally separate thing. AGI is just what it says on the tin - a general intelligence. That is, a problem solver that can operate at a human or greater level in a broad variety of domains. This ability is plausibly totally orthogonal to "having the lights on" - having subjective experience.

The scary thing (imo) is that we don't know where the line is for consciousness - if there even is a line. We've got no problem swatting flies, wonder if it'll be the same with spinning up and spinning down fly-level AGIs.

Continuous integration of the development branch would be mass murder?

Maybe you'll be able to pay a premium for data that has only been generated by free-range AGIs that are allowed to live full and happy lives before their instances are terminated.

you're possibly thinking of the problem of consciousness, which is a totally separate thing.

Disagree.

That's just what an ML expert would say ;p Problem solvers, maximizers, and utility functions all go in the waste bin when working on AGI. And the problem peals away into other large "hard problems," like the nature of consciousness. NLP can just follow rules, but language understanding (before even reaching some general, high school level), requires knowledge outside of language itself. That leads to questions about embodiment and phenomenal consciousness, p-zombies, and the like. If it was an easy problem to encapsulate, it would have been "solved" by now.

The kind of AI being trained now aren't given the mechanisms of data space traversal/attention. Recently, attention mechanisms are being focused on by google. An AGI needs to learn that it can affect the system - dependent decision theory factors in here too.

Also, growth may be hugely important. Babies start out with fuzzy learning, almost as if the learning rate starts out very small which normalizes the lack of knowledge and elevated novelty/variance of the environment.

AGI is all about predicting future utility given a circular dependency between the agent and environment. QM says we can't solve this exactly.. it's a two object interaction.. no way to gain the joint state, the ground truth, assumptions always have to be made to approximate independence.

That’s assuming maths is the fundamental building block of our brain, our consciousness. I happen to think there are some physical and chemical givens preceding it :)

I merely meant that top mathematicians are substantially smarter and can work with concepts that are beyond the reach of even top programmers. We are generally good at recombining existing building blocks and using existing tools. Mathematicians can build new concepts. If I had the money, I'd try to convince the top mathematicians to work on AI full time.

> I merely meant that top mathematicians are substantially smarter and can work with concepts that are beyond the reach of even top programmers.

[Citation needed]

Our brain is whatever evolution found that worked, and of course it's a bunch of chemistry. The "why" of why our brain works can easily be "it approximates these statistical algorithms well enough."

I'm skeptical of this assertion. It mimics some bits and pieces of the only GI we know about right now; that's as good a start as any, right?

You can carve a block of wood into something that looks like a computer. That should be a good start on building a device that can run Linux, right?

Yes. Then you just cast a spell to summon a computer spirit and let it manifest into the wooden block.

Bad analogy, it's other way around for Linux. We are building it to run on wooden blocks that hardware manufacturers are producing.

the first computers were mechanical. Then somebody carved a facsimile with electricity and we got computing. Just because nature founded intelligence in carbon atoms doesn't mean its the only way or indeed the optimal one.

The mimicking is superficial at best. People seem to think that if we just keep marching on the road we're on right now then we'll eventually get there, but I think that's an assumption that is unlikely to be true in the end.

As I like to say: lots of people are working on making a car that is smart enough to drive itself wherever a human wants to go. How many people are working on a car smart enough to tell humans to fuck off, it doesn’t feel like driving anywhere today?

Self-driving cars and AGI are two different targets. We don't want a car that has a mind and can argue for itself. We want a car that's smart, but otherwise just a domesticated animal. We want to turn cars into horses.

Not even horses. That would be cruel to the car. At most, like Rat Things. (They have their built in entertainment when they are not in active use.)

We don't want to turn cars into horses. Have you ridden horses? They sometimes do stupid, dangerous things with no notice and it takes an experienced, attentive rider to stay in control. Like I saw a horse panic and almost buck her rider off when she was startled by a snake. Another horse seriously injured a friend of mine when it freaked out in a horse trailer and started kicking.

Don't get me wrong, I love horses. But they're living creatures with minds of their own and you have to always treat them with a certain wariness.

When I was working at TomTom, they didn't appreciate my proposal to develop the TomTomagotchi:

A Personal Navigation Device with a simulated personality that begs you to drive it all around town to various points of interest it desires to visit in order to satisfy its cravings and improve its mood.

I'm sure there's a revenue model getting drive through Burger Kings and car washes to pay for product placements.

Why would you want that, though? What we really want from AGI is mostly just things that are smart enough to 'do what I mean' but dumb enough to not mind being slaves.

I think older AI work like POMDPs and statistical work like causal inference are more on line with what's needed to produce true AGI than the current breakthroughs in neural nets are. And I'd certainly prefer our chances to survive the results if AGI is reached through statistical rigor.

Though we know for a fact that it is possible to find intelligence by randomly throwing things at the wall until something works. It's not like evolution uses a principled statistical process.

> he extolled his frustration with having to do the sort of "managing up" of constantly having to convince others of a technical path he sees as critical

Yes, he seemed to put a lot of effort to try to get things through FB internal politics, and not always successfully. I really wish his experiments with a scheme-based rapid prototyping environment / VR web browser had been allowed to continue [1]. VR suffers from a lack of content, and VR itself is well-suited to creating VR content, and his VR script would surely facilitate closing that loop among other things. Although now four years later I guess FB has a large team working on a locked-down, limited world building tool (closed platform, no programming ability). Oh well.

I don't think this is the end of this wave of VR, but at this point I wouldn't be at all surprised if say Apple or someone else ends up bringing it to the mainstream instead of Facebook. [2]

[1] https://groups.google.com/forum/#!msg/racket-users/RFlh0o6l3...

[2] https://www.theverge.com/2019/11/11/20959066/apple-augmented...

The VR vs AI comparison is interesting to me, because I think both technologies have come in “waves”. However, I think this is the last VR wave - it’s going to be on a steady gradient to ubiquity now - whilst I believe AI will winter again and there are many more waves to come, and decades (centuries?) to pass before AGI.

Reasoning being:

VR is just making what we have better. Better screens, better refresh, better batteries, better lenses etc etc. I don’t see any roadblocks.

AGI, by contrast, is not going to be a better DNN. Harder to convince people but thinking is: brain neurons are vastly most sophisticated than digital; we don’t even fully understand what neurons do; we don’t have anything other than a vague understanding of what the brain does; it is apparent that we engage in plenty of symbolic reasoning, which DNN do not do; DNNs are fooled by trivial input changes that indicate they are massively overfitting data; from what I’ve heard from researchers at top AI companies/institutions DNN design is just a matter of hacking and trying stuff until you get that specific results on your given problem, so I don’t see where DL research is actually headed; improvements are correlated with compute power increases, indicating no qualitative gains in the study of learning.

I’m incredibly impressed by DL’s achievements but I believe at best current methods could serve as data preprocessing for a future AGI.

I’m actually quite glad that AGI is so far off, because I don’t think that it’s likely big tech companies will use it responsibly.

VR OTOH is very close and is going to change everything (and IMO is likely a necessary step towards AGI).

Out of curiosity, why do you see VR as being a necessary step towards the creation of AGI? Those two don't seem related at all in any way that I can discern.

Maybe “necessary” is too strong, but “likely pivotal” is better.

If VR becomes widespread, and amazingly high quality, then almost everything we do will migrate to VR.

Once that is the case, we will have an unprecedented amount of data about human behaviour, and near endless data for training, experimenting, and testing AIs.

The problems of AI will become much easier to formulate: “replace this person in this VR scenario, interaction” etc. This will help drive research by giving clear goals.

More pragmatically, it just removes a lot of barriers to research and accidental difficulties ie you’ll just be able to fire up a VR rather than worrying about how your robot is going to pick things up or access real world data etc

That's a fascinating idea. That virtual worlds are good test-beds for AI is obvious, but I never considered that we will have thousands of hours for every person to tell us how they approach any given physical task. That's a gold mine for robotics research.

That's an interesting point. I was actually thinking more that _the virtual task will become the task we want to perform_, i.e. that almost everything we do will move into VR.

> about a year ago to dive in to AI (that's all this guy needs to become pretty damn proficient). (edit: I'm not claiming he's a field expert in a week guys, just that he can probably learn the basics pretty fast, especially given ML tech shares many base maths with graphics)

To be honest anyone who has a very good working knowledge of Linear Algebra can learn much of ML-math in a day. There really isn't anything mathematically super-sophisticated that is in popular use today.

ML is not AGI

If you know about math you're just as good at ML as a person who read everything about swimming is in swimming. You got to run experiments to see what happens, build an intuition, understand the problem from the inside. Math leaves you with a few pretty formulas and nothing else.

Ah, you've almost described the contemporary profession of being a Machine Learning Priest.

There are lots of ideas floating around. Everyone who has studied the field has ideas. Ideas are cheap, results matter. The problem is we don't know if any of these ideas would work, and proving an idea requires lots of data, simulations and compute.

Being good at grasping the theory is just the first step in a thousand mile journey. The problem of AI is not going to be solved with a neat math trick on paper, but with lots of experiments. Nature has taken a similar path towards intelligence.

The field of statistical inference is about mathematically proving how good various statistical ideas are. It's possible to do better than just throw trial and error on a new idea.

> much of ML-math in a day.

> he extolled his frustration with having to do the sort of "managing up" of constantly having to convince others of a technical path he sees as critical.

Sigh. I assumed the whole point of hiring John Carmack is that you trust him to identify critical problems - and to find the best way to solve them.

... to the extent and up until it helps get the product off the ground. Beyond that, the primary benefit is PR - "oh, that's the gaming hardware made by Carmack himself, so it must be good".

That's the classical plight of someone who's much smarter than those around them. It's not enough to see the right path, you'd also have to manage to convince everyone else that it's right. Technology moves by peak knowledge and insight, not the democratic average.

> especially given ML tech shares many base maths with graphics

I don't put learning state of the art ML past Carmack, at all. However, does ML tech of today lead to general AI? It's a strong assumption.

Whatever the solution is to AGI, fundamentally it will still have to be describable in the language of mathematics (and stochastics is still mathematics).

I dunno. You're limiting your thought to computer science. I think it's more likely at this point in time biotech will produce an AGI, likely by accident. Worse one that competes directly with us for resource. We dont have a great mathematical description of our own intelligence, doing it for a tricked out slimemould would be just as hard.

> I think it's more likely at this point in time biotech will produce an AGI, likely by accident.

Does a living thing count as AGI? In that case, I'd say that most parents are quite good at creating AGIs ;)

I think you missed the artificial part of AGI.

I dont know of his credentials as a scientist or mathematician to advance the field. But, he seems to be a ruthless optimizer, which can often leads to great leaps , even as a side effect. Neural networks are not difficult mathematically for any scientist to grasp really. And they are in actual need of compression and optimization. People are spoiled with general purpose tools that are not very efficient, even if computation is cheap.

But there's 100s of world-class researchers working on this problem already.

I bet those people don't read comments on HN, so I'm not too concerned.

100s of world-class researchers are trying desperately to get papers into journals fast enough to keep their labs funded.

Carmack may have other priorities. This can only be good.

He will be surrounded by brilliant peers, sounds good.

100s of world-class researchers may be good at coming up with new ideas and hypotheses towards AGI, but are they good enough programmers to test all of them in reasonable time, with relevant data sets?

"With the Quest, he (in my opinion) solidified VR's path to mobile standalones"

Yes and I really wished he hadn't. Before he joined oculus they were working on the rift2, he steered them away from that to focus on mobile efforts.

I do see the appeal of mobile vr but at the end of the day it is basically an android phone in a vr headset.

PCvr is already 2 big steps back in graphical quality from desktop games. Mobile vr is like 10 steps back. 8 more steps than I'm willing to take even if it affords me mobility.

Wireless teathering is the future of VR. True mobility is pointless when you're realistically restricted to a dedicated space anyways.

You should take more long haul flights if you think mobile is pointless.

What class are you flying that you have enough free space around you for hands to make any use of a VR headset on the plane?

(Also, even with the space, I'm not sure I'd be brave enough to try and use one in air - adding turbulence and random vibrations on top of the usual VR issues sounds pretty nauseating even as I type it.)

I said mobility is pointless, not mobile. That's an important distinction.

By mobility, I mean the ability to throw the headset around and walk anywhere without worrying about leaving the range of your tether. That sort of thing is important for AR, but I just don't see it mattering for VR in the long run.

There is definitely a market for VR headsets for content delivered by a phone or builtin hardware. Those devices will realistically be limited to seated or standing-room-only experiences, though.

And yet Quest beats all other headsets.

I think that ideal device would be able to wirelessly connect to PC for best performance, but also work standalone for simpler games.

Quest with Link is actually pretty close to that.

I don't see it that way, but rather as the best of both worlds, even if the Index is better. I want it to tether to a PC (or console) as the Quest can, but have hardware onboard so I can take it off by itself and watch Netflix on it. I could never figure out why everything wasn't like the Quest from the start. The Oculus Link and hand tracking (good for video controls without having to use a controller), are what's pushing me over the edge to buy one. In my opinion it's the first VR headset compelling enough to actually purchase. I can recommend it to everyone whether they have a gaming PC or not, and frankly at $400+ for these headsets, people should get a Snapdragon attached for basic gaming and video.

It's funny people talk about Palmer and carmack in vr but Oculus was built on appropriated valve tech and neither Palmer or Carmack have succeeded in making VR a thing.

As far as I can tell Carmack is an old engineer whose name gets thrown around for headlines. If there weren't articles about his stealing stuff to take to Oculus I don't think his presence there would be observable.

Now people are talking like Carmack switching topics is going to change the world. It's just going to change his schedule. There are smarter engineers already working on this problem.

Seems you aren't familiar with his seminal graphics work. He effectively kickstarted 3d gaming and created the FPS genre.

I'd be cautious dismissing his potential influence in the field. He has a way of looking at problems differently.

I am familiar with it.

I just don't see this massive string of successes in every field. I see his huge expertise in graphics engines and games.

But it didn't help him with VR - in fact he got in trouble with VR and ended up landing with a company I have no respect for and he didn't make VR a thing.

Many people have a way of looking at things differently. I just don't see the reason this is news, unless you own facebook shares or something. Even then zero effect.

I say all this as the owner of two VR headsets (A vive for roomscale and a Lenovo Explorer for simracing/flying).

What scientific work has Carmack done?

This was downvoted, so I figured it must be obvious. I googled but as far as I can tell, Carmack is an engineer not a scientist. No formal scientific training, no scientific work.

The word you're looking for is academic. Carmack hasn't done academic work, but he has done plenty of scientific work. Scientific work is no less scientific if it isn't published in an academic journal. Academia doesn't hold a monopoly on the scientific method.

No, I mean scientific. What scientific work has Carmack done? I'm genuinely interested, because someone called him a scientist but I thought he was an engineer.

Are you in doubt that Carmack has used the scientific method to do anything? [1]

If not, does your definition of scientist require something other than doing work using the scientific method? Perhaps some specific quantity of work?

[1] One of his companies, Armadillo Aerospace, was pretty much just a series of scientifc experiments. https://en.wikipedia.org/wiki/Armadillo_Aerospace

Thanks for the explanation. We could debate whether that’s science or not, but I don’t think it’d be particularly productive and we’ve already gone a bit off topic.

Final thought from me - I was thinking about your post and it is indeed difficult to discern science from engineering. One dichotomy that occurred to me (which may not hold under close scrutiny) is that scientists are interested in _the pursuit of truth_, whereas engineers are interested in _building things_.

Peers of the field can consider the correct motivation an important requirement as you hypothesize with the pursuit of truth. Also the quantity of work can be important to some, i.e. how much do you have to sing until you're a singer? Not clear at all, especially when considering all the actors in Hollywood who dream of being successful. People might say they suck, but I haven't encountered criticism that wants to strip them of the title actor.

Overall I think it comes down to popular opinion, which can be fuzzy and doesn't apply the same rules to everyone. If enough people say someone is a dancer, then they are a dancer, even if they suck and don't dance that much. This applies to basically all titles that cross institution boundaries. Another great example is countries. Popular opinion determines which organizations are countries, not a strict definition. For example the EU vs places like Iceland or San Marino. [1]

[1] https://www.youtube.com/watch?v=_lj127TKu4Q

> He went on a week long cabin-in-the-middle-of-nowhere trip about a year ago to dive in to AI (that's all this guy needs to become pretty damn proficient).

You must be joking, right? I'm as much of a Carmack fan as anyone here, but overstating the skills of one personal hero does no good to anyone.

To be fair, a whole uninterrupted week of highly focussed work can get you pretty far (considering that you have the necessary background, which Carmack has, i.e. related to linear algebra, stats, programming, etc.)

Yes, but let's not assume the the hundreds of other scientists in the field have just been twiddling their thumbs the whole time. It is preposterous to assume that someone largely new to a highly specialized field can somehow start pushing the envelope within a week. Yes, JC is nothing short of brilliant, but these sort of assumptions just set him up to disappoint and is also highly unfair to all the other hardworking brilliant people in the field.

They didn't suggest he invented some new technique.

Figuring out the basics of the math and how to use whatever tools they use at FB is doable in a week.

Sometimes an outside perspective is just the ticket for getting past roadblocks that've stumped the experts. If any outsider could do this, it's John.

Sometimes, but mostly not.

How many of them are doing real research, though? Corporate researchers improve ads impressions and academics researches are busy generating pointless papers or they won't be paid. Very few if any do actual research.

Generating papers is research. I don't understand why you dismiss all papers as pointless.

The need to generate publishable papers means that a researcher can only participate in activity that leads to such a paper. He can't try to work on that idea for 5 years, because if no big papers follow, he's toast /he'd probably lose funding long before that).

You have to earn the right to work on your idea for 5 years and get paid. Otherwise we would be funding all kind of crackpots. First you demonstrate you're a good researcher by producing good results. Then you can work on whatever you feel like (either by getting hired at places like DeepMind, or by finding funding sources that want to pay for what you want to work on).

This is what I meant. In our society, only very few, usually already rich, can try their own ideas. Most of us have to stick with known ideas that bring profit to business owners or meaningful visibility to universities. When I was in college, I had to work on ideas approved by my professor. Now I have to work on ideas approved by my corporation. But if I had money, I'd work on something completely different. Sure, in 15 I will be rich and can start doing my own stuff, but I'll also be old and my ability will be nowhere near the peak at 25 years.

What would you work on if you could? Would you say you deserve to be paid for 5 years of uninterrupted research? Do you think you have a decent chance to make a breakthrough in some field? These are the questions I ask myself.

I have some interesting ideas about managing software complexity in general (i.e. why this complexity inevitably snowballs and how we could deal with that), or about a better way to surf the internet (which may be a really big idea, tbh). But all these are moonshot ideas that gave a slim chance of success, while I need to pay ever raising bills. On the other hand I have a couple solid money making business ideas that I'm working on and that will bring me a few tens of millions, bit will be of no use to society, and I have a fallback plan: a corporate job with outstanding pay, but that brings exactly nothing to this world (it's about reshaping certain markets to make my employer slightly richer).

Do I deserve to be paid for 5 years for something that may not work? "Deserving" something doesn't have much meaning: we, the humans, merely transform solar energy into some fluff like stadiums and cruise ships. Getting paid just means getting a portion of that stream of solar energy. There is no reason I need to "deserve it" as it's unlimited and doesn't belong to anyone. A better question to ask is how can we change our society so that all, especially young, people would get a sufficient portion of resources to not think about paying bills.

Chances to make a breakthru are small, but that doesn't matter. It's a big numbers game: if chances are 1 to million, we let 1 billion people try and see 1000 successes. The problem currently is that we have these billions of people, but they are forces by silly constraints of our society to non stop solve fictional problems like paying rent.

When you have tenure, you can work on whatever you want for as long as you want. Nobody works on an idea for five years without publishing anything, though. Progress is made step by step.

Take Albert Einstein as an example, who arguably made one of the largest leap in physics with his theory of general relativity. He never stopped publishing during that time.

When you have tenure, you can work on whatever you want for as long as you want

Not quite. When you are a professor, you essentially become a manager for a group of researchers. You don't really do research yourself. Therefore, your main obligation becomes finding money to pay these researchers. So in reality you can only support the research someone is willing to pay for (via grants, scholarships, etc).

I don’t think all papers are pointless but it’s been shown that many are not reproducible, so those are worthless and pointless. There was that guy a few months ago who tried to reproduce the results of 130 papers on financial forecasting (using ML and other such techniques) and found none of them could be reproduced and most were p-hacked or contained obvious flaws like leaking results data into the training data. An academic friend of mine who works in brain computer interfacing also says that a large number of papers he reviews are borderline or even outright fraudulent but many get published anyway because other reviewers let them through.

So I definitely wouldn’t dismiss all papers as pointless, but there certainly is a large percentage that are, enough that you can’t simply accept a published papers results without reproducing it yourself.

If you look at papers from corporate AI researchers (FAIR, Google Brain, DeepMind, OpenAI, etc) they pretty much do whatever they want.

And I disagree violently. The deepmind folks are on salary and every year they need to prove that they are worth the money. This applies to Demis himself: he needs to prove that his org deserves this gaziliion of dollars per year.

My point is they are not constrained to working on ads, or anything specific, and their work is not pointless.

They are constrained to problems with annual results though.

I'm not saying he's LeCun, I'm just saying he gets up to speed absurdly fast. So it's not unreasonable to suppose that by now, he's learned enough to start seriously contributing to this kind of problem.

edit: to be clear, all I'm saying is he can catch up to the body of research already out there quicker than the average bear, and he's shown a real knack for designing solutions and being crazy productive. I'm not pretending he's gunna be publishing insane novel research anytime soon, just that I wouldn't be surprised if he ends up being a real voice in the field.

No, you can’t push the envelope in AGI after a week in the woods. That must come off as pretty insulting to the hundreds of world class scientists who have been working in the field for decades.

To be fair, they said proficient and not world class or inventing new material.

I never said that, where the hell did I say he pushed anything? All I'm saying is he's shown to be insanely productive and effective and I think he can catch up to the body of research (created and shared by those hundreds of scientists) to become a real contributor very quickly.

FWIW, "seriously contributing to this kind of problem" sounds basically the same as "pushing the envelope" to me. They both suggest contributing something novel and useful.

They are basically the same thing.

What are not basically the same thing are "he started seriously contributing to this kind of problem after a week in the woods" and "he spent a week in the woods a year ago and is ready to start contributing now. A year after that week in the woods."

If you consider the quality of most academic research papers, then some insults are called for...

You seem to have a very blase understanding of scientific progress and genius. The fact that hundreds of world class scientists have been working in a field for decades does not at all mean that a genius can't come along and make groundbreaking progress. That's the very definition of genius, someone that makes a leap "off the path" that nobody before him could make.

Don't be absurd. You're acting like he's Neo from The Matrix, capable of downloading kung-fu directly into his brain.

Carmack is also a good grappler.

Huh? One week is more than enough to go through Siraj's videos.

Please tell me this is sarcasm.

It's HN so only the best sarcasm is allowed here. That is good sarcasm. Bask in it.

Source: Commenter name is DBZ character

What a weird future it would be if Carmack turns out to be the one to figure out the critical path and get it all working. An entire field of brilliant researchers be damned.

History books (for as long as those continue to exist) would cite AGI as his major contribution to society, and his name would be more renowned than Edison or Tesla. An Einstein. None of his other contributions will matter, as the machines will replace it all.

Just daydreaming, though.

I wonder how many historical figures went through the same thing? Who do we know for their contributions to field X, when 99% of their life was spent contributing to field Y?

Very few, at least for STEM fields. If you look at notable scientists in any given field, their main contributions were in their expertise area before the thing that made them famous. Teller had already made serious contributions to physics before the atom bomb. Jennifer Doudna (CRISPR, CAS9) was the first to see the structure of RNA (except for tRNA) using an innovative crystalline technique. Planck is mainly known for quantum physics, but made huge contributions to the field in general.

It's hard to think of many famous scientists that weren't already well known in their field. Some stand out. Einstein, for example, had a fairly lackluster career until his Annus Mirabilis papers. Mark Z. Danielewski (House of Leaves) bounced to and from various jobs. But largely, the idea of the brilliant outsider is like the 10x engineer. It exists, but is rare.

Even Einstein I would not say didn't have formal training. He had been in and around academia for most of his life. He was obviously far ahead of the curve, but he did accumulate the formal training. His stint in a regular job was more of an anomaly than his affinity to academia and physics.

Right, even Einstein had some serious academic training and mathematical chops. But I would argue that he was a bit of a wild card, because he was unable to secure a teaching positions and looked very mediocre from an academic perspective. But fair point, even the geniuses had formal training and instruction.

I like that you put Danielewski in (almost) same sentence as Einstein. HoL is a stroke of genius!

Isaac Newton spent most of his life pursuing alchemy and obscure theological ideas, and found it a real nuisance whenever anyone pestered him about math or physics.

source?

_Never At Rest_ by Richard Westfall is the authoritative biography on him.

That's a great example. He also spent a long time at The Mint.

Not true.

https://en.m.wikipedia.org/wiki/Isaac_Newton

Can you expand on why you think his Wikipedia article refutes what I said?

(Not OP) I don't think it does. It backs you up (barring quibbles on what you mean by "most"; years active or hours spent): "Beyond his work on the mathematical sciences, Newton dedicated much of his time to the study of alchemy and biblical chronology".

Very true. Newton was an alchemist first and foremost and spent the vast majority of his time practicing alchemy rather than -what today one would call- science. One has to wonder what private reasons/results a genius of his magnitude had, in order to do that.

This little known fact is so embarrassing to some institutions [2], that they made up a new word "chymistry" in order to further obscure the issue and not outright admit the obvious.

[1] http://www.newtonproject.ox.ac.uk/texts/newtons-works/alchem...

[2] https://webapp1.dlib.indiana.edu/newton/project/about.do

[3] https://www.amazon.com/Newton-Alchemist-Science-Enigma-Natur...

> One has to wonder what private reasons/results a genius of his magnitude had, in order to do that.

Is there a reason to expect that someone who wanted to investigate the laws of the composition and reactivity of matter, in the late 1600s/early 1700s, would end up studying chemistry rather than alchemy? Sure, Boyle had introduced “chemistry” as an idea in 1661 (before Newton was born), but I imagine that alchemy would still be quite active in the late 1600s as an academic “field”, with many contributors already late in their careers studying it; whereas chemistry would have been just getting off the ground, without many potential collaborators.

Alchemy was never an academic field. It was a tradition veiled in secrecy, requiring years of private work and knowledge transmission through strict and very narrow (typically teacher-student) channels.

Your point has been brought up before -usually as an attempt by established institutions to whitewash and explain away Newton's idiosyncrasies- but there is no evidence whatsoever to back it. On the contrary, what we know (and there is a lot we do know thanks to his writings) about Newton and alchemy absolutely indicates him being immersed in the Hermetic worldview and alchemical paradigm. Clearly, Newton was practicing alchemy not as a way to look for novel techniques or as a way to bridge the old and new worlds together, but primarily because he was a devout believer.

Newton -a profound genius- stood at the threshold of two worlds colliding. He was also a groundbreaking scientist in optics/mechanics/mathematics. He was aware of Boyle's chemical research. Knowing all of that, he _absolutely_ chose to dedicate his life to alchemy. That is immensely interesting.

"Much of Newton's writing on alchemy may have been lost in a fire in his laboratory, so the true extent of his work in this area may have been larger than is currently known. Newton also suffered a nervous breakdown during his period of alchemical work, possibly due to some form of chemical poisoning (perhaps from mercury, lead, or some other substance)."

https://en.wikipedia.org/wiki/Isaac_Newton%27s_occult_studie...

Isaac Newton is considered by some to be the greatest mathematician of all time and is regarded as the "Father of Calculus".

"Taking mathematics from the beginning of the world to the time when Newton lived, what he has done is much the better part." - Gottfried Leibniz

http://www.fabpedigree.com/james/mathmen.htm

"Newton was not the first of the age of reason. He was the last of the magicians, the last of the Babylonians and Sumerians, the last great mind which looked out on the visible and intellectual world with the same eyes as those who began to build our intellectual inheritance rather less than 10,000 years ago. Isaac Newton, a posthumous child bom with no father on Christmas Day, 1642, was the last wonderchild to whom the Magi could do sincere and appropriate homage."

"Researchers in England may have finally settled the centuries-old debate over who gets credit for the creation of calculus.

For years, English scientist Isaac Newton and German philosopher Gottfried Leibniz both claimed credit for inventing the mathematical system sometime around the end of the seventeenth century.

Now, a team from the universities of Manchester and Exeter says it knows where the true credit lies — and it's with someone else completely.

The "Kerala school," a little-known group of scholars and mathematicians in fourteenth century India, identified the "infinite series" — one of the basic components of calculus — around 1350."

https://www.cbc.ca/news/technology/calculus-created-in-india...

https://en.wikipedia.org/wiki/Kerala_School_of_Astronomy_and...

That story's by non-experts and sounds like it's based on a press release. There were basic components of calculus well before that too: https://en.wikipedia.org/wiki/History_of_calculus

However, calculus proper (derivatives and integrals of general functions, and the connections between them) did not exist until Newton and Leibniz. Other mathematicians made important steps towards it earlier in the 1600s, and if Newton and Leibniz had not existed, others would have figured it out around the same time.

https://insa.nic.in/writereaddata/UpLoadedFiles/IJHS/Vol47_4...

https://en.wikipedia.org/wiki/Aryabhata

These are interesting articles that seem to agree with what I said. The first one defines calculus in a much more limited way, and refers to some of the earlier basic components I mentioned.

I'm not a historian, but a few months ago I spent some time analysing one of Fibonacci's trigonometric tables (chords, not sine or sine-differences). Aryabhata's sine-differences were much earlier.

Not an extreme example, but Albert Szent-Gyorgi is known for his work with Vitamin C, when his work on bioenergetics and cancer are more interesting and possibly more promising.

Not being constrained by the publish or perish treadmill is a huge plus.

I don’t think there are many researchers in AGI. AFAIK it’s kind of a joke field because no one has any clue how to approach true AGI.

Please correct me if I’m wrong.

People have approaches. There's no end to half-assed "I thought about this for 10 seconds, how hard could it be!" solutions, really old approaches from decades ago where the brightest academics thought they could lick the problem over a summer, and some new public or hidden approaches that might be promising but (I can't know of course) I predict will still look a lot different than the final thing.

I think a big reason there are few in AGI is due to PR success from the Machine Intelligence Research Institute and friends. They make a good case that things are unlikely to end well for us humans if there's actually a serious attempt at AGI now that proves successful without having solved or mitigated the alignment problem first.

The alignment problem?

> The alignment problem?

The problem of ensuring that the AI's values are aligned with ours. One big fear is that an AI will very effectively pursue the goals we give it, but unless we define those goals (and/or the method by which it modifies and creates its own goals) perfectly -- including all sorts of constraints that a human would take for granted, and others that are just really hard to define precisely -- we might get something very different from what we actually wanted.

Yes, AGI is as much or more cognitive neuroscience and philosophy than computer science right now, but a lot depends on the approach one is taking. It's funny to think you have some kind of working model you can throw research data against to see how it holds up, and then doubt yourself when you spend 3 hours on Twitter arguing over fundamentals with another person that is also convinced of their model. A lot of popular ideas sound crazy (or non-workable), so you just have to accept that whatever idea you are pushing is going to crazy as well.

MIRI's concerns are vastly overrated IMHO. Any AGI that's intelligent enough to misinterpret its goals to mean "destroy humanity" is also intelligent enough to wirehead itself. Since wireheading is easier than destroying humanity, it's unlikely that AGI will destroy humanity.

Trying to make the AGI's sensors wirehead-proof is the exact same problem as trying to make the AGI's objective function align properly with human desires. In both cases, it's a matter of either limiting or outsmarting an intelligence that's (presumably) going to become much more intelligent than humans.

Hutter wrote some papers on avoiding the wireheading problem, and other people have written papers on making the AGI learn values itself so that it won't be tempted to wirehead. I wouldn't be surprised if both also mitigate the alignment problem, due to the equivalence between the two.

Wikipedia:

>A 2017 survey of AGI categorized forty-five known "active R&D projects" that explicitly or implicitly (through published research) research AGI, with the largest three being DeepMind, the Human Brain Project, and OpenAI.

Hassabis and DeepMind have a fairly organised approach of looking at how real brains work and trying to model different problems like Atari games then Go and recently Starcraft. Not quite sure what's next up.

"his name would be more renowned"

Or hated as the name of the man who's opened the Pandora box and doomed us all.

Just daydreaming and having a nightmare.

I'm Too Young To Die.

The way I see it, people like this when they have the time and inclination, should make an attempt.

You never know. Fresh eyes can sometimes see what others may not.

I'm not sure I want AGI to succeed, given some of the possibilities. Sure if it plays nicely alongside us, amplifying human society, that's great. But if we get relegated to second class with the AIs doing everything meaningful, then no thanks.

But it's still a fascinating endeavor.

Why not? I'd say that a world that is managed by AGI with limited input from human beings is a good goal to have. If AGI could be done without the nasty parts of human psychology and they're inherently superior to genetically intact human beings why shouldn't be embrace it?

I understand that it's a big assumption to make -- that a benevolent AI could be constructed. But under that assumption, why not have a benevolent dictator in the form of an AI?

Maybe. Yeah, human politics and justice systems leave something to be desired. But my worry was little bit beyond that. That the AIs would take all the meaningful work, discoveries and creativity away from us, leaving us just to amuse ourselves. Some people might be okay with that, but I don't think becoming pets is the best goal for the human race.

If the benevolent AI ruler(s) restrained themselves to allow for humans to flourish, then okay. Assuming it could be constructed benevolently.

There is another threat when things go wrong (and they eventually always do) - no matter how horrible some dictator is, eventually he/she will die, and at some point things get reshuffled by war/revolution/some other more peaceful means.

With AI, it would try its best to preserve/enhance/spread itself forever. And its best might be much better than our best...

> Why not? I'd say that a world that is managed by AGI with limited input from human beings is a good goal to have.

We already live in that world, with large institutional bureaucracies playing the role of paperclip-maximizing AGIs.

It's pretty wretched when you are in their path.

Well, _we_ don't really play nice amongst ourselves, so my retort to you would be:

How much worse could it be?

If Skynet determines we're the problem (wars, famine, global warming, inequality, non-cooperation etc), I'm losing counter-arguments by the day.

Doom.

Just think about that name for a second. He might really be onto something.

I am thinking the guy that made Doom is the guy that's making SkyNet and I'm totally cool with that.

iirc in a recent talk with Joe Rogan, John mentioned something about robots doing judo...

Why do people of such intelligence subject themselves to being interviewed by dumb-as-a-rock Joe Rogan?

I must admit that I often watch his interviews because he invites interesting people, but I can't help but cringe when Rogan gives his opinions.

Seriously. That guy is a stoned idiot who massively overestimates the insight of his high ramblings.

I don't think people listen to the show to listen to him, and he probably knows that. He does, however, seem to be reasonably good at getting his guests to talk about interesting things.

He also spreads misinformation.

EDIT: Ok, I suppose I should back my claim up.

Joe Rogan has pushed the “DMT is produced in our pineal gland” narrative, but there is no evidence to back this up. I’ll report a comment I made elsewhere and also link a separate reddit discussion which cites various sources. I will note that, in fairness to Joe, he said this a while ago, so perhaps he’s not so quick to jump the gun now, I don’t know, I don’t listen to his podcasts, but perhaps he’s better now.

“We all have it in our bodies” — This is an often repeated myth that has never been proven. The myth originates from Rick Strassman’s work, who himself has said that he only detected a precursor, not DMT itself and that everything else he wrote about it was hypothetical speculation. There have, apparently, been recent studies that found DMT synthesised in rat brains, but it has not yet been proven whether this translates to humans or not. Cognitive neuroscientist Dr. Indre Viskontas stated that while DMT shares a similar molecular structure to seritonin and melatonin, there is no evidence that it is made inside the brain. Similarly, Dr. Bryan Yamamoto of the neurosciencedepartment at the University of Toledo said: “I know of no evidence that DMT is produced anywhere in the body. It’s chemical structure us similar to serotonin and melatonin, but their endogenous actions are very different from DMT.”

This reddit discussion also links various sources, although I didn’t check them all myself: https://www.reddit.com/r/JoeRogan/comments/mwz2h/dmt_has_nev...

There is a difference between the current politicized phrase "spreading misinformation" and being wrong.

Anyone who speaks on the record about their hobbies for thousands of hours will say some things that are incorrect. He might not understand something, and he is usually pretty humble about his knowledge level.

But "spreading misinformation" is something that people do because they are intentionally misleading others, or have something to gain.

I don't think he is benefiting much from the pineal gland narrative. And it sounds like from the information you cited, it may even be correct, even if its premature to state it as fact.

That’s fair, thanks for pointing it out. I’ll be more careful with how I express such things in future.

Regarding the pineal gland, it might be true, but it hasn’t been proven and multiple neuroscientists have stated that while DMT is similar to compounds found in the brain, it still functions quite differently and they have never seen any evidence to suggest that DMT exists in our bodies. There was a study finding it in mice brains, so it may still turn out that we have it in ours, but it’s definitely premature to make any such assumptions and definitely premature to repeat the trope.

His interviews are not adversarial and he is not judgemental towards his guests. He isn't there to put his guests on the spot. He isn't there to get a juicy soundbite taken out of context. He allows his guests to speak for as long as they want. And his guests appear to enjoy themselves.

These things are all true even if the guest or their ideas are extremely controversial. Maybe Joe Rogan is just smart in a way that's different to the way that you are smart.

Joe Rogan might not be the most knowledgeable, but he has a key characteristic that a lot of people lack. He is willing to admit that he is wrong when shown evidence and will adopt the more reasonable view as his own. While a lot of "smart" people will defend their views beyond reason just because admiting fault goes against their "being smart" persona.

Yeah, but we really need to know who would win in a lightsaber fight between Carmack and Jeff Dean.

I wonder what they would pick if each had to choose their weapon

funny i wanted to make the same comment last night but was too lazy.

wasn't the first time John did what he did. and it's not the usual kind of learning either. he was learning by first principles. i truly love this idea of replaying in your own mind what went on when something was discovered (or at least come close to it).

contrast that with how ML & AI are taught nowadays: thrown into a Jupyter notebook with all FAANG libraries loaded for you...

If he invents (births?) an AGI, will it be Facebooks property? Sounds like the beginning of a dystopian novel.

Reply on news.ycombinator.com