Hacker News Reader

Comment by thedz | Hacker News Reader

> “I understand the state-of-the-art papers,” he says. “The math is simple. For the first time in my life, I’m like, ‘I know everything there is to know.’ ”

Yep, he's still in his twenties.

View on HN

But that belief is enough to attempt something that more experienced people would hesitate to start.

Naivety is a very good thing at times.

I've seen average people achieve incredible things, and not because what they did was incredible... but just because they started work on things that no-one else thought they could complete. Some way into it, when enough progress has been made, people have rushed to give support because "halfway there but badly done" is a hell of a lot better than "not even started yet".

I don't disagree with that. I've worked with some very smart people in my 20s who sounded similar to Hotz -- enthusiastic, retrospectively naive about their understanding of a field, but above all, superbly intelligent. They did really great things, things that maybe didn't work perfectly or as envisioned, but still things that might scared off more experienced folks.

But also now that I am in my 30s, and they are as well, we frequently look back at that time and laugh about being that young. "Man, you were fun to work with, but also what were we thinking"

So I definitely wish Hotz all the luck. If nothing else, the more smart people working on the problem of self driving cars, the better.

My comment mostly stemmed from amusement of his quotes.

There have been people in my past that wanted to start a project that I didn't think they were capable of finishing, because either it was too large, they didn't have the skills/smarts (not that I thought they were stupid, just that I thought it would take exceptional intelligence), or both. A few of them succeeded, either in the original task, or the effort and journey was well worth the price paid.

Part of this was hubris. The thought of someone I considered less capable than myself accomplishing something I felt I could not damaged my ego. This was humbling.

Part of this was experience. The experience to know that attempting the hard or impossible is sometimes worth the effort, whether you succeed or not. This was educational.

Part of this was ambition. Ambition to do something new, to ignore the naysayers and noways when needed, and forge your own path, which I've always felt short on, but have steadily worked on over time. This is ongoing.

Another part of my problem is that I have too many projects I want to do. Learning about AI is one example, but I've instead done a series of web and mobile apps which are much closer to success. It would take a lot of time to read all the AI research and become good enough to tackle a problem like self-driving cars, and I've only got my spare time at home, with which I must also make sure my wife remains happy (ignoring her seems to make her unhappy for some reason) and keep my sanity (read fiction or play a video game some times) and take care of my house (the lawn just won't stay mowed).

I do remember being about 19 and thinking I was the best programmer in the world. By about 22 I had rewritten as much of my old code as I possibly could because it was so horrible. Somewhere between there and now I've gotten a cynical bit of humility to tamper my ego. I think the cynical part is that my ambition has not lessened, just my belief that I can succeed.

One Steve Jobs philosophy is focus and say no. I'm guessing I could do better if I said no to all but a single project.

A friend of mine in college had a very good saying about this that I always keep in mind:

"There's nothing like succeeding at something you weren't even qualified to attempt."

When you fail at something you're not qualified for, it doesn't feel like failure. You're able to get right back up without even bruising your ego.

Thanks for sharing.

I like this.

Sure. But that belief is a genuine worry when you are talking about creating something that'll move a hulk of metal down a road at 70mph.

if we don't have laws currently, we sure as hell need them. i can't imagine letting everybody try their own self-driving software.

Millions of drunk people drive on the roads. People can buy assault weapons, with no training and background checks. People who make self-driving software should be the least of our concerns.

There are laws against drunk driving (and harsh penalties for those that are caught), and you can't buy a firearm without a background check from a dealer (with more states requiring gun show dealers perform background checks now, too).

People building untested self driving cars is an entirely legitimate concern.

There aren't laws against driving while teenage:

"Nationally, 963,000 teen drivers were involved in police-reported motor vehicle crashes in 2013, which resulted in 383,000 injuries and 2,865 deaths"

I'd worry about that more than the odd geek with a laptop.

Legitimate: yes. Worthy of concern: absolutely not.

I don't think this would help at all because 1) most people are not interested in making their own self-driving car and 2) the small niche who is interested isn't going to worry about following the laws, as Hotz states in the piece.

From the article:

“I live by morals, I don’t live by laws,” Hotz declared in the story. “Laws are something made by assholes.”

Indeed, I would have a word with whoever made the 2nd law of thermodynamics.

This is just nitpicking. He is clearly talking about social laws, not fundamental laws of nature which are clearly different in scope and application.

So he'll feel morally okay if he kills or cripples someone by being reckless?

i wonder where he believes laws came from ...

From assholes. i.e. other people.

"Assholes", it would appear.

This was my first thought when I read the article. There ought to be some test track qualification before allowing a new system to be tested on the public road.

we already have enough laws. responsibility doesn't change because a computer does some thinking for you. Besides, how many people are actually writing their own self-driving software?

>Naivety is a very good thing at times.

Thats what I think 'foolish' means in "Stay hungry, stay foolish."

"Naivety is a very good thing at times."

He might not be naive.

>>I've seen average people achieve incredible things

In my opinion, we must never underestimate people.

>>but just because they started work on things that no-one else thought they could complete.

Nothing fails like smartness. The reason why a few people achieve the impossible while far more intelligent and smart people don't is because of the curse of intelligence makes them believe certain things are impossible.

The fool didn't know it was impossible, so he did it.

I would totally agree with you IF this kid hadn't proven his chops with iPhone and PS3 hacks, not to mention building a self-driving car in his garage.

I also realize this kid probably won't end up making a huge dent in the universe.... but.... statistically speaking, there should be several "Leonardo da Vinci"- level humans alive right now. Why not this kid?

> I would totally agree with you IF this kid hadn't proven his chops with iPhone and PS3 hacks, not to mention building a self-driving car in his garage.

Impressive as they are, his chops still don't support his claim to "know everything there is to know". The Dunning-Kruger effect is in full swing.

Sure, but that claim was made in the context of deep-learning networks. He went to work for an AI company, and realized that he knew -- from reading cutting-edge academic papers -- as much as the forefront of the field. He wasn't claiming to know everything there is to know in general, or even in software development, just that he can understand and implement machine learning with the best of them. Personally, I don't doubt that claim.

The field doesn't require a particularly extensive background either. A good grasp of linear algebra and multi-variable calculus basically has you set to understanding even the state of the art in the field. Of course, coming up with the papers would require a whole lot more work.

"I know everything there is to know."

I kind of took that to be like how Musk talks about needing to know first principles. In the article you can see that he was humble about what he thought he knew, took jobs here and there and eventually confirmed that he was at the cutting edge, that he knew 'everything there is to know' about this special area.

That's when he realized that he was qualified to try this. IMO, anyway ;)

On a smaller scale, I remember one day realizing that I, a self-taught programmer, knew more than my boss. Within six months I left to start my own company.

Nice. I'm still struggling to feel I know enough to do my own thing other than lead gen and optimization for others.

How smart was your boss and how did it go with your company ?

The boss was pretty smart in the sense of knowing how to work with big corporations to build large decision support systems. But his technical knowledge was fairly shallow.

I sold my first company and the investors did very well, but I made tons of stupid mistakes in the process. Not least of which was holding on to dotcom stock that I thought would go to the moon but which mostly went down the drain.

That statement is clearly tongue-in-cheek, come on. "I well-understand the cutting edge of this narrow research area" is less fun to say, but that's the meaning.

Typical twenty year olds don't read all the state-of-the-art papers on a subject before saying they know it all. It sounds more like he's caught up on the latest AI research and fundamentals.

If he really did read the papers then it's clear he would not say this. The papers aren't an end. They describe incremental progress. Having just returned from NIPS where most of the researchers are saying "We don't know" all the time, it is ironic.

Precisely. We have some stuff that works and we don't know why.

People in their twenties wrote those papers.

The math in any of the paper's he's most likely referring to aren't some theoretical pde maths or abstract algebraic geometry stuff... it's pretty understandable if you can grasp a "graduate level" linear algebra course.

That's true. My point was to respond to the breathless reporting about that this guy has achieved so young. He's replicating the work of other mostly young people. Doing it the first time is the trick.

(I'm very familiar with this literature - see my username)

To be fair, this is machine learning we are talking about, not algebraic topology. The experts in ML are still proud of the fact that they figured out the chain rule...

> The experts in ML are still proud of the fact that they figured out the chain rule...

I assume you're talking about using backpropagation with gradient descent. Backpropagation itself isn't all that interesting. The interesting part is that it works for practical problems and doesn't get stuck in shallow local minima.

Nevermind that they have no idea of the behavior of the partial derivatives nor attempts to model such when presenting their "latest and greatest", at least from most of the stuff I've read that's been posted here…

Its so much fun to see people who crossed their 20s doing nothing close to great, get so jealous.

I cringed. Saying "the math is simple" -- ouch. The writer must have been barely able to suppress his glee when that one popped out.

I don't doubt it at all. Keep in mind that "simple" is relative, we have to ask "simple compared to what?" For lots of people, I be that the math involved in these neural nets is the most complicated math they've ever done. They would never say it is simple, because they themselves barely grasp it. But in my experience, topics in mathematics have a funny way of becoming very simple the moment you "graduate" to thinking about a slightly more general mathematical framework.

Someone who has digested enough of the AI literature to think about the methods in aggregate is very likely to be in a position to see any particular method as a "simple" implementation of some more general set of principles.

As a general observation, what you say has some truth to it.

But the particular quote is referring to learning rates in autonomous robotics, especially visual classification in complex real-world scenes.

I have worked and published in ML since the early 1990s, was a program chair for the learning track at NIPS one year, participated in the same DARPA learning-to-drive program that Yann LeCun did, and don't consider the math behind "state-of-the-art papers" to be simple.

Just taking deep learning: there are a lot of tricks and recipes (e.g., rectified-linear activations, number of layers, staged training) that are not mathematically understood. It's exciting, but mathematically still a jungle. Just because a neophyte can code and optimize a network does not mean that the math that explains why it actually works is simple. As engineers, we need to understand why it works before using it in a safety-critical situation.

While a good point that simple is relative, if you specifically look at deep neural networks we don't understand how training a non convex function converges with gradient descent - the fundamental component to create a usable model. In practice, it often works, and there are a few intuitions why this works. But its naive at best to claim that this is simple. If it was we would understand it better :)

While I don't know a lot about the subject, I would bet that's likely right. As in, there are very hard problems, but the actual mathematics are not all that hard.

The math to implement a working neural net is indeed simple. Even if you consider all the commonly used engineering practices to ensure its correctness and improve its accuracy (like dealing with under/over-fitting), it's still not that hard. In the end, it's just doing multiplications over matrices, calculating derivatives and propagating values back and forth.

Now, to understand WHY the algorithms work, and gives you the result it claims to calculate, is quite hard, but that understanding is not required to implement those algorithms.

Well, a lot of people on HN certainly comment on technology without immodesty and an air of authority. It does feel boorish to have someone say that out-loud though.

I'd typically tend to agree with you (that a guy in his 20's saying he knows everything is ludicrous), but geohot is an actual savant.

I take that to mean that for the first time in his life he read the papers and fully understood them without needing additional background, not that there isn't more to learn outside of those papers.

He may well know everything there is to know today, but there's bound to plenty more breakthroughs in AI research. It would be like Newton saying "I know everything there is to know about physics today".

Nobody created anything great by first fully appreciating the size and difficulty of their endeavor. I would say, underestimating a problem, and overestimating one's skills are crucial to innovation and progress.

or just legitimately an expert on the field.

> I know everything there is to know.

If only he knew about the Dunning Kruger effect...

I wish I could upvote this more. A person in their 20's knows nothing, but thinks they've outsmarted the world. It's not until your 30's that you realize how big of an idiot you were/are and how much of the world you actually understand (read: little).

This whole: "Twenty year olds don't know shit but 30 year olds are so enlightened" sentiment needs to stop. I agree that many younger people think they understand more than they do but that's just part of growing up and we all go through it.

You are misunderstanding the sentiment. No one is saying 30 year olds know everything, they are saying 30 year olds _realize_ they don't know everything.

Or as Socrates put it, "the only true wisdom is in knowing you know nothing"

(probably one of the more Buddhist-ish gems from Western philosophy)

Why does it need to stop if you agree that its a fact of life. No one is saying 30 y/o's are enlighted, just that they have a bit more perspective. The same can be said for twenties vs. teens. It's not that teens are idiots - they are just teens with the life-experiences & perspective of a teen

This:

> A person in their 20's knows nothing, but thinks they've outsmarted the world...

Is a dangerous and gross generalization. I totally agree with the changing of perspectives point, but feel that this community has a very clear bias from the older gen (30s and up) against the younger gen (teens and twenties). That's all I'm saying. It's divisive. Instead of saying they "know nothing", it should be phrased, "still have a lot to learn."

That is just a semantics argument.

The semantics are relevant though as it highlights the sentiment I'm trying trying to shed light on in my original comment.

I dont believe that's accurate though, saying people in their 20's know nothing, or people in their 20's still have a lot to learn is just a way of restating the same fact. But that statement doesn't mean that people in their 30's are enlightened or smarter, only that they now understand how much they don't know.

> This whole: "Twenty year olds don't know shit but 30 year olds are so enlightened" sentiment

Deleted comment

Age is shorthand for "I've spent X years making a lot of mistakes and learning from them."

Watch Geohot do a CTF live: https://m.youtube.com/watch?v=aZJM-iIpbqc

I think the point you are making is generally valid... But he is a savant. I don't think it is wise to apply generalities to him.

Yes, age will change some of his sharper edges, but he is already pretty unusual.

I took his meaning to say that with all the stuff about technology and AI, he's at a point where he feels like he can start innovating because he's learned enough (it says he went back to school to get his PhD and worked at an AI company before quitting to work on the car), hence the reason why he feels so sure that his technology is better than Mobileyes.

I don't think this has anything to do with saying he knows everything he needs to know in the world.

I'm not disagreeing, but it's in bad taste to make a personal judgement on someone without meeting them personally. The guy could be cocky, or the writer of the article could've just made the guy appear cocky.

Anyways, wouldn't you agree that it is better to be empathetic rather than thinking you're an idiot?

...said a guy in his 30's

Busted. Which is why I didn't say shit about what it's like to be in your 40's because ... I have no idea.

Oh it's horrible, you've seen whole cycles so know how even the good things you could do next go bad in the end. It's easy to fall into excessive cynicism, plus stop learning new stuff because of the 30s lesson of how hard it really is to learn in full.

To be honest, I recommend faking to yourself that you're in your 20s still :) Much healthier attitude.

Isaac Newton invented calculus in his early 20s

He also nearly killed himself with alchemy experiments. He was very right about some things, and very wrong about others.

And he started doing this in his 30's/40's when he had stopped contributing to physics. Maybe you grow more senile the more you age.

how many pioneering mods / hacks did you do in your 20's ?

Reply on news.ycombinator.com