No One Knows What They're Doing (2010)
jangosteve.com
jangosteve.com
“I think it's much more interesting to live not knowing than to have answers which might be wrong. I have approximate answers and possible beliefs and different degrees of uncertainty about different things, but I am not absolutely sure of anything and there are many things I don't know anything about, such as whether it means anything to ask why we're here. I don't have to know an answer. I don't feel frightened not knowing things, by being lost in a mysterious universe without any purpose, which is the way it really is as far as I can tell.”
If this were one of my open source projects or programming articles, I'd say, "Let me know if you have any questions." But I'm not sure I could answer anyone's questions on this subject. I'll sure try though if you do have any.
If you wait until all the studies are complete before trying to do something, you will never take action. Life is more about trial and error than knowing how everything is known.
Most of the time I think it's a matter of spotting which the heavier factor is.
Sounds a bit like confirmation bias in action to me; there are almost certainly times you are crystal clear on what's going to happen and then it doesn't happen.
argmax(sum(probability_of_event * utility_of_event) given action)
So do you open a door ? There could be a bear behind the door that will eat you if you open the door. The probability of that is not zero (and even if it was, it would not matter). The utility of that is -inf. The utility of not opening the door is also -inf. There could be a bear behind you and the door could be your only way to safety.
So a rational actor would never do anything at all, as the rational function would never converge.
Ergo, nobody and nothing is rational (as soon as the world has a certain minimal level of complexity).
There are various suggestions given out of this conundrum, but none are even mildly satisfying. For example, one could claim that you don't know those odds, so you don't care. But that is the same as saying you can't really be rational, so it's not a solution.
Every decision has a ROI. The line where ROI meets costs is your maximum budget for a decision, after which point it costs you more to remain in the decision phase that it costs you to follow the best of the few "imperfect" decisions you have in mind (one of them includes the decision to stop "deciding", do nothing, and move on - that's still deciding).
That idea so far is rational. If you wear nothing you'll remain naked. If you don't eat you'll die.
And most of all, being stuck deciding forever just about anything is not rational.
So this person is unable to sense his "decision budget" and decides forever, while most people have facilities (impatience, impulsiveness in some degree, boredom etc.) to work around us being stuck forever after our decision budget is spent.
Spending to get positive ROI is rational. Spending infinitely beyond that point is not.
(In that, at least in some sense, hunger is an irrational desire for food, it isn't reasoned out)
> The utility of that is -inf. The utility of not opening the door is also -inf.
Humans don't value anything--even their own continued existence--at inf, just at pretty average-levels-of-high numbers. We seem to get "intelligent" things done despite this. What's wrong with building an AI this way?
But aside from that, humans are not rational. Depending on your definition, humans aren't even intelligent.
When it comes right down to it, it's pretty obvious what the human mind does to solve problems. It copies behavior. That's all it's capable of. It doesn't try to be rational, good, ... It is like DNA in that way (where 99.9999999999% of behavior simply comes from the parent(s)). People keep forgetting that evolution didn't create a "human", evolution is simply trying to duplicate itself. Your brain is nothing more or less than another trick to accomplish that.
More detailed, here's what it does :
1) it looks at it's inputs, and models them : it finds how the feedback loop works between outputs and inputs (e.g. you move muscles 382, 182 and 138, your hand moves in the image of your eyes. That sort of thing and more complex)
(technically it doesn't really do this. Your brain models the outside world as if it's part of your brain. That it models accurate prediction models is just a consequence of that)
2) it "shortcuts" the feedback loop. Essentially if neuron B always sees something happen 10 ms after neuron A sees it happen, they will form a connection and neuron B will report the event when neuron A starts seeing it's other event. If this means anything, it's a huge autocorrelation machine (with only positive offsets)
3) it copies behavior from it's predictive model to it's outputs. A human effectively tries to be the most "average" human in existence, based on his/her idea of what that is. It also copies (much less) from other events, like trees moving, animals, even just random signals that somehow get into your nervous system (some were put there on purpose). And lastly it copies from various internal sources that were put into our nervous system, like various clocks (I'm sure you can see why a clock would be useful to correlate things with), and chemical signals that get translated (like hunger, pain, or exitement, or ...).
Keep in mind that I can split it up into functional things as much as I want. "Phase 1", "Phase 2", "Phase 3", but they don't really exist. The brain starts with a random signal and then does one operation continuously. That one operation implements 1,2 and 3 as a consequence of
The informal definition of rational is having good sense and good judgment. If your formal definition results in agents that never do anything, or are in basic conflicts with millions of years of evolution and survival, then it's far more likely the formal definition is wrong, or your understanding of it is wrong, and not humans.
I understand the benefits of strict formal rational analysis. You can reason about it. But then if you'll talk about that, don't compare it to intelligence. Strict rationality is not intelligent. It's a tool we've made ourselves that has specific strengths and weaknesses born out of its restrictions.
And strict rationality is absolutely dependent on the model you apply it on. Wrong model + perfect rationality = disastrous decision. No one talks about the models, ever, it seems.
This is the real problem with AI - that we try to implement our very rough (at this point of time) idea of what intelligence is. If we knew all the details of animal/human intelligence, we wouldn't be sitting here debating it, we would've just implemented the AI with it.
Assume a person has a bunch of algorithms available to them. They may not be able to tell if algorithm X works with problem Y until they start interacting with problem Y and find out that heck algorithm X does work.
With this approach, we can imagine knowledge that actually can only exist hidden within multiple of level not-knowing, at least if knowing is being able to fully characterize one's abilities.
What he forgot to add was the crucial fourth term: the "unknown knowns," the things we don't know that we know-which is precisely, the Freudian unconscious, the "knowledge which doesn't know itself," as Lacan used to say.
If Rumsfeld thinks that the main dangers in the confrontation with Iraq were the "unknown unknowns," that is, the threats from Saddam whose nature we cannot even suspect, then the Abu Ghraib scandal shows that the main dangers lie in the "unknown knowns" - the disavowed beliefs, suppositions and obscene practices we pretend not to know about, even though they form the background of our public values."
http://www.lacan.com/zizekrumsfeld.htm
I agree with Zizek's assessment that unknown knowns are even more dangerous than unknown unknowns.
If you ask an undergrad student, the answer is always a "yes" or a "no".
If you ask a master's student, the answer is always a "may be".
If you ask a doctoral student, the answer is always "i don't know".
Nor does it ever connect the "recognition of what you don't know" ability back to the nurse's situation: how does that account for why she got such praise? What observable shortcomings did the other students have, and she lack, that are connected to the unknown-unknown problem?
The way it was meant to be brought back around to the nurse's situation was that, she actually was really good at what she was doing for her level of experience and progression through schooling, despite knowing that she had a long way left to go. As to what observable shortcomings other students had compared to her, I cannot say. I wasn't in the class or at work with her. The point was that her professors and supervisors, all of which gave her praise, were probably better able to see that than she was, given their experience and level of context within the field of nursing and teaching.
The real issue, though, was in the cognitive dissonance between what she felt she was capable of and what others knew her to be capable of. This was an attempt to give at least some small explanation for that cognitive dissonance. It is possible to know more than anyone else about a given subject, and still know practically nothing about it compared to the amount of information about that subject contained within all of mankind or even the universe.
>But of course, that more experienced peer group also tends to be much humbler and aware of their shortcomings as well, so I'd say the article still addresses that.
I have to disagree: however humble they might be, someone on the projects I mentioned has to have a deep enough understanding to know how to handle the cornucopia of problems that can derail it: unexpected soil problems, dependencies not working like intended, aerodynamic lift not being as high as the model.
I compare it to my understanding of long multiplication, which I've tested against students new to it. "How do you do it?" Here's how. Why is it done that way? Well it's an expansion of this formula. "Why in this order?" If you don't do it that way, you're actually multiplying these other numbers; you have to correct for it like so. "What if I forget some of the entries in the multiplication table?" Then here's how you multiply single digit numbers; here's multiplication as a version of addition. "What if I forget how to add?" Break out the number line, let's review.
That kind of understanding is revealed in any large-scale, finished, working product.
>The way it was meant to be brought back around to the nurse's situation was that, she actually was really good at what she was doing for her level of experience and progression through schooling, despite knowing that she had a long way left to go. As to what observable shortcomings other students had compared to her, I cannot say.
Then I don't understand how this was supposed to reassure her; if I offered an explanation, I would test its implications against the rest of the scenario: is there a way overconfidence manifests in the typical student? Is she returning the (correct answer) "I'd have to ask someone" more often than the other students? As it stands, you've just listed something that might be consistent.
Remember, the problem is not that "she was good for her level of experience"; the problem was that "she was called the best student they'd ever had".
(I note with some irony that, for you claim to know general competence levels just after college is itself seemingly outside your knowns!)
I agree with what you said, which is why I have a hard time understanding what exactly you disagree with. Are you saying that more experienced people tend to be less aware of their shortcomings? Or maybe we're taking humble to mean two different things? I'm using "humble" in the "not arrogant" sense, not the "feeling inferior" sense.
I think you can be confident in your abilities while still being humble about them, and that's really what I meant by it when referring to the more experienced peer group. In fact, the most experienced people usually have an innate sense of confidence that allows them to avoid having to constantly tell people "I know what I'm doing". They don't have to tell people, because they can show it. That's the level of humbleness (or modesty may be a better word) that I was referring to.
> Then I don't understand how this was supposed to reassure her; if I offered an explanation, I would test its implications against the rest of the scenario:
That is generally the next step. You make observations, develop theories to explain the observations in a consistent manner, and then devise new tests to test those theories. This post was simply at the "developing a theory" step.
> (I note with some irony that, for you claim to know general competence levels just after college is itself seemingly outside your knowns!)
The irony is not lost on me. To be fair though, I didn't and still don't claim to "know" general competence levels, I just blogged my thoughts. Of course I could have littered my post with all sorts of qualifiers like "maybe", "perhaps", "sometimes", and "I think", but that doesn't make for effective writing. The post was not written or meant to be read in a vacuum. Context is important.
The "No one knows what they're doing" bit. The people on those projects -- they know what they're doing.
http://en.wikipedia.org/wiki/Four_stages_of_competence#The_f...
Early in life you get the impression that parents know everything, then it's teachers, then it's bosses. When in fact they don't.
We're not robots with perfect abilities, understanding that everyone to a degree doesn't know what they're doing gives you a different outlook on the world.
Note, that doesn't mean everybody is incompetent.
How can you possibly make this assertion? I have a direct counterexample; I know what I'm doing. There's a lot of stuff I don't know, but I would not describe my state of being as "not knowing what I'm doing".
>everyone to a degree doesn't know what they're doing
This is very different from "Nobody knows what they're doing". Just because no one has a perfectly clear idea of everything doesn't mean they don't know what they're doing.
EDIT: Actually "all the time" doesn't quite cover it, I'd say it's "most of the time".
Then so does the statement "Nobody knows what they're doing.". The author has no idea what people know or don't know.
... Er, no. There's only so much complex, useful stuff one can produce, before you have to conclude, "okay, that person understands something."
That seems like a very broad claim that can't possibly be true without some sort of qualification. Are you saying that there are people who know 100% what they're doing 100% of the time in 100% of endeavors across every category relevant to living life? I feel like what you mean to say is that there are people who know exactly what there doing when it comes to X, where X is a well-defined, specific subject-area.
Assuming that statement is true with the scoped qualifier, even that wasn't always true for this hypothetical person. They had to have learned how to do X and gained enough experience over time to know exactly what they're doing in the matter. Which means, there was a time when they didn't know what they were doing. And I'd say this hypothetical person at that hypothetical time before they knew what they were doing is really the target audience for whom the article was written.
> There are known knowns; there are things we know that we know. There are known unknowns; that is to say, there are things that we now know we don't know. But there are also unknown unknowns – there are things we do not know we don't know.
Nevertheless, whenever I feel like I'm confronting bullshit artistry on a profound and deliberate scale, Donald Rumsfeld is my yard stick, and a distorted image of his smirking face is conjured in my mind's eye, and replaces the bullshit artist's face, and that quote rings through my head with lots of reverb and phased, flanged echo.
The next unit of measurement in scales of bullshit artistry, in terms of greater precision and smaller units, is Bill Clinton's debate regarding the meaning of "is":
It depends upon what the meaning of the word 'is' is. If
the—if he—if 'is' means is and never has been, that is
not—that is one thing. If it means there is none, that
was a completely true statement.
The conversion ratio between Rumsfelds and Clinton-Lewinskies is 1 Rumsfeld = 1 Mega-Clinton-Lewinsky.
Knowledge isn't the only thing required to be successful at something, it doesn't account for skills. When I say skills I mean something that requires training over time to be able to do.
I know I don't know how to balance on a rope, if I learned successful methods for balancing on a rope I still probably wouldn't be able to do it. I wouldn't be successful until I put in the hours to dig those deep trenches between feedback and response. I might know how to finger a chord on a guitar, kick a soccer ball or make a compelling argument but I probably won't be very good at those things until I practice them
Maybe my interpretation of the word 'know' is too narrow but it seems like these 'learned skills' would fall under a seperate category of things that you can only get better at with experience.
For this reason, I've come to love the phrase "developer". I am someone who develops, and keeps developing. I am not a fixed point. And this is why you hire me. [See Fixed Mindset vs. Growth Mindset]
But of course Steve's analysis is easier to apprehend.
Reminds me of Taleb's concept of epistemic arrogance, which is more prevalant than not among educated people in my experience.
never underestimate the vastness of what we don't know that we don't know.
No. I have to know years worth of mountains worth of knowledge and skills to get even the slightest nod from anybody. Most software people don't give away praise for free.
Do you feel like a fraud...?
No. I'm a good programmer, my job is to program things, and my employers seem to have agreed. Like many people here, I've made lots of shit that works, and made lots of customers happy.
Shit you don't know you don't know only matters when it matters. And when it starts to matter, it moves into the shit you know you don't know category, and hopefully with some effort into the shit you know category.
A single human being can't know much, so out of necessity we act like a cache. We also tend to work in teams. I frequently encounter problems I can't solve, that a 3-minute back-and-forth discussion with the two nearest coworkers does solve.