Does Irrationality Fuel Innovation?
juliagalef.com
juliagalef.com
You probably want to aim for utility maximisation within acceptable risk bounds.
Our modern world is more complicated than what our brains were used to. It's no longer just social issues and immediate survival in small communities. We aren't used to dealing with things too small to see with your eye, and distances too large to walk. We aren't used to a world when most rewards cannot be attributed to a particular person's work. We aren't used to a world, where the main threat to your well-being isn't bandits or wild animals, but the constant barrage of salesmen and advertisers, utilizing hundreds of years of research to trick you into doing something stupid. We aren't used to a world, where almost all relationships are transient.
For the last point, consider this comment. I'm an Internet stranger. You've probably read my other comments in the past, you'll probably read some more in the future. But quite likely, you won't connect all of them together and attribute to a single person in your head; you'll see each as a piece written by a different Internet stranger, next to all other comments here. Every comment is thus a momentary interaction without prior context, and which doesn't give you any context for future interactions. The same is true for most news you read, and most social media comments you see. This is not exactly useful for our intuitions, which are optimized for processing actual relationships with people. So I think it may be helpful to study rational thought in order to calibrate your intuitions for these new times.
The reason is that we operate under deeply unscientific environments like partial-information, uncertainty and unpredictability.
On the contrary, we developed tool of thoughts for working in environments of vast uncertainty. Consider the battlefield, for example, commanders must make decisions in the fog of war. They are certainly interested in trying to make the winning decisions to achieve their strategic or tactical goals.
Technically you can try to be always rational in your life and play the 'risk minimization' game, but you will probably have a very mediocre life with limited success.
If a mediocre life isn't what you're looking for, then it isn't very rational to make choices that result in a boring life.
A focus on eliminating cognitive biases seems like it should lead us to the right solutions, but outside of certain academic fields, it often doesn't because the focus is more on not having incorrect thoughts instead of doing something, even if that something is wrong (at first). This means a mastery of all the material on "LessWrong" likely isn't going to make you better at innovating.
I don't think rationality is overvalued so much as "epistemic rationality" is overvalued.
It's clear that there is far more information processing occurring in our brains than we are ever consciously aware of. We end up being conscious only of the final result and we tend to call this "intuition". It would be a mistake to dismiss using intuition as irrational just because we aren't conscious of the mental activities that generated it. Of course intuition isn't necessarily right, it needs to be consciously evaluated and we should also use our conscious minds to train our intuition to increase its accuracy.
“The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.”
Irrationality is just an extreme form of being unreasonable, is it?
To get back to the article, yeah, some irrationality is probably necessary, because it takes irrational (unreasonable) people believing they can do things other people gave up on. But the important thing is probably being aware that you can be unreasonable and not letting that drive. There's a book, Becoming Steve Jobs, that is probably my favorite Jobs bio, because it's really about him learning to mitigate those flaws and becoming better in his second go around because of it. Steve was certainly delusional at times, and he shouldn't have been encouraged. You know, like denying the paternity of your child. Steve really personally hurt himself and others because of that kind of irrationality and it wasn't a positive in that context.
I think there are a lot of other people that do some really kind of delusional stuff, but we accept that because they are successful. You know, like Musk slurring an innocent person (or being really premature on every self-imposed deadline, or Holmes being a complete fraud. I've got no doubt that given enough time the latter probably would have figured out enough to pull half of a rabbit out of a hat to prove to people they had it all figured out, but it doesn't mean they weren't delusional, and it doesn't mean that that's healthy for them or us as societies to encourage lying to yourself and others, even if sometimes you get lucky.
Guess I'm just complaining. I see genuinely crazy stuff posted on HN and other places and to me irrationality isn't sexy, it's something that we need to do a better job of being honest about.
"The distance between insanity and genius is measured only by success." - Bruce Feirstein.
The only thing that "unreasonable" means is that when you tried to convince someone they should stop a certain course of action, you could not convince them and they did not refute you using logic. Mostly, the term is used as a polemic device to point out people whose axioms disagree with your own.
A lot of people simply have faith in their abilities, and a priori argumentation is not even a valid way of testing whether that faith is misguided. They are irrational because Empiricism is the opposite of Rationalism.
That could be a bubble that throws off so much surplus money that some of it gets spent on things that end up "failing" but generating a return in the long term. It could also be large government R&D expenditures, and that's indeed what it was through the most innovative period of the 20th century. Lastly it could be a massive cash cow monopoly like Ma Bell and its Bell Labs, but that usually overlaps a bit with government since usually those kinds of monopolies exist because of some form of government action in the marketplace.
All work happens because of this essentially. A result of over-zealousness. Irrational thoughts that keep you going that extra mile at 2 am because you really believe in an idea (more than it ought to be believed many times). If people didn't feel irrationally strong about things, then nothing meaningful (sometimes low probability of success, but high expected value as the author mentions) would get done.
To not have group think is considered 'irrationality' since most people don't think the person is correct. So perhaps you are playing with words.
I wouldn't have thought a large company just ignoring local laws could use this as a driving tactic to win a world market.
But I think that was my group think over their irrationality.
Life evolves by doing random mutations, and throwing away everything that fails hard enough. No analysis is performed at all, and reams of completely pointless changes are tried.
This process has produced the most innovative and advanced chemical, mechanical, and information-processing devices known to date.
The problem of course is that the random-walk approach immensely wasteful. In our daily lives, we strive to somehow less waste, and tend to avoid being an unfortunate random mutation destined to fail. Here genetic evolution gives way to memetic evolution.
But the memetic evolution has many of the same properties: many "random" things are tried, and a number of most spectacular successes are results of lucky coincidences and following patently wrong initial ideas. Of course, most of such crazy ideas just fail, like in biological evolution.
I would compare innovation to stock market: while you can sort of predict some local movements with a chance better than rolling a dice, you still face a fundamentally random process, and it's rational to approach it like one. For one thing, it means to accept its unpredictability and the limited usefulness of any rational models. Following a hunch may be as good a strategy as careful analysis. Probably a mix of both would give the best results; it looks like what some famous innovators did.
I suspect many people have deep drives of this nature, featuring multiple values - in this case, being daring (a matter of character) and benefitting others (a matter of welfare).
It would be irrational in such case to pursue a project: (1) that isn’t daring, (2) without regard to whether it’s daring, (3) with exclusive regard to whether it’s daring, (4) that doesn’t plausibly benefit others, (5) without regard to whether it plausibly benefits others, or (6) with exclusive regard to whether it plausibly benefits others.
3 and 5, and 2 and 6, are corollaries.
Is irrationality necessary or helpful to fuel innovation? Of course depends on what we mean by "rationality" and what value(s) rationality is meant to serve. My answer would be "no".
And I think that's part of the game. How much leg do you need to reveal to get the signature?
They've all been worthwhile projects in the end, however.
The other side of that coin is the fact that on the technical knowledge side, if they know the details they have to consider, there's a kind of hubris that blinds them to the actual effort required to cater to those details. Not maliciously, but just by the fact it's considered a known rather than unknown it just may not get the pre-project attention necessary to determine time and effort required, and so it gets under-estimated.
I think what I'm trying to say is that the tension between good, fast, and cheap is the fundamental reason, in my experience. Management tends to want all three, and so that's what's catered to to get the thing kicked off.
Yes, it's irrational. Almost by necessity.
Innovation deal with unknowns.
The unknown may be known, thus become rational!
Embrace the incompetence to expell it.
I think startup founders are irrational, crazy optimistic, because the odds of making it compared to the effort put in are completely off. But they get to do what they enjoy, perhaps for a while. This is possibly even more true for the employees in such companies.
Rational people go into corporations, and get an MBA, which is at its core a course in (economic) rationality. And the result of this rationality is bottom line is everything, even if it means to suck the soul out of everything or even shrinking the company, or even destroying the Earth.
I admit, I despise (economic) rationality. Rationality is only ever reactive, it cannot give you a goal, or a vision. It cannot give society any values. The values or goals have to be irrational, by definition.
There are some nice examples from game theory underlying this, but probably the best is Varoufakis' story about students of economics and business he taught.
When Yanis came to his game theory class, he gave a proposal to students, instead of them having to work and getting a mark for the course based on their knowledge, the students would just secretly vote on what mark they should get and they would all get the minimal mark anybody voted for. But the proposal would only be implemented if all the students agree. Inevitably, there were some people against this. Later Yanis asked these students in private, why did they vote against the proposal, they could certainly save themselves a lot of effort! They always, and without any hint of irony, told him that they believe there would be some idiot who would irrationally put in a bad mark. The irony being, of course, it was them, who was irrational!
I think this story beautifully underlies the futility of the quest for ultimate rationality.
And I find it hilarious that while free market ideology touts rationality as the ultimate goal, the reality of capitalism is that it actually requires people to create startups and small businesses, which is inherently irrational behavior.
It sounds to me like your problem is other people don't share your values, and you are calling this "rationality", whereas other people have the same issue, but often call it "irrationality". Yet you are saying at the same time that rationality is orthogonal to values. If so, then it doesn't oppose them. It all seems incoherent yet cliched to me.
But I think the paradox is deeper, and it ultimately means that we cannot really define "rationality", it is contradictory. And I think that's what economists are trying to do (and what they teach MBAs), to make a theory of how to make the best decision, regardless of values. The paradox seems to be that you cannot follow any theory like that blindly.
Perhaps it is cliched, I don't know. I wish Yanis would write a popular book about the paradox (he is certainly fond of it, see his book Economic Indeterminacy).
If "rational people go into corporations" then it can't be the case rationality serves other values (as you also note) because in that case we'd need to know the employee's values.
"But they get to do what they enjoy, perhaps for a while." <-- Then pursuing a startup is perfectly rational if you prize doing what you enjoy. I don't think you'd disagree, just noting that multiple meanings we ascribe to "rational" complicates discussions like this.
Money and free market (and economic rationality) is our current best shot at defining rationality (what makes sense to do) through a mechanism (algorithm).
We can talk about utilons instead of money. But then any theory of decision making is worthless, because we cannot apply it to groups of people, they have different units! It pushes the problem one level away.
Don't take this thread the wrong way, I like rationality, and I like Julia's writing on the topic. But it seems to have a fundamental limit, so there is a bit of irrationality always required.