H5N1
blog.samaltman.com
blog.samaltman.com
Sam is a smart guy, so I really don't want to come off as sounding like a jerk here, but this grossly underestimates the technical feasibility of creating such a virus. Computer folks routinely overestimate how much biologists actually know about the systems we study. We know jack about how the vast majority of biology works. We have the most fleeting glimpses of understanding that are regularly crushed by the complexity of dynamic systems with nested feedback loops and multiple semi-overlapping redundancies. I won't say it's impossible, but we don't even know enough to know whether the three things: high mortality, long incubation, and ease of transmission are even possible. While we can imagine it, there might be biological and epidemiological factors that prevent such a thing from existing.
This also commits the logical fallacy of ascribing superpowers to the bad guys cooking up viruses while assuming the good guys are sitting on their duffs letting bad things happen. H5N1 was a pretty good example of international collaboration. There were academic competitors and industrial labs working around the clock collaboratively on it in the early days before much was known. Whole vaccine divisions at pharmas were all over it. If we're instead talking about a mythical time in the future when we do understand enough biology to engineer something like this, one would have to assume the good guys possess the knowledge to develop countermeasures.
I'm not arguing that pandemics aren't something we should worry about. Europeans were almost wiped out by the plague and in modern times Africa has been decimated by HIV. These are real problems that the human race has faced and will likely face again, irrespective of lab-created stuff. Biotechnology is the primary mechanism by which we're going to be able to survive when the next one comes, wherever it comes from.
EDIT: Fixed wrong word usage in 2nd sentence.
Sam is a smart guy, so I really don't want to come off
as sounding like a jerk here, but this grossly
underestimates the technical feasibility of creating
such a virus.
That being said, "Don't Panic (tm)". The likelihood of any of these scenarios is extremely extremely low and people have been thinking about and preparing for them for decades.
As for what is much more likely to happen with naturally occurring viruses like the H7N9, natural variants of the H5N1, etc... see my comment from a few days ago [3].
Disclaimer: I'm the first author of [1] and a collaborator of the two first authors of [2].
[1] http://www.nature.com/srep/2013/130717/srep00810/full/srep00... [2] http://www.biomedcentral.com/1741-7015/11/252 [3] https://news.ycombinator.com/item?id=6839147
BUT, more importantly--you don't even need fancy biotech to engineer a terrible virus. No sequencing, recombinant DNA, fancy BSL-4 labs, well-educated virologists, none of that. All you need is a captive population, something any self-respecting evil warlord should be able to get. (Certainly the North Koreans, the Taliban, etc. have access to plenty of prisoners.)
The same simple mechanism used by Fouchier (the guy who created the controversial bird/ferret superflu) can be applied to humans: serial passage. Put any moderately bad flu virus in a certain number of prisoners, then expose them via air circulation to other prisoners. Take the sickest people from the second group and expose them to another uninfected group via air circulation. After five or six passages the virus in the last group will show extraordinary virulence and transmissive capabilities because you've applied artificial selection.
To seed the virus and start an attack, you take a few of these prisoners, expose them to the worst virus, and put them on planes to your target country while they are still in the incubation/transmissive period.
Obviously, this is a nightmare scenario that I hope never happens, but the idea that terrorists or evil dictators need fancy science to engineer superbugs is false. The same methods farmers have used for centuries to grow taller corn and leafier lettuce can be applied to viruses by anybody with enough prisoners and moral depravity.
[1]: It should be noted that in-house synthesizing costs are coming down, though, and we won't be able to rely on the safeguard of companies automatically BLASTing ordered sequences against a CDC blacklist for much longer.
Can you comment on the models that are used for simulating outbreaks? Do they factor in natural evolutionary pressure and change of the virus? In the case of smallpox, you have something fairly stable, but influenza is highly variable over time. I can't imagine how one does that but it would be cool to know if it's possible.
Do they factor in natural evolutionary pressure and change of the virus?
If you're not trying to create a "supervirus," but just shotgun a series of viruses to create maximum health disruption, then (as you know!) it's already within our technical grasp. Expensive, time-consuming, and probably failing 90% of the time, but accessible nonetheless. Considering how bad our hit rate is on the trivalents, I could see a plague of multiple, high-lethality strains with a wide variety of hemagglutinin and neuramidase antigenic shifts as a plausible bioterror scenario.
It's not a doomsday scenario, since we have such good reporting and analysis infrastructure, and much of the heavy engineering on the problem (like alternate vaccine production methods) was done during the earlier bird flu outbreaks, but it would be a very nasty kind of terror attack.
One point that I didn't have time to make was that high mortality is not generally evolutionarily advantageous. Even if you cooked up a strain that was especially nasty, it could take considerable effort to prevent it from mutating in the wild into something less so, since the mutated virus would have a survival advantage of not killing its host. Unlike machines, a creator can't really control what happens to a biological system in the wild as it interacts with the environment. In the scenario you describe, there is considerable uncertainty as to whether it would actually spread as engineered.
This is the crux of what I've clearly done a poor job of saying: we know so little that all these scenarios still rely on incredible amounts of luck more than technology.
Not terribly reassuring :) Perhaps 1918 wasn't a very high-fitness virus, but it still got ~5% of us before burning out.
That leaves the threat level >> (conventional) terrorism.
>If we're instead talking about a mythical time in the future when we do understand enough biology to engineer something like this, one would have to assume the good guys possess the knowledge to develop countermeasures.
This is naive.
Understanding of weapons != Knowledge of countermeasures (since were talking about logical fallacy)
It's a weapon that has no counter measures. None that I'm aware of, except prohibition.
You can say instead: "underestimates the difficulty" or "overestimates the feasibility."
The rest of your argument is very far fetched. One person's difficulty formulating a sentence could not be less related to biologists' collective capabilities in tailoring viruses.
I have apparently underestimated my ability to craft a joke.
Nuclear weapons have been around for over 50 yeras. We do not yet have __in place__ any ability (other than treaties and fear) to prevent a nuclear holocaust. Missile shields, "Star Wars" -- all of those are of questionable capability, and none of them are deployed.
Given that our only way of preventing nuclear winter is to agree not to launch (and go to war to prevent Bad Guys from getting them?), an option which is not available when dealing with a disease, I'm not optimistic about our future ability to prevent a superbug from wiping out humans.
But that's not the point I think Sam was making. I read the article as discussing a purposeful designing of a virus with specific properties. My contention is that the knowledge required to engineer something that can evade the immune system, spread easily, and cause high mortality is very likely the same mechanistic knowledge that would help you to defeat such a virus.
Remote sensing is what held off nuclear holocaust during the Cold War. The ability to reliably and quickly detect and respond to nuclear first strike creates the "mutually assured destruction" strategic framework aka deterrence.
At this point in time I'd still say that we only have the ability to reintroduce old disappeared pathogens. We do not actually have the ability to design new effective viruses.
Unlike the seasonal flu shot, this vaccine is tailored to the emergent strain.
Uh, nope. What does "almost wiped out" mean to you? 30% of population dead is "almost wiped out"? It's horrible, of course, but there were still 70% who stayed alive. Well, I could say that I can jump almost three meters high with that logic. :-)
We would be knocked back to the ... 1970s!?!?
In the 1970s the world's population was about 3.5 billion and today it is double that. Sure it would be very disruptive, but there would still be plenty of creative people around to pick up the pieces.
If society breaks down then you'll suddenly be looking at countries being unable to support anything close to that number of people.
Grab a copy of The White Plague. It's an entertaining read.
Imagine the genetically-engineered counterpart to the CryptoLocker ransomware. Private and public antivirus research. MacAffee for your white blood cells. Imagine ad-supported biotech.
The internet has created a sort of commoditization of fraud. Scam spam, ad-ware, spyware, etc, often run by companies operating right out in public. Professionals openly discuss security from both sides of the line.
Imagine this sort of bizarre professional attitude in the a low-cost engineered-virus biotech field. What would the Windows of this world be - a suite of programmable bacteria with a solid API?
Don't you dare.
That pressuposes that the "good guys" would be the victims here. How about the opposite?
What about regular "democratic" superpowers doing it to poorer countries they want to control, as they have done similar things all along the colonial and post-colonial era?
The way dictatorships in Latin America and Middle East got weapons, supplies and a helping hand from the US for example against their people or neighborhooding countries.
It's the other way around. The "good guys" are cooking up viruses and publishing them as science (in this case at least)
Proof of evolution
[...]
> If we're instead talking about a mythical time in the future when we do understand enough biology to engineer something like this, one would have to assume the good guys possess the knowledge to develop countermeasures.
------------------------
The amount you need to destroy is more or less a constant. When the efficacy of your technologies is limited, then a technology that only gives you a small percentage edge over your opponent's technology - say Iron vs Bronze - is survivable for the defender provided they have an edge in some other area, though not necessarily pleasant. However, as the efficacy of technology increases, you only need a small percentage edge over your opponent in terms of relative efficacy of technologies to have more than enough power to destroy all that you need to in order to remove them forever.
Consider that armies could be separated by hundreds of years of technology in the past, and still fight on a roughly equal footing. Technology did not more very fast, nor was it very powerful. Then imagine what an army of today would do to an army of a hundred, or even fifty years ago. In the Iran-Iraq war two armies with Cold-War level technology faced each other off for eight years. The Iraqi army were, however, swept aside very quickly by a more advanced force.
The timespan in which there's a rough parity in power shortening. Even small difference in development with respect to time can rapidly become insurmountable when you're dealing with a high rate of change and powerful technologies. What you're defending is more or less static: people, land, resources, they're not getting any more durable, while weapons are always becoming more powerful.
You only need one world ending plague. The defender has to be on top every time, the attacker only has to surpass them once. There is no chance to adapt to what they make, or to try again any more than biological evolution can adapt people to a bullet in the head - because any minor adaptation in that direction makes no difference when compared to the sorts of forces that are imparted.
And I don't think we should have much confidence in the idea that the defender is going to be on top every time.
The questions seem to me to be ones of whether a reluctance to destroy the world is characteristic of organised systems, and whether organised systems will always have the edge over individual effort. If we get to a level where someone can create a suitably devastating bio-weapon in their garden shed, will we also be in a position where that's effectively analogous to creating any other outdated weapon in your shed?
I don't know, computer viruses don't lead me to much hope on that point. Enormous energies are being expended to restrain the energies of a few, with no clear victory in sight. The playing field is not always slanted the way we might like.
Preface: What we're talking about is probably biochemically impossible (truly no bad side effects, no tolerance, etc.). So, everything that follows is a fun thought experiment, and should be taken as nothing more.
Let's say someone produces a true wonder drug that is relatively easy to produce and produces extreme happiness 100% of the time, with no side-effects, and no diminishing returns due to drug tolerance. This drug produces more happiness than any other activity that we could be pursuing with our time. As a result, all anybody wants to do is take this drug all day.
The author presumes that this is a bad thing, but let's question that assumption.
If everyone is completely happy 100% of the time, and - more importantly - happier than they would be if they were doing whatever it is they would be doing with that extra ambition, why should we assume that this is a bad thing?
Of course, somebody would need to maintain production of the drug. This means that people either would take it only part of the time, to maintain enough ambition, etc. to produce the drug on their own, or (more likely) we would have some lucky people who take it all the time and are always happy, and a few people who are tasked with producing all the joy for the rest of the world.
This exact premise (the second version) has already been explored, in short story form. http://en.wikipedia.org/wiki/The_Ones_Who_Walk_Away_from_Ome...
(I agree that this situation sounds bad - most people would have a negative emotional reaction to it, but it's fun to explore why we have an aversion to the thought of pure, unmoderated happiness.)
The negative consequence there is that often times we don't learn about ourselves until we attempt to make such a change. If you hate debugging Java, and you can take a pill that makes you not mind it, you might never discover that you could actually be a much better Ruby programmer than you ever were a Java programmer -- or that you really were never cut out to be a programmer at all and should pursue something totally different. You'd just chug along debugging Java your whole life and never realizing your full potential.
> If you gave me a perfect happiness drug, I wouldn't want to sit in bed all day and take it. I would want to take it and go about my normal day only without the burden of misery
I've been reading some meditation related books lately, and this was one of the ideas they were trying to get across. That meditating for a sense of fulfillment doesn't replace your desire to go about and do good work, it merely changes your perspective on how you feel while doing those things. Additionally, it (allegedly, convincingly argued) enhances your ability to do the things you like well, and to make better choices in general.
> I suspect this is just because many of our current drugs do this as a side effect of inducing euphoria, and we're imagining that instead of a real happiness drug.
A useful way to contrast that is with uppers. I took my friends aderall a few times in College - in addition to staying awake, I felt profoundly happy - yet I didn't give up my goals (study all night) - I merely had a good time doing them.
I know there's some science around parts of that - far too lazy atm to search and link it (sorry!). But perhaps it is a bit counter-intuitive that feeling deeply happy and satisfied could also drive a person to do more meaningful work at the same time.
If there were a beverage that worked like coffee except was more MDMA like, that our physiology could sustain in daily use, I dare say a lot of us would drink it.
Reason is a poor way of estimating behavior, in special, your own.
This doesn't compare with anything a normal person experiences during life. There's no rational expectation to be had of how anyone's behavior would be if such a drug existed.
Sure, once you started the drug, you wouldn't care, you'd be hooked, but why assume everyone would take it in the first place? There's no chance I would try a drug that I had observed make someone just sit in their bed doing nothing all day, even if it was clear that they were ecstatically happy beyond anything I had ever experienced. In reality, I think very few people would take it.
I guess you could still go off and live in the woods with those that do not ascribe to a drugged life, and build a separate society. But even that still means the original society is doomed.
Pursuing pleasure as an end goal rarely leads to pleasure. A lot of addictions seem to stem from the pursuit of pleasure as an end. We treat people for doing something that makes them "happy" to the exception of all else (whether that be gambling, alcohol, drugs, etc.). I honestly think if there was a happiness pill we would consider treating people for taking it.
When I talk about doing something, I'm talking about what you do with what life throws at you. Everyone has different circumstances, but what is common is the ability to choose how we respond to those circumstances. Whether we make something (whatever that means to us) of what we are given or not.
Taking a happiness drug and checking out from the rest of life is deciding to do exactly nothing with what life has given us. And I honestly don't think most people would want that.
I'll end with a quote from Viktor Frankl's excellent book Man's Search For Meaning:
"By declaring that man is responsible and must actualize the potential meaning of his life, I wish to stress that the true meaning of life is to be discovered in the world rather than within man or his own psyche, as though it were a closed system. I have termed this constitutive characteristic "the self-transcendence of human existence." It denotes the fact that being human always points, and is directed, to something or someone, other than oneself--be it a meaning to fulfill or another human being to encounter. The more one forgets himself--by giving himself to a cause to serve or another person to love--the more human he is and the more he actualizes himself. What is called self-actualization is not an attainable aim at all, for the simple reason that the more one would strive for it, the more he would miss it. In other words, self-actualization is possible only as a side-effect of self-transcendence."
I am sure I have butchered my own thoughts, but hopefully this helps make some sense of what I'm trying to say.
There are of course many who don't subscribe to these traits (at least not consciously). I believe most of them are subject to cultural belief systems that promote social cohesion and cooperation: the nobility (and intellectualism) of altruism, the teachings of compassion by many religions, the promise of a release from guilt by donating to charity.
But if society were certainly stable, then what is there for one to actualize toward? Perhaps it would be that other engine that propels us: Curiosity. But isn't the pursuit of one's curiosities primarily selfish?
Beyond that, there is creating art, raising children, forging and strengthening romantic bonds (it's very difficult to phrase the pursuit of love in way that emphasizes its potential other-centeredness), making people laugh, advancing knowledge in some field. All these things still have meaning in a stable society.
My term for this is "the want-like distinction", and the best reference is Yvain's "Are Wireheads Happy"? http://lesswrong.com/lw/1lb/are_wireheads_happy/
The question of why healthy people tend to be opposed to lotus-eating is also not difficult to answer, especially given that what TV Tropes calls the "Lotus-Eater Machine" has been constructed several times in reality, and its addictive nature and harmful effects well documented [1]. Both in individual life and on the scale of societies, what we typically call "civilization" is nothing more or less than an unending war against entropy; this is frequently, if imprecisely, formalized in the common dictum that "it's easier to destroy than to create" -- and easier, too, to sit in one's own filth than otherwise, in the absence of some motivation to do otherwise. Unhappiness serves to provide this motivation, and it is therefore precisely that motivation which the lotus-eater machine removes. After all, if sitting in your own filth doesn't make you unhappy, because you have a wire in your brain which ensures sublime delight no matter your circumstances, why bother to do anything about it? -- and, as we see from one of the cases described in the page I linked, indeed you will not.
Consider, too, that the English language already has a well-known and clearly defined word for the concept of "pure, unmoderated happiness". That word is heroin, and while I recognize the modern unpopularity of the proposition among those who've never seen firsthand what it does to its unfortunate devotees, I maintain most firmly that that damned drug is illegal for excellent reason.
[1]: http://mindhacks.com/2008/09/16/erotic-self-stimulation-and-...
Right, but... why is sitting in your own filth objectively "bad" if you're happy? There could be negative externalities (e.g. maybe your unsanitary conditions are facilitating the spread of disease to others) but I don't think it's clear what about the filth itself is objectively bad.
The 'negative' emotions are very important for the functioning of any animal or mind - they are the drivers that motivate us to do stuff instead of happily dying.
I could write a long essay on how pain, fear, frustration, boredom and burnout are totally neccessary to implement even for a simple non-human artificial autonomous agent, in order to prevent it to be stuck on 'broken' loops forever - TL;DR is pain is needed to ensure that the 'mind' avoids damage; fear is needed to ensure that the 'mind' avoids serious risk of damage; frustration is needed to ensure that the brain doesn't try the same thing over and over again if it doesn't achieve the goal; boredom is needed to ensure that the brain isn't stuck in repetitive loops unless they bring great rewards; and burnout is needed to ensure that the brain at least tries takes a chance at escaping poor conditions, instead of happily suffering through them to death.
Put a brain in unconditional happiness - and it will be so useless and vulnerable, that you'll soon find out why evolution (or designer, if you so fancy) implemented the conditions on happiness.
The other and perhaps different explanation, is that Philosophically, man is happy only when exercising his facilities. This has been said as either his 'reason' or his 'power' depending on your taste in philosophers. But in either case the notion of perfecting some craft-work. In particular, this should further some functional inter-relationship with the outside world (not to mention family/informla political skills). The idea one could be both 'vegetative' and 'happy' simultaneously tends to imply 'ok, but that's not a human life...that is a vegetable one', or something similar.
These are good arguments, in many ways. Enough so that the burden of proof should be on those putting for completely contrary ideas, eg: "why is sitting in your own filth objectively "bad" if you're happy."
IMHO, Its not the filth, or the sitting, but the lack of anything evidently positive. Sitting in filth might be, after all, a corralary to "getting shit done" with limited resources. For instance, sitting is rest from other work; and 'filth' is just debris that has yet to be cleared away or moved away from (the latter being imminent).
> But in either case the notion of perfecting some craft-work. In particular, this should further some functional inter-relationship with the outside world
> These are good arguments, in many ways. Enough so that the burden of proof should be on those putting for completely contrary ideas
I respectfully disagree. My instinct is to question your premise that "man is happy only when exercising his facilities." I'm assuming that this has roots in ancient Greek philosophy, and I'm not well versed enough in that area to refute the reasoning that leads to that premise, but I will say that I'm hesitant to accept it at face value. Even if scientific studies indicated that man is happy only when exercising his facilities, it would only indicate that exercising your facilities has instrumental value in providing us with satisfaction, not intrinsic value.
To me, this idea that productivity is an objective Good seems like a statement of personal values rather than anything fundamental to the Universe. Of course, personal values matter; I'm just trying to draw a line here between productivity as a means to an end vs. productivity as intrinsically necessary. If someone is able to achieve happiness without "exercising his facilities," I don't think we can assume that their happiness must be impoverished compared to those pursuing more mainstream paths, even if our gut instinct tells us otherwise.
And to reiterate: this is ignoring, for the sake of argument, the topic of negative externalities. Obviously, if unproductive behavior has a detrimental effect on others, the issue becomes far more complicated.
I guess it depends on how we define happiness.
In my view the perfect happiness drug would have the exact opposite effect, i.e. make people more functional, less prone to stress and anxiety.
Realizing happiness that is independent of conditions, would result in may result in diminished drive from activities that are driving by the seemingly insatiable ambition for greater perceived social status that is currently the normative state of the human race.
But that would be compensated because a greater portion of our drive coming from compassion and curiosity.
So more curiosity, and more solving of useful problems.
Some evidence of this can be had from contemplative activity can dismantle the mechanism in the brain the inhibits happiness without the aid of drugs*.
This sort of expertise is difficult and requires extended practice, so it can be confused with disengagement. Many often later go on to have considerable impact on human culture and values.
This is both subjectively verifiable, and objectively via brain scans one example outlined below. http://brainimaging.waisman.wisc.edu/press/NCCAMOct08.pdf
As well, your hormone receptors can't take hormones all the time. They take time to recover, otherwise they will go numb.
That's exactly what Meth does for instance, it releases all your happiness hormones at once, that's why you feel so suuuper amazing. However, once the drug subsides, you're super hungover and feel like shit, because you're all out of happiness hormones.
Even worse, even once you stop doing Meth or whatever, your receptors have already become numb, so events that made you happy before you used drugs, such as getting a raise, raising investment, getting married, don't do anything to you anymore.
If you stop these drugs your receptors will become more sensitive with time again, but that takes a couple of years and might never go back to normal. So, once you take drugs, you're happiness from certain events will always be lower than it was before.
That's why it is so hard to get off drugs.
Lets say Paul takes artificial hormones to increase his level of happiness; numbing his receptors. What stops Paul from also taking a drug which improves the recovery time of his receptors?
Edit: In a galaxy far, far away. Where E.T. comes from and science is rockin'.
I think what chimeracoder is trying to point out is our underlying assumptions about what individuals and humanity should be aspiring to. I'm somewhat of a nihilist, so I consider satisfaction with one's existence as the closest we can get to an "objective" good. I see "ambition" as a personal choice, something that provides satisfaction to those that desire it, but not necessarily a prerequisite for happiness.
I don't think chimeracoder is suggesting everyone should want to take a happiness drug. Rather, the question is: does it matter if someone does choose to live under the influence of such a drug? Would we consider their choice to be objectively "wrong", or just another way to live?
Eh, I don't know about that. Opiates are pretty close, and their most harmful side effect is the legal environment which proscribes insane punishments for involving oneself with them. Said another way, even if it's biochemically possible, it would likely be legally impossible for this happiness drug to be distributed at all. Best to point one's though-experiment apparatuses in that direction, I think.
It seemed that most answers mistakenly assumed that "Happiness" was something you can define for everyone, rather than something that individuals have to pursue for themselves.
Happiness is a catch-all term for various "good" feelings, such as joy, satisfaction, contentment, bliss, victory etc.
If any such a drug were to be developed, it would indeed sap ambition from many people. Even assuming it accounts for all existing relevant genetic variation, it wouldn't prevent genetic mutation. So those born with mutations that make the drug less effective will procreate with relative ease, eventually correcting for the drugs adverse effect on the survival of the species.
Of course if cultural (political?) forces also established that any genetic variation was bad, then we would indeed be in trouble. Why? With no threat of pain, and the absence of pleasure replaced by unlimited pleasure, what goals would exist? What would motivate action? Or given a relativistic perspective, what would just motivate procreation?
Philosophy has been thinking about this for a while.
In my opinion, happiness is like life, it evolves. Targeted short-term pleasure is possible through extraordinary drugs, maybe in the near future, but probably the brain re-levels. It doesn't approach a stasis. At least, not completely (though happiness gets increasingly more refined over human history, so what seems like happiness now may be this sweet spot in the future that is its own ballpark which we largely reside in and explore -- again, I think happiness gets more and more pinpointed, compared to say Neanderthal times, but it doesn't necessarily ever approach a constant).
Assuming the entire world can be maintained perfectly and automatically, nothing, but that's a bigger assumption than the hypothetical wonderdrug.
What happens when the Hoover Dam starts cracking? Or when a new disease is found and a new cure needs to be found? Or when we run out of a resource (oil, helium, it doesn't really matter which)?
We don't live in a world of static status quos, we live in a world of equilibria. Many of those equilibria factor in the ambition of individuals to maintain, to discover, and to solve problems.
Then we'll be happy about it
However, the adaptations we execute, as biological beings, don't really care about the health and welfare of their own far-future selves; they're more concerned about the Net Present Value of different choices they can make. So, one hour spent Really Happy, outweighs a year spent Just Happy, because that Really Happy is all received by your present self.
So we can do the math as rational beings, and as much as we want to be rational (which is itself a function of our biologically-trained impulses), we can look out for our future selves and keep the world running. Or we can accept our nature as natural beings, and wirehead. It's really the explicated meaning-of-life question, going forward.
I find making decisions that end up along lines of this really, really difficult. The rational part of my brain knows very well which one should I choose, but it has really, really hard time arguing with that more... I don't know, primal? part of me.
I think it's really fascinating, how relatively weak our conscious self is in arguments with our short term desires.
Exerting exactly this type of willpower (setting long-term goals before short-term rewards) is what "spends" dopamine in the brain. The more dopamine you have available, the longer-term you tend to plan. And when you run dry, you feel "restless" and want to do things "on a whim."
---
In the face of this, it's really quite fascinating how meaningless a term like "willpower" becomes. Doing what you want-to-want to do basically comes down to:
A. having enough dopamine in the first place (children should really get checked for ADD/ADHD at about the same time get checked for nearsightedness),
and B. making pre-commitments with strong consequences for reneging (e.g., losing money you've bet; or making you look low-status to people you care about.)
There's nothing else to it.
There is a reason we appear to be physicaly unable to experiment that kind of happiness this thread is talking about. The only question is what we'll choose once we are able to change our own brains to make it possible, but then, that's a completely unkown context, so we'll probably choose some option that we can't even imagine today.
Mass solar power, maybe?
Or perhaps, if everyone were perpetually swept away into the wonderland of the perfect happiness drug, the human body itself could be appropriated as a battery, leading to a deliberately created Matrix. </humor>
The thing is, if "being happy" is the only goal, then the rest of it is just an efficiency question.
I think people view robots as either empowering us or somehow getting into lethal combat with us as the dominant life form on the planet. I seriously doubt it will be either. They'll just give us what we want. The race will naturally die out within a few generations.
Should we focus on preventing terrorism? Well, if 9/11 was the worst case scenario then no. If on the other hand a terror attack could bring down the entire country it's certainly worth being paranoid about. Suppose terrorists poison our food and water supplies to the extent that we get country-wide food riots. A civilization is only 9 meals away from anarchy after all.
So the essential question is this:
- Is our civilization essentially fragile or fundamentally robust?
If our civilization is fundamentally robust we can simply focus on growth and deal with setbacks (global warming, terrorism, imperialism, wars) as they come. In the long term prosperity will go up and up. Not always as fast as we'd like and not always in ways we deem fair but if we keep making progress we'll get there eventually. This is the whiggish view.
The opposite view is that civilization is fragile. Kingdoms come and go and foolish decisions can and have lead to centuries of regression. The upward trend we've seen in the past couple of centuries does not mean our species has grown up in the slightest. Every new weapon of doom we discover we play with and we're no better than our imperialist and bloodthirsty forefathers. Our civilization is determined to self-destruct by either nuclear war, environmental disaster, political insanity or runaway capitalism. A civilization that is not capable of planning ahead will eventually walk like a lemming of a cliff. The best thing we can do is put tons of safeguards and regulations in place to improve our odds of surviving at all.
Those are the two main views. And the kicker is we don't have enough data to know for certain which view is correct.
'We' (our respective nation states) are our imperialist bloodthirsty fathers. What country do you live in?
This is also why, in the face of globalism, we should work to make life multi planetary [2].
And for people who think this is just silly, it might be a good idea to have a look at some recent history [3] and consider how close we were to being in a very, very different place. This is not science fiction.
Good piece.
1: https://en.wikipedia.org/wiki/Nassim_Nicholas_Taleb
But maybe there are some tail risks we should really worry about.
and indeed that's what much of Taleb's body of work is about. I am partway through Antifragile and do not find it as compelling as The Black Swan, however.
Ha, I did the same thing.
> I am partway through Antifragile and do not find it as compelling as The Black Swan, however.
Personally I think Antifragile is much better, in that it it's a more complete work that contains his previous books and irons out what the consequences are. It's possible you need to have a similar view on history, culture and modernity in place before you appreciate it - I was into stoicism, empiricism and classics in general before I read Taleb - as he's largely using that as "the other side of the barbell" in his explanations, with the first side being mathematical (see his technical online textbook). I've read The Black Swan once or maybe twice, whereas I am already on my third reading of the Antifragile (I can't help myself, it's too relevant).
It turns out that "mud daubers" (wasps that make houses from mud) are responsible for at least 2 major airline crashes in the last 33 years killing at least 223 people. http://en.wikipedia.org/wiki/Mud_dauber#Involvement_in_Flori...
One in 1980, and one in 1996... that we know of.
Apparently these mud daubers love living in long cylinders. If they find one, say on a plane's uncovered instruments, they'll set up shop.
If you look at it under the right light, mud daubers are approximately 1/20th as powerful and threatening as the world's terrorists of the last 33 years.
This URL has some numbers of terrorist caused deaths over a similar timeframe: http://en.wikipedia.org/wiki/Patterns_of_Global_Terrorism
> This species also brought down another plane in Washington during 1982.
Which, in addition to the crashes of the Florida Commuter Airlines flight (1980) and Birgenair flight (1996) makes it 3 planes mud daubers have brought down.
Exactly. We're not talking about rational agents anymore, it's about lowering the barrier of entry both financially and technologically so ordinary (arguably insane) people can become Bond villains on a shoestring budget. There are a lot of possible motivations why a person might do this, for example being a religious nutjob, or because date night took a turn for the worse.
Unfortunately, in reality, such attacks are likely to cause huge amounts of suffering which muddies the water.
But if there was a magic button to press that'd instantly eliminate all life on earth, I think it'd only be fair to push it.
And the memeset I'd personally put my money on being the proximal cause is "humans are a blight upon Mother Gaia and should be removed at all costs". We don't have to imagine the existence of people who would push the Kill All Humans button if given a chance... we need merely read the right bits of the Internet. Sure, most of them would at least think twice... but there's enough that wouldn't.
Or another way to put it, why is the pope-mobile bulletproof?
If you don't believe me, hear it from the horse's mouth. Here is a video from a Muslim peace conference in Norway: http://www.youtube.com/watch?v=bV710c1dgpU#t=45s It will take five minutes of your time, but I think it shows best how sincere these people are.
I'd imagine the threat of release would make for a pretty compelling bargaining chip. Potentially, you could gain a lot of leverage without having the backing of a sophisticated military-industrial complex (unlike nukes).
It's kind of like a DIY WMD for the millennial, "maker"-terrorist generation.
[1] http://www.cnn.com/2013/12/04/world/americas/mexico-radioact...
However, I don't think that it is outside the realm of possibility that relatively small entities, on the order of a small-cap corporation or private laboratory, could eventually create a weapon that has far more global destructive power than a radioactive dirty bomb.
This would afford the entity more leverage at a global bargaining table- as opposed to a negotiation with a single government. Combined with the relative anonymity one may be able to maintain when releasing the virus, these entities might see a higher expected value in engineering a virus.
It would be "DIY" in the sense that you wouldn't need the backing of a large nation state to produce an ICBM, or opportunistically steal some radioactive material to create a dirty bomb.
Also, a virus is implicitly cheaper to distribute: with a longer incubation period and an exponentially increasing rate of propagation, a virus could reach critical mass before there's time to react.
In a sense, this makes them more effective than other types of WMD- which may have lower singular chances of successful detonation, have high per unit capital costs, tend to have local area effects only, and are generally difficult to produce far removed from military industry at scale.
Personally, I think the world is more likely to end with a bug than a bang. I honestly don't know which is more terrifying to me.
The risk comes from widening access to a given technology, and the amount of harm it could do. It's hard to calculate the risks, but bio weapons potentially have wide access and great harm.
And "bad guys" are not necessarily required for a global catastrophe. Human error can do the job just as well.
That said, I think biotech is an amazing force for good in the world, we just need to be responsible with how we deal with it. Which means defining and justly implementing 'responsible' is a core tech challenge.
With bioweapons, the perpetrator is much harder to track down. And if you don't know who your attacker is, you can't retaliate.
Take this researcher, Bruce Ivins:
When analyzing risk, it's important to estimate costs as accurately as you can. Unfortunately Sam missed the boat on this one.
Sam's saying "let's spend X amount of that $100 billion on preventing biological disasters, rather than just terrorism".
I think Sam's getting at something important but it would be best to first start with estimating the costs as accurately as possible. As it stands Sam's estimates to the cost of terrorism are off by several orders of magnitude, so you can understand why I pointed that out.
That's not to say that things won't change, but if I had to pick a serious biological threat that exists right now, it would be antibiotic resistance. Thanks to air travel and long incubation times, we're not that far away from a global pandemic of multi-drug-resistant TB, yet almost nobody is talking about it.
Depending on how you define "novel", it could indeed be a "grad student project", or it could be a paper that deserves a Nobel. But it still falls firmly in the "improbable as a weapon" category of threats. I'm still far more scared of XDR TB than hypothetical synthetic viruses.
"Also in 2011, some researchers figured out how to reengineer H5N1—avian influenza virus—to make it much scarier by causing five mutations at the same time that all together made the virus both easy to spread and quite lethal"
This is a little dismissive. Nuclear war under any plausible scenario wouldn't be an isolated 1945-type event. It would be a global event that drew in other players and more than likely would conclude in a mass launch by one of the world powers. We're not nuking Paris, Moscow, or DC and walking away. There will be retaliation.
Unlike biotech, these things are here, ready, and primed to hit targets. If there's a tail risk to worry about its human extinction via nuclear arms.
See http://www.nickbostrom.com/existential/risks.html for a nice summary of potential existential risks to humanity.
Before we start to accusing others as 'terrorists', first step back and think: Why someone would sacrifice their lives to attack us? They are crazy people, some may say. But why we suddenly see so many 'crazy' people in the recent years? What caused them to be so desperate and so angry to the extend they want to waste their lives to do damages?
There are countless ways to cause mass damages in the modern societies, and unless we can understand the _rootcause_ of the attackers' motivation, trying to seal off all potential attack vector is no more effective than trying to remove weeds from ground without taking off the roots.
Disagree. Of course it's certainly part of the answer.
If not then why not publish all the details of when you are home and how your house is protected for anyone to see? And exploit if the appropriate "nut" decides to? Some walls are helpful as a barrier.
Security (by obscurity?) does provide some protection. Locks do keep some people out. Going in the other direction (and making it easy for someone and very available) is not a solution to making things safer.
Drug companies are no longer doing research on new antibiotics, because antibiotics actually cure things and hence are not profitable. Today's drug companies are only interested in treatments that last a lifetime. Pfizer was the last drug company doing research into new antibiotics, and they closed that division because it wasn't (and wasn't going to be) profitable.
I think the fact that drug companies and medicine in general are not focused on important problems, but only on profitable problems, is a major issue we should be addressing. See the recent Frontline documentary "Hunting the Nightmare Bacteria"
http://www.pbs.org/wgbh/pages/frontline/hunting-the-nightmar...
Minor quibble: I've never thought (most) people actually "fear" terrorism, rather they have (justified) anger and perhaps an excess feeling of "something should be done" since terrorists are actual human beings who can be brought to account.
Perhaps a better comparison would be fearing airplanes over cars, but there I think much of the fear is in the novelty of the flying experience.
The comparison with nuclear weapons is really interesting, particularly as the response to fear of nuclear annihilation on the part of most people was overblown (digging bomb shelters under houses etc). On the other hand, the actions of the relevant governments (US/USSR) were mostly rational in a game-theoretic sense (acknowledging the prisoner's dilemma at hand).
Biotech may be different, as Sam mentioned, since only nation-states have the means to build nukes. On the other hand, computers/the internet still work despite Y2K and rtm's worm :)
In a sense though, this notion that technology can give individuals unimaginable power has also been a theme with Robert Heinlein (I think), where he wrote about an easy-to-manufacture superweapon which basically gave anyone the ability to destroy the world. It was also a theme in Kurt Vonnegut's "Cat Cradle" although that's a little bit more of a stretch.
It's true that our destructive capability scales with the amount of energy that we can harness. And it's also true that biological threats are under-perceived. But it seems to be that biology is inherently messy to avoid being too susceptible to annhiliation. That is, yes, it may be possible to create a plague that wipes out 50% of humans - but in the scheme of things, that's not the end of the world. Certainly not the end of humans (not even close!). And it seems that the likelihood of creating such a pathogen is exceedingly small. Indeed, I'd estimate that you couldn't kill more than 10% of people with a single plague.
But since we're talking about catastrophe, it's interesting to wonder about whether a single human could end the world, and if so how. The most likely way (and rather dramatic way) would be to maneuver a large asteroid to impact the Earth. Energetically, it's entirely possible to do. As for nuclear war, I'm not entirely convinced that would really be the end for humans - although it's certainly possible. Another way to end the world might be to release mega-tons of CFCs into the upper atmosphere, intentionally destroying the ozone layer. I can't think of any other scenarios!
Why is it bad to "try to keep things secret" but good to "spend a lot on proactive defense"?
How would that apply to hydrogen bombs? Should we open source the specific details on how to engineer a maximally efficient hydrogen bomb from the most accessible materials, and just spend a lot on hydrogen bomb defense? (Which is what exactly?)
When you omit support for conclusions, it implies you think the reasons are obvious. But it is not obvious that we should do away with efforts at secrecy around hydrogen bomb tech. Nor is it obvious we could defend ourselves from widely available thermonuclear bombs by being "proactive". It's a hand-wavey answer that appeals to the "information wants to be free" sentiment, but not actually well supported.
And the people who are even smarter realize that people will worry about what they will worry about, and respond to threats proportionally to how much people worry about them rather than chiding them about how much they should worry about things.
Human beings aren't rational when it comes to fear, but the products of that fear are very real. We live in a world where people freak out if an adult talks to a child, but happily drive their kids around in the death traps that are motor vehicles. Not only that, but we've gone to great lengths to structure our society to treat the former as abnormal and the latter as totaly normal. Telling people to be rational isn't going to make them that way.
This kind of research is conducted in BSL-3 labs, and there's a not insignificant number of laboratory accidents, accidental exposures, etc. in those labs, by well-intentioned, well trained people.
I saw a presentation recently that estimated, using fairly conservative numbers, that 10 labs working on those viruses for 10 years had ~1600 deaths in expectation. Now that distribution isn't normal - lots of zeros and then some rare but catastrophic outcomes, but like many things, it doesn't require anyone to do anything actively malign. Just screw up.
We shouldn't worry about terrorism because the likelyhood of dying from it is very low, but we should worry about genetically engineered virus because the likelyhood of dying from it is very high?
Last time I checked NO ONE ever died from a genetically engineered virus.
Then again, who am I kidding. We're talking about politics.
My own personal opinion is that that funding should be increased orders of magnitude; a few trillion over the next decade seems judicious.
That would be a pretty bad side effect in itself. There are some pretty nice drugs that approximate what you say, yet as a population we mostly get on with our lives.
It's not in our nature to be satisfied with any persistent state - positive, negative or neutral. So as a tail risk worth worrying about I'd put this in your terrorism category.
Theres plenty of dangerous technology out there, right now. We need not conjure superviruses. The only thing saving us is as usual the incompetence and scarcity of those that actually want to cause harm on a big scale. Don't think for a second it's the security theatre that keeps the numbers down or the lack of weapons of mass destruction.
It means one of several things:
(1) Humans are more resilient against plagues than we think.
(2) It's harder to produce a super-disease than we think, so it's a very rare event.
(3) We've just been lucky as hell.
With that kind of logic we can get anywhere. For example it is way more likely to die after breathing air than after 'breathing' water. It just takes, say, 75 years on average.
Comparing this with other non-natural death causes, such as murder, would be much more fair.
You can protect and save orders of magnitude more lives by focusing on the real threats and ignoring terrorists.
> But another possibility is that we engineer the perfect
> happiness drug, with no bad side effects, and no one wants
> to do anything but lay in bed and take this drug all day,
> sapping all ambition from the human race.
Yes, there are terrorists who do not shirk from mass murder to achieve their goals, eg. flying planes into the heart of the enemy's military and financial centers. But creating a virus that that respects no religion or national boundary and will kill everybody it touches serves no rational goal.
The only reason for somebody to do this would be if the extermination of the human race was the actual goal, and that's just so far-out that even certified nutcases like Japanese subway sarin attackers Aum Shinrikyo would blanch. The ideology of these groups is invariably that, while the rest of the human race may be doomed, they are the Chosen Ones that will survive, but viruses don't play dat. It would thus only make a smidgen of sense to do this if you had an absolutely solid antidote/vaccine that would ensure that your group can survive the onslaught... and if that exists, the rest of humanity can develop one as well.
hacking our bodies will likely be more powerful than hacking bits
Could we please get a link / source to this current data?
Congrats to the prepressive nations who are incentivizing the development of these technologies by being opressive.