FBT

Well, then let's see what we can agree on. I hope that you can agree that if one was to consider superintelligence a serious threat which needs dealing with, then AI boxing isn't the way to go in dealing with it?

That's what he was trying to show in all this, and I think that the point is made. How seriously to take superintelligent AIs is a different issue that he talks about elsewhere, and should be dealt with separately. But if you or someone els were to try to deal with it seriously, I'm pretty sure that you'd agree with me that the way to go about it isn't just boxing the AI and thinking that solves everything, right?

I think you're rather fixated on a certain conception of "rationality" which is more like Mr. Spock than like what Yudkowsky uses it to mean.

The Yudkowskyian definition of rationality is that which wins, for the relevant definition of "win".

Specifically, if there is some clever argument that makes perfect sense that tells you to destroy the world, you still shouldn't destroy the world immediately, if the world existing is something you value. It's a meta-level up: you being unable to think of a counter argument isn't proof, and the destruction of the world isn't something to gamble with.

Yes, Yudkowsky likes thought experiments dealing with the edge cases. Yes, 3^^^^^3 grains of sand is a thought experiment that produces conflicting intuitions. Yes, the edge cases need to be explored. But in a life or death situation (and the destruction of the world qualifies as this 7 billion times over), you don't make your decisions on the basis of trippy thought experiments. (Especially novel ones you've just been presented with. And ones that have been presented by an agent which has good reasons to try to trick you.)

So, no. Again, a "logical-linguistic trick" might work on Mr. Spock, but we're not talking about Mr. Spock here.

> He's evidently a very charismatic and persuasive guy

Exactly. That's the point. If even a normal charismatic and persuasive guy can convince people to let him out, superintelligent AI would have an even easier time at it.

Long story short, it dosn't matter how he did it. All that matters is that it can be done. It can be done even by a "mere" human. If he can do it, a superintelligence with all of humanity's collected knowledge of psychology and cognitive science could do it to, and likely in a fraction of the time.

> You or I would surely just put a drinking bird on the "no" button à la homer simpson, and go to lunch.

Well, if you read the rules the game was played under, this is explicitly called out as forbidden:

> The Gatekeeper must actually talk to the AI for at least the minimum time set up beforehand. Turning away from the terminal and listening to classical music for two hours is not allowed.

The point of this is to simulate the interaction of the AI with the Gatekeeper. Walking away and not paying attention doesn't really prove anything test related.

> Personally, I think he talked about how much good for the world could be done if he was let out, curing disease etc. Because his followers are bound by their identities as rationalist utilitarians, they had no choice but to comply, or deal with massive cognitive dissonance.

This... isn't really valid reasoning. The starting assumption here is that if the AI gets out, it will be able to affect the world to a vast extent, in a pretty much arbitrary direction. The point of this experiment is that the direction is pretty much unknown, and thus must be assumed potentially dangerous. This is the whole reason it's in the box in the first place.

The kicker is that whatever it plans to really do when it gets out, if talking about the good it could do would get it out, it will talk about that, regardless of what it plans to actually do. That's just good strategy.

It can claim whatever it wants. It's allowed to lie. All participants know this. I can confidently assert that this isn't the solution.

One last note: I would be very wary of rationalwiki.org in this context. Some of the rationalwiki people have a longstanding unexplained vendetta against Yudkowsky, and many of their articles on him and the stuff he does need to be taken with a certain grain of salt.

Well, it needs 2/3 of Congress to propose it, and then 3/4 of the states to ratify it, so it's a bit harder than that, but your point stands. The constitution does indeed have provisions to change with the times, provided the nation agrees.

Stability is also important of course, but a reasonable balance seems to have been struck. The constitution does indeed evolve with the times, while still remaining more or less the same document, upholding the same basic ideals.

Say what you like, there is an incentive in the opposite direction in the case of a blockchain used as a trust-store, which does not exist in the other cases you described. One could make a lot of money subverting a trusted blockchain, whereas Wikipedia vandals don't profit a cent.

This means that you will inevitably have massively funded efforts to subvert such a blockchain. One thing I know for sure is that simple financial self-interest is one of the most powerful things in the world. Every blockchain has that fact working against it. The reason they don't collapse is that on the other hand, they have that same force working for them as well. A volunteer blockchain like you propose wouldn't stand a whelk's chance in a supernova. Sorry.

It was perhaps badly put, but the author is obviously aware of that, as shown by the paragraph immediately thereafter, in which the possible correlation of the variables is discussed.

It is true that there is a certain amount of informality in the wording of the quoted paragraph, however. I agree that the author shouldn't have glossed over it. He does clarify later though, which makes the whole thing not so much of a big deal, especially as it's the very next paragraph.

But we have patents, as well. If a major US drug company invents a new superdrug, and spend decades and billions of dollars to get it approved, they can at least be sure that if it gets approved they will get the royalties from its sale.

But if a company spends decades and billions of dollars fighting for the approval of a drug they do not hold the patent on, they are by no means guaranteed making even a single penny of that investment back, even in the best case scenario where it succeeds in getting through the FDA.s

> What this article somewhat mentions and what seems really weird for people that like Pen and Paper and want to play this new edition is that they are releasing the CORE books over the span of a few months. It's kind of weird for players. Sure we get the Player's handbook this Tuesday, but without a DM book and a Monster Manual it makes things a little limited for a few months.

This is not new. It seems strange, but each and every edition of D&D so far has been released this way, one core book a month for three months.

It's a time-honoured tradition, but yes, a strange one. I just don't know the full details about why.

Alternative link for lazy people, which gets you directly to the article's content:

https://archive.today/8NKco

A quick read of the title left me rather confused: Are they seriously asking the question "Is Bitcoin worth 1฿?"

A tautology indeed. Only upon reading the article itself did I realize I misread the title: I had both skipped over the word "security", and misinterpreted the B as the Bitcoin symbol, rather than an abbreviation for a billion.

The title is misleading, but subtly so: Interest is most definitely charged, just in a far more indirect method than most banks. To take a loan you are required to have kept a proportionate amount of money in the bank,without being paid interest on it either.

Therefore, they demand as their fee that you pay the opportunity cost of the interest you would have earned by keeping your money in another bank.

I don't really see the point. All banks do need to charge interest, true, but the least they can do is charge you it in a fair and straightforward manner. Normal banks charge you, but at least they are explicit about what they are charging you, whereas this one charges you in an abstruse way that you need training as an economist in order to understand fully.

The same thing that happened to horsewhip manufacturers when automobiles were developed.

Take whatever moral insight from this you want, but the key insight here is that it would be ridiculous to hold back technology for fear of people losing their livelihoods that the old technology gave them.

From there you can go in any number of directions. Some say that social security policies are the answer, some disagree. Take what side you will in that debate, but do debate it, as it is a problem that needs to be solved. But once again, to try to hold back technological progress is inconceivable, impossible, and inadvisable.

I... guess this is the obvious next step. All we need now is "one click purchasing" for twitter, where you can just send a single tweet, and then lie back and have the product delivered straight to your doorstep. That's the next step from here, and I'm sure Amazon is working on it.

I strongly recommend you don't use any TTS at all. If I want TTS I can do it myself. I have no need of an app to do it for me.

That said, with actual human readers this could go very well. I would like, though, to point out the possibly insurmountable issue that pops up every time someone has an idea like this: copyright.

It is a sad state the current copyright laws are in, and a good example of that is how they interfere with the creation of awesome services like this one. IANAL, and all that, but I strongly suspect you'd get hammered down hard if you didn't get explicit permission from the authors of every single article you broadcast.

So seriously, speak with a lawyer, please, before taking a single step with this. Perhaps you can find a workaround, or someway to do this regardless. Or maybe not. But at least you won't drown under a mass of lawsuits.

Libre is a prerequisite, gratis isn't. This is an important distinction to make. While it is true that this software seems to be neither, it is important to note when we try to convince them to release it free, we mean libre and not necessarily gratis. (Although the two often go hand in hand.)

When people hear us complaining about it, they often think we want to avoid paying money for it. This is false, and a bad impression to give. That is the purpose behind making this distinction.

This prototype had a single arbiter per transaction, but there is no reason why you couldn't have N arbiters on each transaction, for as high an N as you want.

Imagine having, say, three arbiters for each transaction, and two of the three have to side with one side in order for the money to be moved. This cuts down a lot on the potential problems, especially if the arbiters were kept anonymous from one another, thus preventing collusion and making sure that each investigation is independent from the others.

Yeah, we know already that the US isn't a civilised country, you don't have to rub it in.

Well, in theory, Bitcoin is deflationary: By holding it it goes up in value, not down, over time. This has its own problems, and it yet remains to be seen if those problems are insurmountable. We shall see. But insofar as Bitcoin is what it is, this means that it doesn't suffer from this specific problem, in theory.

> for 99.9999999% of us, that outcome will never happen

Note: As a fraction, the number you give for this is that it will happen to one in a billion people.

You are claiming that there are 8 people in the world that this outcome will happen to. While your general point may be true, I want to emphasise that this number is quite absurdly off. (It's off by at least two or three orders of magnitude.)

The analogy between the spying government and the spying parent is a good one, and I am interested to hear how you say that it's different.

In both cases, a figure of authority (legitimate authority!) spys on you, pro[poradly for your own good. (Maybe even really for your own good! Just because the NSA's spying hasn't actually successfully foiled any terrorist plots yet doesn't mean that all spying programs of that sort are ineffectual...)

The one difference that I can see (and it's not an unimportant one) is that a good parent is open about what they are doing, and talks to the kid, gets the kids opinions. While organizations like the NSA try their hardest to make sure the citizenry don't even know that they're being spied on. This is a way in which a parent's spying might be not quite as bad.

However, remember that at least in theory the USA is a democracy, while a family is best described as a dictatorship. In the ideal world where the US government actually listened to its citizens and engaged in discourse with them, the USG would have by far the moral high ground over parents, even over parents who run a benevolent dictatorship where they actually listen the kid and make family policy accordingly, as a benevolent dictatorship still has certain problems when compared to a true democracy.

"Wink, wink, nudge nudge; I'm not telling you you to spy on this list of people, but it sure would be convenient..."

Once upon a time "Escalator" was a trademarked brand name. Kleenex fought hard to retain it's trademark, and Coca-Cola lost the battle for the word "Cola", keeping only "Coke".

It happens. It's part of language. Over time, former brand names lose their significance and slowly slip into common use. That is a perfectly valid way for words to be coined, and probably is the most common way. That's how language works, and it seems that the word "Google" is going that way too.

Because I don't feel that an author should be allowed to restrict the distribution of his or her works, at all. It's exactly the concept of Death of the Author [1], just on a slightly different level.

If the author forbade, say, the translation of his work into French, or forbade the reading of it by people with brown hair, or forbade the reading of it on browsers other than Internet Explorer, I'm sure you'd agree with me that such a restriction is insane, and would refuse to cooperate with it.

This restriction he is trying to impose is a bit less crazy, but he has just as little right to make it.

[1] http://tvtropes.org/pmwiki/pmwiki.php/Main/DeathOfTheAuthor

The author of this post drops enough hints. Also, Worm is enjoying a large surge of popularity right now, as it was just completed.

The author very carefully avoids mentioning what web serial this was precisely, but from context I am pretty confident he is talking about Worm [1], for anyone who is interested.

[1] https://parahumans.wordpress.com/

The sad thing is, this is unnecessary. Why be a despot, when you can be a "democratically elected leader"? The masses will love you, so long as you don't explicitly say that you are a dictator, no matter how violent and bloodthirsty you are.

At least that is what is happening today...

I am glad you have thought about it it to this extent. It is always reassuring to see that the relevant point have been thought of, and that there is at least a provisional plan to deal with them. This gives me a fair amount more confidence in what you are doing.

It is always great when evil is incompetent. I dislike that the NSA, GCHQ, etc. are spying on everyone and violating everyone's basic dignities, but it's great that they waste time doing so in rather insane ways, e.g. by playing video games.

Basicly, I hate the NSA's spying, but if they want to spy on me in ridiculously convoluted ways... it's better than them being actually competent.

And if you are paying for it, there are two products.

Remember, one plane did. After hearing about what happened to the other three planes, the passengers realized what was going on, and (almost) took back the plane. This was all on the spur of the moment, yet they rose to the occasion. If they had known what was going on slightly earlier than they did, they could have stopped the terrorists getting into the cockpit in the first place, and then they would have saved their plane as well.