sawwit

Ah I dimly remember that it also has an Occams razor built-in, so the prior probability falls off with the lengths of the strings that describe the Turing machines?

I cannot judge to which degree these theses are bullshit, but I've found LW a tremendously rich source of thinking tools and I'm convinced that reading or skimming a lot of the sequences have improved my thinking.

Regarding the rape sequence in HPMOR: It's a terribly chosen trope to convey that the fictional society has very different values from ours. Apparently it ties into various parts of the story, so that EY didn't remove it and only toned it down after it was criticized.

The prior of Solomonoff Induction is a uniform distribution over the enumerable set of Turing machines?

What's the cause that follows from that it mostly affects the bottom 50%?

It possibly associates the painting texture with poor skin health.

From a data privacy point of view I would very much appreciate if you added a prominent "delete upload" button to the result page.

I've been using Emacs for 1.5 years or so, then switched to VIM for a brief time period and I fell in love with the way it uses keybindings in a grammatical manner, using verbs, prepositions, subjects and objects. For example, the key sequence di" means "delete in parentheses", and similarly diw means "delete current word". It's also very intuitive to switch back and forth between a 'text insertion' (INSERT) and a 'command' (NORMAL) mode, because quite often you do multiple commands at once (e.g. cutting things here, pasting them there). It seems very natural to have completely changed keyboard bindings for a command mode (instead of relying on finger stretches for each command). I also very much prefer line-wise editing (e.g. in visual line mode) now, even though I thought I would never like it. Then I heard about Spacemacs, that it gives you some peer-reviewed configurations and VIM keybindings and I was immediately sold.

Nonsense, please quit that strawmanning. Reducing the pace of consumer technological progress to the benefit of more equality would very likely beneficial for humanity; and it does not imply returning to underdeveloped times (assuming it would reduce it significantly in the first place).

This is not what OP is saying.

It is much older than this. People have started to come up with all kinds of differentiable structures ever since they've found out that the leaning rule back-propagation of error can be derived using the chain rule in the early 1980s or so.

Perhaps this could be used to suppress subvocalization during reading:

(1) Look at a lot of people reading the same text, divide in to subvocalizing and non-subvocalizing group.

(2) Find difference in neuronal activation patterns.

(3) Let subvocalizer read the text and apply electro shocks if the fMRI shows the typical patterns for a subvocalizer. Perhaps the cortex will self-organize to turn off the subvocalization.

On the other hand the creative, intellectual minority that wants and needs fast access to information is left behind which is a cultural and economic collateral damage of fashion.

I think I would have added "attractor states" and "constraints".

I never said am certain, I'd assign both cases about the same probability. Since I've already stated why I think the malevolent simple AI scenario such as a paperclip maximizer is plausible, you haven't stated why you find it implausible. That is why I asked.

Also, what does the Intermediate value theorem have to do with it?

Needs are what your neural architecture has evolved to optimize (e.g. sustenance, pain avoidance and affiliation). Goals are the preferred states that the brain learns to optimize its needs (via reward). Values are cultural or (fortunately, but also necessarily due to evolution) Schilling point memes which manifest as virtual rewards that mostly enable human cohabit (e.g. you are good if you don't litter the street, i.e. that will increase your chance of future social reward).

- Biases and heuristics are both priors, but they are particular kinds of priors. The former mostly refers to priors which systematically slightly shift our thinking away from the truth and on which we can only counteract by self-reflection. The latter is a prior which we consciously choose which is very rule-like and probably overly general.

- Aumann's agreement theorem assumes that all actors are perfect Bayesian actors and have plenty of time to talk to each other. Since goals and values are not preprogrammed (unlike needs), it follows that they can update on these things as well, if one party convinces the other one of better goals and values for maintaining/achieving their needs. Unless I'm overlooking something, it must assume that the actors have the same needs.

What are the advantages?

I made that assumption explicitly, actually. To make it worse, it seems plausible that these kinds of dangerous AIs will be the most simple ones to build, since they won't require a lot of goal engineering.

Intelligence is a superset of feedback loops. It is concerned with cases in which feedback loops are not sufficient to optimize the agents objectives, and attempts to reach them using learning and prediction. The complexity in behavior comes from interacting with a complex environment and having many model parameters; not (necessarily) from the initial goals. (As an intuition pump, have a look at GoogleMind's atari reinforcement learner. The essential parts of the code fit onto a single page [1].)

[1] https://github.com/kuz/DeepMind-Atari-Deep-Q-Learner

What the thought experiment tries to get at is that it is hard to predict how a super intelligence will evolve from a set of simple objectives. So far, it appears that a set of objectives is in fact necessary for intelligence.

The question is whether the AI will go crazy like a mentally ill person, if it lacks empathy and curiosity. It may seem intuitive that a superintelligent AI will understand our values (since it is superintelligent), but, assuming intelligence is necessarily an optimization process of predefined goals, why would it be interested in us in the slightest, if we don't pose an advantage for it optimizing its objectives (e.g. sustenance)? Worse, we might be in its way because we could end up competing with it for resources such as sunlight, carbon compounds and oxygen.

> It ticks me off a little bit when people discount it as "definitely" a "cargo cult" thing. If you don't like it, that's fine.

Read more carefully. I merely said that people mostly do it as a challenge or as a cargo cult, doubtfully because it is a useful skill (unless you happen to spend an extremely large amount of time writing, which likely only applies to a small percentage of those that get told to learn Dvorak).

I think this one is a waste of time unless you seek a mental challenge. In my experience, it's just not worth getting confused on QUERTY keyboards all the time, and the 5% speedup or so is really not worth it. It's definitely more a cargo cult/challenge than a useful skill.

> Switch from emacs to vim or vice versa.

Switch to emacs + evil, or alternatively to Spacemacs: https://github.com/syl20bnr/spacemacs

Let me recycle an earlier comment of mine:

---

> "don't turn humans into paperclips" is part of the context of "make more paperclips"

The idea expressed in this thought experiment is not that the AI gets its objective by parsing a sentence in the context of human culture (then it would likely comprehend that the actual intention is to maximize the economic success and eventually the human preferences of its creator). What is meant is that the objective is crudely implanted into the AI as an ultimate goal, in a similar way to how sustenance, pain avoidance and affiliation are very basic goals in our cognitive system. It is not entirely obvious that this is a stupid thing to do; hence the thought experiment. Will the AI suppress its urge once it comprehends human culture enough to understand the intentions of its creator? Will it rather successfully learn all the tricks to convert matter into paperclips before it considers studying human values? If the AI does not have a curiosity objective, it will likely not care about us very much, apart from the information that helps it optimizing its objective function, human values likely not being one of them.

What makes you so certain?

> "don't turn humans into paperclips" is part of the context of "make more paperclips"

The idea in this thought experiment is not that the AI parses this sentence in the context of human culture (then it would likely comprehend that the actual intention is to maximize the economic success and eventually the human preferences of its creator). What is meant is that the objective is crudely implanted into the AI as an ultimate goal, in a similar way to how sustenance, pain avoidance and affiliation are very basic goals in our cognitive system. It is not entirely obvious that this is a stupid thing to do; hence the thought experiment. Will the AI suppress its urge once it comprehends human culture enough to understand the intentions of its creator? Will it rather successfully learn all the tricks to convert matter into paperclips before it considers studying human values? If the AI does not have a curiosity objective, it will likely not care about us very much, apart from the information that helps it optimizing its objective function.

Alternatively in the browser:

https://webirc.hackint.org/#32c3

https://webirc.hackint.org/#32c3-hall-1

https://webirc.hackint.org/#32c3-hall-2

https://webirc.hackint.org/#32c3-hall-G

https://webirc.hackint.org/#32c3-hall-6

https://webirc.hackint.org/#32c3-everywhere

JSON of all streaming URLs:

https://streaming.media.ccc.de/streams/v1.json

LPT: There is an option to export to iCal so you can set up a notification if you want to watch it live.

Though, aren't there also plenty of other mechanisms that affect it? For example increasing speed of manufacturing procedures and improving knowledge about how to design complex systems. The end seems to be extremely dense blocks or spheres of highly optimized computation substrate, where each atom sits very precisely at the right place. But the state of the art is merely a few layers of crude photo-lithography. Imagine nanobots being programmed to move atoms around to assemble 3D circuits at the 10nm scale. I think this is where we are heading, and this will likely grant a couple of decades of continuing exponential progress. And by then our computation power will be insane. An equivalent of the simplest human brain models will likely fit into pea.

Well, there are at least 40, if not 500 years ahead of me.

On the other hand, in my experience absolute equality can also result in stressful situations because of things like diffusion of responsibility and communication difficulties. Sometimes it's just more efficient to outsource decisions to decision makers rather than worryig about them yourself. I think the Goldilocks principle applies here.