jsnathan

> It's not putting them above a constitution. There is a process in place to resolve constitutional issues. In this case it worked perfectly.

If transient violations that might take many years to resolve, and often present a slippery slope, do not face any censure, then they are practically above the constitution, even if there is a theoretical process.

While you point out that it worked in this case, that is nothing but an anecdote. I could equally list many cases where the exact opposite is the case. Consider for example the difficulty faced in bringing security policies before the courts in the U.S., due to the government claiming that the very discussion of the details of the implementations of these policies would harm national security.

> Punishing people merely for passing a law would itself be unconstitutional. There is no enumerated punishment nor is there any reasonable way to assert culpability.

I'm sorry if I was unclear, but I tried to address both of these points in my first reply. Let me try again to put it more clearly.

First, they should lose their office for abusing its privileges. This has nothing to do with the fact that they violated, or worked against the constitution specifically, but that they violated the oaths of their offices, which happen to preclude that.

Secondly, the punishment does not require newly introduced 'ex post facto' laws, as you seem to be implying, but should be in keeping with the facilitation of the violation of actual existing laws, which prevented the unconstitutional actions from taken place before the new laws were passed.

In other words, a politician that facilitates a law that unconstitutionally provides exceptions to previous existing laws, should be liable as an accessory to the crimes specified by that pre-existing law.

As far as determing their complicity is concerned, I believe that the fact that most everything is on record makes that fairly trivial.

Are you suggesting that elections should give people a status that puts them beyond the countries constitution?

Remember that these principles are so general, they must apply equally to Western politicians pushing security policies, as to e.g. Muslim politicians in Egypt or other ME countries pushing for Sharia-inspired policies.

What their constituents want is a nice idea until this violates the most basic rights of citizens spelled out clearly for this purpose, i.e. to guide and limit any future laws.

That being said, there is always the possibility of revising the constitution. Even if that is not a good idea. But for good reason, that is a lot more difficult to accomplish.

Intentionally violating the constitution is a violation of the privileges of office. If it can be proven, they should be forced out of office.

As for the "punishment", it should clearly correspond to the actual law having been passed, and in what manner it infringed on the constitutional rights of citizens.

Popular support does not make this any better, as minorities are usually the ones who end up hurt.

There's a breakdown of their spending in their S-1 filing [1], under Consolidated Statements of Operations.

According to this, they could be profitable if they wanted to be, but they are using their profits and re-investing in growth.

E.g. they more than doubled their spending on marketing from $17.8m to $39.6m from 2013 to 2014.

[1]: http://www.sec.gov/Archives/edgar/data/1370637/0001193125150...

I think it's not a bad idea to remind people to add a license to their public github projects. It's probably an oversight when they publish code for others to reuse, but don't add a license.

I've had to email people before to ask what the license is.

But what exactly is the point of publishing this bot? Do you want multiple people to start running it? Will the bot create multiple automated issues per repo? That sounds like overkill.

Maybe add a note about that.

Not to say that this is not a useful standard, but I do think we are beyond "inconvenience" here.

Following the current trajectory, surveillance is only going to grow as technology makes it easier and easier to gather more and more information without being noticed.

If the treatment is a product of the current pharmaceutical industry, I think one might have to worry about that - especially in the beginning.

It might be seen as even less important than distributing food to the starving, and medicine to the sick, of which there is still not enough going on.

But there is at least one other pathway. As an AGI researcher, I believe that artificial intelligence will solve the problem for us, and, all other things being equal, the cure will be made available everywhere, at the same time.

My own research makes me believe that we are not very far from that outcome.

Until the possibility is established, many people are going to make fun of, and sideline those working on a solution.

However, I think this thought experiment can bring them to understand that their own beliefs are temporary. If there was a popular movie that followed the plot of how the cure of aging was slowly accepted in society, for example.

Just a short story might be a start.

I think the problem is not that people "support" the idea, but that they are not properly aware of the alternative.

Suppose we cure aging. If there is such a "culture" of pro-aging, then supposedly you expect them to refuse the treatment?

I do not see that happening in the long-term. Maybe a few fringe sects, but never the majority.

Instead, as soon as some people use such treatment, most everyone else will as well.

What this conjecture tells me, is that this "culture" is probably just based on the belief that anyone who refuses to accept aging, must be a fool.

If however a cure is demonstrated, this belief will be disproven, and people will quite immediately embrace longevity.

Or.. do you expect otherwise?

Precedent is a powerful thing.

I wouldn't give up hope yet, for the anti-surveillance movement.

There were some rumors [1] recently that Google was considering to buy Twitter.

Now I don't know if there's anything to that, but in light of Facebook snatching up Instagram, (not to mention WhatsApp), Google moving in on Twitter seems like a fairly logical counter-move.

And just think of all the data!

My point is, this deal might be at Google's initiative, not Twitter's, and might represent the first step in that direction. It's just conjecture, of course..

[1]: http://venturebeat.com/2015/01/23/google-is-absolutely-posit...

> Well, since the paperclip story is only trying to show that even very simple goals can lead to large impacts in the hands of an AGI,

That may well be the original intention. But that's not how it's used in practice, is it? It's cited as an argument that AGI (in general) is unsafe. But it isn't an argument that AGI is unsafe! It's an argument that says that single-objective optimisation can/will violate some of the constraints we did not encode but really want it to respect.

But who cares about that? It's a completely irreal thought adventure, and has no bearing on the actual problem or its possible solutions.

Of course we can say: look, it's a scary thought! But only if we simultaneously admit that it has nothing whatsoever to do with the actual technology.

The only thing it says anything about is a toy variation that noone in their right mind would ever consider building.

> I take this to mean that you think one can bound the impacts if one chooses a complex enough set of goals.

Ehh, precisely. I would actually go further and say that it is fairly simple to do so. If you can tell it to build paperclips, and expect it to understand that, you can also tell it not to damage or disrupt the ecosystem in the process, directly or indirectly.

(Or, to strip away the last shreds of this idea: you can tell it to make no more than 100 million paper clips, and to rest on Sundays.)

> Personally, I would read the actual academic making the argument rather than reposts of it by folks on the internet.

Yudkowsky [1], who runs that site (which I don't frequent btw), seems to be pretty close to Bostrom. They've collaborated in the past. And if Bostrom's arguments can't be paraphrased (by his own friends, no less) without losing validity, that doesn't really seem like much of an endorsement either.

I have nothing against these people. But please stop alluding to hidden treasure troves of arguments that cannot be reproduced and made apparent. I've asked enough times in this thread already if there is any highlight in there.

I have a background in philosophy myself, and I know what to expect if I did pick up this book. And in that vein I really feel no need to do so.

[1]: http://en.wikipedia.org/wiki/Eliezer_Yudkowsky

Now we're talking! The testing thing is indeed impossible to formulate using the code, it has to be formulated in language. But that is not an issue, because (in most bootstrap scenarios I'm considering) language acquisition happens before (or alongside) acquiring the intelligence necessary to understand itself to the point of re-formulating itself. So we can formulate the tests in English, and subject to the language acquisition having succeeded (which we can establish independently) we can trust that the tests are correctly understood.

This actually makes the tests more robust, because the AI self-describing its own plans in its own internal "vocabulary" is quite able to recognize vague associations, such as the is-it-death-or-not scenario you describe. If it is a similar enough conception that it can be (internally) described as "death", (past a threshold), it will (with a testably sufficient probability) trigger the associated constraints.

The idea that a "buggy" intelligence could acquire language, and understand it at a level sufficient to "digest" advanced technical material, is sufficiently unlikely.

More to the point, the "ethical" tests and constraints are only the topmost level of a pyramid of testing that ensures that all the pieces perform exactly as intended along the way. The system at no point behaves in a random or inexplicable way. So by the time such capabilities are unlocked we know that the algorithm works in accordance with theory.

You are also completely right that this initial "dumb-almost-AI" would have to be pretty smart already. The trick is that it grows within a set hardcoded framework at first, which is sufficiently flexible to allow it to in fact perform language acquisition and (later) intelligent design.

Only then would re-writing its own hard-coded base become interesting. Therefore the design I am working on already passes human-level capabilities. But that does not mean the AI cannot improve itself even further. And the trouble in the context we were discussing it in is that we need to be sure that if it does rewrite itself - and in a way that we might not be able to easily comprehend - it still cannot break out of the mold.

As far as the goal function is concerned, it is as you can imagine rather complicated to do. And the solution will undoubtedly differ based on the architecture you are working in.

What I'm doing personally is pretty straight-forward; it involves iterative constraint solving using quantitative metrics to increase trust and/or doubt in various concepts and plans, and this trust is established based on consistency and coherence with previously acquired (probabilistic) background "knowledge" - as well as consistency within any single context of thought. A single context might be a sentence or a paragraph reading, or a logical puzzle being solved, etc.

Over time the probabilities increase (at least, in theory), both for any single consideration and across the entire knowledgebase. (See NELL [1] for comparison. Except, it's a lot smarter than NELL.)

So the goal function is a fairly generic optimisation algorithm, and the initial goals defined are equally generic - increasing overall consistency, coherence, and efficiency. (Measuring these is fairly straight-forward.)

The trick is in setting up the reward structure autonomously, so it keeps advancing itself. The learning algorithm needs to create sensible (long-lived) intermediate goals and maintain their relative priorities over time. Among other things.

(I really can't explain how it works though without going over a whole bunch of other supporting algorithms).

In any case, to begin with the A.I. need not have a desire to do anything at all except learn - and it improves within the constraints of the hard-coded base framework. But the model can still advance (at least, in theory) to the point where it acquires the ability to comprehend its own design and re-engineer itself completely.

By this time however it is able to understand and apply the ethical constraints we defined in English, alongside all the other intermediate goals (but with specific higher priority).

[1]: http://rtw.ml.cmu.edu/rtw/

I understand the paperclip objection, but I do not consider it valid.

I tried to point out the problem, which is that it focuses on single-objective optimisation when the only interesting question is multi-objective optimisation.

I haven't read the book, sorry, but I have seen some of these arguments reposted on the net, especially at [1] - and what I have seen so far did not inspire in me the desire for more of the same.

There are a lot of interesting questions and problems about making AI behave as expected. But from my perspective they are all technical problems that have specific solutions in specific architectures. And none of them are particularly daunting.

[1]: http://lesswrong.com/

Having spent years researching AGI myself (and as I intend to start a venture based on my research in the near future) I can say that I really do not care for taking thought-experiments about specific toy architectures and trying to extrapolate from these to actual designed systems. If you set your model up to be naive (intentionally or not) and then conclude that the problem is oh-so-difficult instead of concluding - like an engineer would - that we need another conditional check right around here, I'm not surprised that you are daunted by the problem.

You're not telling me anything new here.

To address your points, if I understood them correctly:

First of all the code doesn't have to be self-modifying - it can create a new separate version, which is then subject to testing administered by the old version (both the goal function, which designs the new code to spec, and a hard-coded control mechanism as a backup).

Secondly, I see no correlation between producing a next iteration of itself and running the test suite. What do you mean here?

Thirdly, the AI does not run "arbitrary code", it runs inside of a sandbox/VM, with tightly controlled inputs and outputs. Same as we think inside of our brains, but act using our bodies, arbitrary code execution would be bodily action. But the AI does not have a "body" at this point, and it cannot interact with the OS.

The solution I pointed out cannot not work. Only magic could make it break. And there is no magic.

If someone reasoned out that actual _testing_ of the code does not work, and his reasoning is valid, then he must have made some unnecessary assumptions in his model of the computation. If you have any specific paper in mind, please point me to it and I will try to tell you how to fix the toy-model to remove the problem.

> if and only if it wanted be friendly and respect those constraints in the first place

No, not if it "wanted" to - if it was written that way.

> otherwise you should expect for it to figure out how to work around them to achieve whatever goal function was implemented

Yes, if the goal function did not include preserving certain (behavioral) invariants in its own programming, then yes.

But, it would.

> Making a spec "AI, do X" that isn't catastrophically bad is a very hard unsolved problem.

No it's not. That's just something you throw out there. I just gave an example of exactly how to do it. Write a simple test to check whether a certain outcome is endorsed or rejected by the A.I.

If you are the designer, you will not stop there either. You will want to check that the reason it is rejected is because the machine is working in exactly the way you predicted.

Do this for a couple hundred scenarios, and make sure that the clock ticks right, and I see no reason to introduce any talk of "hard unsolved problems".

Don't take this the wrong way, but this book is precisely the kind of thing I was talking about.

That paperclip idea I talked about is also something that Bostrom thought up [1]. I didn't make it up.

If you have any argument in mind (from that book) that you find convincing, please go ahead and state it outright. I'm truly curious.

Edit: In response to your edit(?), I do agree that the most worrisome problem is some bad actor gaining control of this technology. But that is different from saying it is dangerous in itself. Most anything can be abused for ill, and the more powerful the more dangerous. I completely agree on that.

[1]: http://wiki.lesswrong.com/wiki/Paperclip_maximizer

AGI will completely upend society by giving every human being access to practically infinite resources, making even the poorest of the poor as wealthy as today's billionaires. Couple this with extended life spans and intelligence enhancement, and the entire (human) power structure as we know it today is going to melt away like an iceberg into the vast oceans.

Please do not accuse me of hyperbole for I have not even described a fraction of the possibilities for this technology, as you probably know.

But the idea that it is therefore "dangerous", is nothing but a pre-theoretic misunderstanding of very complicated machinery. When we finally do build it - and that will be sooner rather than later - it will perform exactly according to spec, and in no other way.

I may be talking non-chalantly, but the fact is I have seen nothing but the _most absurd_ arguments supporting the idea that there is a great danger in this: along the lines of "well what if we told it to just go ahead and make paperclips and then it decided it had to kill us all and use our bodies as raw material?". Boy, oh boy. You think we might just program it not to do anything so downright "retarded"?

And that _is_ the point. Not that it is bloodthirsty or cruel, but that it is clearly in violation of the constraints that any sane designer would encode in the software - and _test_ for before production.

>> nok (is-good-idea (turn-humans-into-paperclips))

Seriously, I have not come across anything but this kind of apocalyptic-sci-fi-plot style fear-mongering; if there is any serious (technical) argument to be made, I would very much like to hear it.

Wonderful summary of the arguments that can be used to defeat this amendment on the floor. I hope we can collectively make many of the British MPs aware of the situation and help to stop this legislation from going forward.

The actionable part of the EFF's article is at [1]. It's a list of Twitter accounts of British politicians that you can contact directly to make them aware of this.

[1]: https://act.eff.org/action/tell-britain-s-lords-don-t-let-th...

It might improve usage per household which agrees to pay for the nets, but what about the people who refuse to buy them?

Couldn't giving them out for free increase coverage even if the average usage is lower?

Strong guarantees of privacy are essential to enable political dissent. Without it, opinions opposed to the prevailing majority view point are not only side-lined, but the very expression of an interest in them becomes a cause for controversy and social rebuke.

Without privacy individuals are unable to explore, consider and evaluate diverging ideals, and eventually reach new personal convictions which may in time turn the tide against the status quo. Instead, the fermentation of ideas is cut short, as most people are unable to stand the push-back inflicted on them by society at an early stage in their thought-process, leading to self-censorship and group-think.

Society incurs much more damage from so many angles, ranging from murders to car accidents to diseases, that it does not need to jump in such abject terror at the thought of political violence - still a comparatively rare occurence in most places on this earth - so as to trample on the very foundation of every democracy:

Which is the freedom of the individual in his own thoughts, to form his own personal opinions unimpeded by institutionalised social pressure.

The mass-scale erosion of privacy must be stopped before it is too late to reverse the trend.

The ethical necessity of addressing global human suffering, which an AI would be supremely equipped to do, let alone the incredible gain in wealth and power that goes along with it, makes it incredibly unlikely that any entity smart enough to develop AGI would try to keep it a secret for any large span of time.

I agree with you. There seems to be a new marketing trend right now, where any product can be tagged with the term "artificial intelligence" if it uses any of the algorithmic techniques developed in a field of AI research.

Some people are saying that "artificial intelligence" is the new "big data".

One example of this is thegrid.io, which as far as I can gather uses a constraint solver to help in positioning items on a page, and they call it "AI websites that design themselves".

Whether this is a good thing or a bad thing I don't know, but I think it is here to stay. Because it sounds good..

One might make the same point about the US government.

Sad to see this post flagged of the front page without further discussion.

That makes no sense whatsoever. If the law is unethical then so are the consequences of that law.

This is like saying that laws against homesexuality may be unethical, but trying to avoid being punished for being gay is wrong.

I do think there are some interesting tidbits here, (albeit none of them exactly new), only that OP does not explicitly highlight them. The article is simply a narrative of their success story, with most of the analysis being left up to the reader.

That being said, what I got from it was:

1. Acute Observation. They had a prior product (a tiny social network), and they noticed that one of the features they had implemented (experimentally, I expect), namely the photo-filters, was disproportionally popular among their user base. This is a prime example of what Drucker calls "unexpected success", and drawing lessons from that.

2. Focus. Their original product had a much broader scope and they narrowed it down considerably. Not only did they discard a lot of their existing feature-set, but they also reduced the platform support in order to launch earlier, and test their idea ASAP.

3. Traction Can Carry. They had an existing user base, albeit small, and they managed to cross-sell some of those users on their new product from the start, using them as beta testers. This is similar to the "halo effect", only on a much smaller scale.

4. Luck. As OP puts it: "[Our] combination of being photos-first and public-by-default would prove to be a combination that solved an unmet need."[1] I suppose they thought this through, but as far as turning out to be right, I suppose one could say they got lucky.

The take-away for me then is that we could try to observe "unexpected success" of experimental features in our own, or even other people's products, and use the same strategy of applying focus; and that in the end there is always an element of chance, so one needs to keep trying different things, or different variations of the same thing, until one gets it right.

[1] Someone else mentioned Radar.net as a precursor to Instagram; but that app did not have the feature of being public-by-default.