darkmighty

You have some private data. You want to compute a hard function of this data, and your computer isn't powerful enough.

For example, this could be a rendering problem: you provide the models, a distributed system computes it (for a cost). This system cannot be private, efficient and reliable. It can be private and reliable, but hugely inefficient, or efficient but non-private and non-reliable. With a non-distributed system (i.e. Google compute, Amazon cloud, etc) you get pretty much all 3 of them, with a slight caveat that privacy is guaranteed circumstantially/legally instead of mathematically. But it's extremely safe to assume you'll be fine.

You want to run and real time multi-user application.

For example, and MMORPG. This one is a fun challenge for sure, but it's a complete nightmare to develop, and again needs sacrifices. In this case you probably can't achieve speed, security, and large number of players simultaneously. Every decision (like "what did this monster drop?") needs to be done through some large scale consensus to guarantee security, which kills latency completely. If you're extremely lax with security and consistency, then you can probably make it work, but accepting users might cheat, possibly game-breakingly. Or you could achieve it all with a handful of players, and reasonable efficiency[1], but your scope will inevitably be limited. I believe Ubisoft's ForHonor[2] uses P2P gameplay, but I'm not sure how safe their system is. Also I'm sure stats and most persistent data is stored centrally.

[1] See Mental poker:

https://en.wikipedia.org/wiki/Mental_poker

[2] Some reddit discussion from quick googling:

https://www.reddit.com/r/forhonor/comments/5u8jlh/why_forhon...

Note that the players largely loathe this system.

I would reiterate that simply because there are problems with naive visualization, we shouldn't discredit visual thinking.

There are several key elements to effective visual thinking. The primary importance is to keep it grounded in proofs and theorems, so you know exactly what are your limitations. Often you can use a geometric argument on top of a few theorems and you get a very strong result intuitively, and then use this intuition with a tiny amount of algebra to prove it (which might take you forever to arrive from a purely algebraic perspective). Another key is that there are several ways of visualizing things. You can almost always transform a problem into an equivalent one that is easy to visualize (just need a little bit of care with the transformation, etc).

---

For example, you can show functions form a vector space, visualizing some interesting algebraic properties about them, even if it constitutes an infinite-dimensional space.

You can show several operators (such as d/dx) are linear, you can give it a norm, internal product, etc. This trick lets you use visual tools (and linear algebra tools) with arbitrary functions. You can visualize projection of a function into a subspace, or into some non-canonical basis -- yielding useful applications -- such as Fourier analysis.

Fourier analysis itself is a fertile ground for visual thinking. You'll be finding trivial arguments for seemingly difficult decisions such as "Does this linear system have a bounded output for any bounded input?". There isn't one right way of thinking about anything.

---

On the other hand, it can't be stressed enough the importance of keeping track of formal assumptions, axioms, definitions, theorems to construct valid, correct proofs. That way you minimize the risk of fooling yourself, and can safely use your intuition.

This 3B1B video exemplifies many of those elements:

https://www.youtube.com/watch?v=zwAD6dRSVyI&t=633s

See also Deluxe Music in the Amiga 1200:

https://youtu.be/OwROU4UkE84?t=45s

Indeed I'm not a fan of "blockchain" as a buzzword at all (especially now that people are trying to apply it to things that shouldn't be distributed); but clearly there are many things that do benefit from distribution.

Things that come to mind are Namecoin, I2P, IPFS, and other tech on the pipeline that can bring the internet closer to it's original ideal of distributivity, each person responsible for their content, etc.

Those are also free software ideals: while great strides have been made in open sourcing user-side software, (mainstream android, Linux on servers, etc), the internet just hides a massive closed system: before you could at least inspect how things work through reverse engineering, but querying web servers is literally a black box model. The problem that not everyone wants to run their own server or even deal with renting out can largely vanish from certain applications. We've started with the easiest ones, which were file sharing (reputation-free), then Bitcoin, with more basic web tasks incoming.

It's important to keep in mind most of those are fit for low-efficiency. Unless we have a breakthrough with Fully Homomorphic encryption, the bulk of computing itself will be centralized (or you will have to accept a mix of high cost or unreliability).

> If you expressed the efficiency as "2 joules of heat moved per joule of mains electricity consumed" you eliminate the canceling that would let you phrase the ratio as 200%.

But that's the performance. Efficiency is usually a measure of how good the performance is.

In this case there's a practical reason not to call it efficiency, because there is a well understood maximum performance which is the Carnot limit, so for practical purposes (to avoid confusion) PumpedHeat/Power is called "coefficient of performance"and CarnotLimit/PumpedHeat is called "efficiency", and it has the usual properties that it cannot exceed 100% and it measures how well your system works in a more or less absolute sense.

Looks like a very nice paper, thanks.

That's an adversarial picture, but not nearly as dramatic as the epsilon-adversarial examples -- and in particular there are no logical inconsistencies (confusing a cat for guacamole or a turtle for a riffle). I don't think anyone doubts a form of camouflage or isomorphic deception is unavoidable.

I don't doubt there are weaknesses, or even fundamental lack of interpretability for classification spaces that overlap (i.e. have a "morphing sequence"). One example that sticks out from my childhood is this image:

What do you see here?

https://seeklogo.com/images/A/Antarctica-logo-9202568406-see...

My entire childhood I saw a weird face (without thinking too much about it). This is the logo of a brazilian beer brand. When I was a teenager I saw an ad with two penguins, and then it clicked.

Optical illusions are well documented too, some classic examples:

http://www.optics4kids.org/osa.o4k/media/optics4kids/womanil...

https://obasandbox.files.wordpress.com/2012/07/carolines-vas...

http://i.telegraph.co.uk/multimedia/archive/01120/blackballs...

But it's quite probable we'd found any glaring adversarial issues by now. After all artists can conduct a semi-whitebox, mostly blackbox adversarial optimization of illusions (I'm sure some process like this is how they came up with the Old/Young lady illusion). Note however that even those are particular errors like incorrect brightness estimation, or near-complete dichotomy (it's not that either the young or old interpretation are incorrect in some sense). An epsilon-failure seems much more difficult to come up with. The distinction seems mostly in the ability to apply basic logic on top of pure pattern recognition, sort of greatly enhancing the decision boundaries through recursive thought. Eventually logical features (a quick "proof" of sorts) win. You "prove" that what you're seeing indeed can't be a rifle, it must be a weird turtle.

This logical approach probably has gradations in power. In general it might even be algorithmically undecidable whether an image is logically valid. Generalizations of Escher illusions:

https://upload.wikimedia.org/wikipedia/en/e/e8/Escher_Waterf...

come to mind. In practice we cut off the decision process when we've found main logical features and connections, so unless the image inspires superficial uncertainty a deep logical inconsistency could slip by (that a more intelligent person may not miss by universally applying a deeper consistency check).

I see. I ask because there's a common observation that a human (and perhaps an AGI) would never really even run the risk of confusing say a turtle with a riffle, which I tend to agree with.

Part of it is mistaking the forest for the trees. It may be just an artifact from the requirements we place on image classifiers (which are very lax) and the way we train them, not anything fundamental.

Indeed I believe we tend to think with more solid logic, specially when the decision becomes difficult. A DNN will look at the statistics of a feature set and make judgement upon that. A human can categorically reject certain hypothesis from definition requirements: a Riffle is a weapon. It must have a barrel to guide the projectile, and a muzzle for it to exit. It must have some firing mechanism (usually a trigger). Even if at a glance we get confused by exactly what an image is picturing, we can make quick logical judgements on sub-features to make epsilon-misclassifications almost impossible.

A network that would act that way would need some recursive behavior (to implement the varialble-time classification efficiently), a recursive "logic module" or "language module" plugged into the end of naive feature classification.

Hi there!

What do you think are currently the best method to fight adversarial weakness?

Would simply generating an augmented training set including adversarial examples be sufficient? (in your case, for example, you would include random poses of the adversarial 3d models in the training set) Or do you think a totally new architecture of training method is necessary to deal with it?

And if it's the latter, do you think this is a symptom of a more general shortcoming of current architectures, or a localized issue?

Thanks, great work btw!

(A note from elsewhere)

A reminder that a state of the art GPU renderer can produce photorealistic scenes at 4k resolution (about 8000000 pixels) at 120 fps (i.e. render the scene in less than 8ms).

In other words, this performance is completely inaccessible to current DNN architectures (and I suspect will never be as efficient). As another algorithmic illustration, we have already near-optimal algorithms for sorting large lists of random numbers. I guarantee a large neural network even trained for massive lengths of time cannot beat the sheer efficiency of traditional algorithms. It should approach the asymptotics of the optimal algorithm, but there's too much overhead in how DNNs are structured. A better task, I believe, is to use DNNs to write code that improves renderers, shaders, or sorting algorithms while maintaining image quality (in the case of computer graphics) or correctness (in the case of sorting algorithms).

---

In this case the problem could be coming up with a variety of good realistic textures. I'm sure Google could hire a few artists to do a fantastic job, but if you want overkill I guess a GAN could generate the textures (to be rendered by a conventional algorithm).

With China being such a manufacturing powerhouse, I can imagine loopholes are essential to keep international business and trade in order.

> The device has to know it, in plaintext, in order to actually do anything

How so? You can use pins for encryption, I believe that's what tptacek was referring to.

If you use a strong key derivation function with an efficient hardware (i.e. within a few orders of efficiency limits of current generation), while using maybe 10J of energy, will economically protect at least about $1000, or $16000 with an 8 character pin. In practice significantly more because of hardware costs for a parallel attack, and the casual cracker wouldn't be willing to spend that much on a totally uncertain reward. At moderate hardware costs it could take years to crack.

Correct me if I'm wrong, but I don't think AMD physically swaps out defective cores. I believe they're disabled individually in some kind of firmware. That's effective because the interconnect region is significantly smaller than the core area (and possibly made more reliable through feature size manipulation?). I think this has been standard practice almost dating back to multi-core introduction, where they sell high end multi-core chips as low end with some cores disabled.

Well as simulations improve that tends to be less of a problem. Engineering by now is highly reliant on simulations, building actual prototypes is only done as a last validating step as you mentioned (specially for something as expensive and complicated as a fusion reactor). So validation devices are a O(1) step in the innovation chain, I don't see a scaling issue here. You can also simply make arbitrarily many experimentation/testing devices.

I think a greater issue is the general misconception of intelligence as something of a magical attribute, and even more so "superintelligence", and even more so "recursive-self-improving superintelligence" (RSIS for short?). There are not only various limits to hardware, some of which discussed in the article (size of atoms, capacity of communication between parts), but also there are limits to software. For example, while it seems that an RSIS could really solve any problem it wanted, Turing's almost century-old Halting Problem solution already proved that no such computer exists. There's no algorithm that takes an arbitrary conjecture, e.g. the Riemann hypothesis, and outputs either {yes,no,malformed_problem}. In fact the general expectation of what AGIs can do (think creatively, solve arbitrary problems, replace mathematicians, replace programmers) are things that we ourselves can't really do. We solve problems using a small set of heuristics and lots of trial and error, with no success guarantee.

Another fact is that there are many asymptotically optimal algorithms that are already know, and the trivial fact that almost all tasks performed by this super-intelligence (many of which are ordinary tasks we already do, analogous to maybe sorting, database queries, and search engines) will just improve constants on near-optimal algorithms.

A god-like creature those AIs will be not.

Oh interesting. Like I wrote, I'm aware there are certain problems that can be verified efficiently, that is, whose computation time is much larger than verification time, ver_t/comp_t<<1, but is it possible to take a generic program and offload it with (1) cryptographically secure verification afterwards and (2) ver_t/comp_t << 1?

As a challenge, take the problem of computing 1000000 iterations of a memory-hard hash function, such as scrypt or argon2d. If someone presents you a hash (of length N), the best you can do to verify it is actually compute the iterated hash function (assuming it has no cryptographic weakness). This would prove, in particular, that my required method is impossible (i.e. there are at least some problems that can't be efficient offloaded).

What kind of problems can the technologies you cited offload?

Well, but if port knocking was a great effective security tool, everyone would be using it, which defeats the obscurity advantage and would bring it back on par with good old passwords.

It might be a decent extra tool for very delicate situations, but I find it reasonably clear that it's largely self-defeating and not to be relied upon. It can also introduce further bugs.

I can see it could have an appeal as a proper password system though (not an obscurity device), since it's really the first step in interacting with a server. With password protocols, you usually have a more complicated interaction that can open a greater surface area. It seems good practice to authenticate absolutely as soon as possible.

I agree with you. Our current results are impressive enough for me to believe we're not that far from AGI (of the kind that would satisfy the author here). This article is already largely outdated because of advances in Reinforcement Learning and other Deep Learning applications that show very strong primordial features of what he claims as epitomes of human capability. For example, trained RL agents will recognize objects without being able to label them, having internal representations for position, object "type", functionality and relevance. You just have to train the agent with a problem that requires this kind of comprehension for efficient performance (kind of comprehension which isn't needed in simple image labeling).

AlphaGo and even labeling are indeed milestones in superhuman performance, and I believe Terence Tao's view that AI is a moving target is relevant here.

---

1) Regarding the Fermi paradox, I haven't read too much technical or scientific work on it (it's highly speculative anyway), but I find Isaac Arthur's videos lay the basic ideas pretty well: https://www.youtube.com/watch?v=oIva_60l3ww&t=1016s

It becomes pretty convincing that technological intelligent life is an incredible coincidence. There might not be any huge "filters", as they are called (stages which reduce the probability of observing TIL), such as a superintelligence extinction event, but an enormous sequence of minor filters, ranging from low probability of a planet with adequate conditions, to low probability of actual technological development, to low probability of occurrence in our light cone (we can only see fairly young distant galaxies), and limitations to the visibility and spreading velocity of such civilizations.

There is a wide range of parameters such that this does not contradict the generalized Mediocrity principle, such that there is probably more than one TIL in the universe, but they're few and far from each other in space and time.

2) I don't follow. This objection is only valid for discovering new laws of physics -- technological development can happen even with fixed knowledge of basic physical laws. At this stage it's not clear that even continued refinement of physical models. For example, the discovery of quarks certainly helped understanding nature, but it hasn't led to any direct technological applications due to quark confinement, and the fact that the particles are just too small. Neutrinos are another conceptually important discovery that doesn't really have applications due to low interactivity with matter.

Why not just a password? (you can also lock out after X incorrect attempts) I don't see benefits except for additional security through obscurity factor, which crypto wisdom dictates should not be relied upon.

Interesting, can you give me a good link on this?

Overall I'm skeptic (even of PoUW). That's because verifying work is usually almost as difficult as computing. I believe the only true solution to this is Fully Homomorphic encryption (FHE), but known constructions so far have had absolutely massive constants.

Is there enough demand of highly-assymetric and verifiable NP-hard like problems?

EVM's efficiency is abysmal, since operations are replicated almost everywhere.

They are on the early stages of developing sharding/"sub-chains", but it's still unclear if it's even viable. And it will never be a truly efficient computing platform -- for them it must be completely secure and only cheap enough to enable the most interesting distributed apps, which is a different beast from PoUW discussed here.

Wat. The Central-West region of Brazil which is primarily agricultural produces less than 10% of Brazil's GDP. I'd say given the possibly disastrous local climate consequences from destroying most of the Amazon, and similar total area, we could say that at absolute best case we might achieve a similar gain. Which would grow our economy by at most 10%.

Destroying the world's most biodiverse ecosystem (seriously, read some statistics on number of species in the Amazon -- it's mind boggling) to grow the GDP by 10% is not just incredibly selfish (supposedly exchanging a global treasure for national gain). It's not even the best for our own interests; it's myopic, greedy, inhumane. Even the actual direct economic value of scientific research with industrial applications such as biomimicry research, plants with applications in medicine and food industry, etc. would completely dwarf a one time 10% gain. And then there's the hardly measurable value of lost knowledge and natural richness.

Don't be the child exclaiming "But my brother was greedy too!". This is our treasure, and our responsibility.

It's also possible to binarize DNNs to use the faster bitwise operations

https://arxiv.org/pdf/1602.02830.pdf

The question is, how many people will move there, and how are they going to pay for it? (if they are not billionaires, of which there are maybe ~2000 in the world)

I agree that maybe a handful might go there because they can. But very soon you need to start producing enough to at least keep yourself alive. So much that I believe we ought to start developing those technologies (which takes time) before other steps.

We can probably fabricate regolith-lookalikes on Earth and start doing basic chemical and industrial engineering for pretty cheap, much cheaper than the billions you need for a mission.

Edit2: Forget everything I said :) My geometry is quite rusty I guess. Not only any non-isotropic metric cannot have an Euclidean equidistant projection, any non-Euclidean metric clearly can't have an Euclidean equidistant projection. Metric spaces are defined by pairwise distances! What you can do however, is make the distance to 1 or 2 arbitrarily chosen points correct.

First of all, I believe that [the dream of] just making a colony on Mars, per se, is obviously pointless, and actually wasteful. Per se, a small research station would be much better, and could be productive from a global utilitarian perspective.

However, things change when speaking of an economically self-sustaining base; and eventually a base theoretically capable of full-self sustenance (not dying in case there was a transport disruption or Earth itself suffered a disaster). This simplifies the funding discussion to start up costs and r.o.i.

In that case, humanitarian-utilitarian arguments such as expanding the Human civilization to other planets (Musk's call for making life multiplanetary), backing up life, etc. also start making a little more sense (if the colony isn't self-sustaining all of those points are moot).

Is that feasible? Certainly not today, but perhaps in 50 years?

I mean, a civilization doesn't actually need that much to sustain itself. The basic needs are (a) energy, and (b) materials. Obviously Mars receives Solar power (about 45% as much as Earth per m^2), and it has materials (all basic industrial elements are present in the crust).

The question is, can we start an industry on Mars that is efficient enough today such that the cost of producing and maintain infrastructure (solar panels, mining equipment) is lower than the output from the products? If your solar panel gives 10MJ during it's lifetime, you need to use at most 10MJ (used in mining and manufacturing) to produce it. That's an engineering analysis, and I'm it will be eventually feasible. It also improves with economies of scale.

Perhaps you can go even further and try to make it so that any two points have pairwise correct distances, like in the Azimuthal equidistant projection?

https://en.wikipedia.org/wiki/Azimuthal_equidistant_projecti...

Perhaps there are metric functions where this is not possible (a quick proof eludes me...[1]), but approximations might be possible anyway.

[1] Edit: actually it's quite simple. Any non-isotropic metric cannot have an Euclidean equidistant projection, simply take a pair of points lying in rays (1,0) and (0,1) in an L^p metric, p not 2, as a counterexample.

It is, but I'm not trying to replicate religious experience at all. In particular I don't care about mysticism, regardless of the range of emotions it evokes.

I've been thinking of ways of making some sort of rigorous analogue (in the sense of obeying a small set of properties) to higher dimensional manifolds, like spheres and cubes.

For example, an unit sphere inscribed inside a high dimensional unit cube looks a lot like an astroid[1] inscribed in a square in 2D with regards to metric properties (using an L^2 metric):

- The astroid touches the cube, while it is very far to the cube in the direction of the corner

- We can see that the corners, being far from the unit-L^2 distance manifold, occupy most of the volume

- As the number of dimensions increase, the astroid becomes increasingly compressed near the origin

[1] https://upload.wikimedia.org/wikipedia/commons/thumb/0/03/As...

Yes, CRTs used electric fields to accelerate electrons, no such things with LCD/LED displays. Not to mention those claims are probably bogus.