Leslie Lamport revolutionized computer science with math [video]
youtube.com
youtube.com
A discrete decision based upon an input having a continuous range of values cannot be made within a bounded length of time.
Quoting from https://lamport.azurewebsites.net/pubs/buridan.pdf:
"The significance of Buridan’s Principle lies in its warning that decisions may, in rare circumstances, take much longer than expected. Before the problem was recognized by computer designers, some computer systems probably failed regularly (perhaps once or twice a week) because arbiters took longer than expected to reach a decision. Real accidents may occur because people cannot decide in time which of two alternative actions to take, even though either would prevent the accident. Although Buridan’s Principle implies that the possibility of such an accident cannot be eliminated, awareness of the problem could lead to methods for reducing its probability."
In the accompanying notes at https://lamport.azurewebsites.net/pubs/pubs.html, Lamport states:
The four reviews ranged from "This well-written paper is of major philosophical importance" to "This may be an elaborate joke." One of the other reviews was more mildly positive, and the fourth said simply "My feeling is that it is rather superficial." The paper was rejected.
Wouldn’t the Sampling Theorem give an answer in some bounded time? Is the idea that the time required to crank the algorithm can grow without bound?
"A discrete decision based upon an input having a continuous range of values cannot be made within a bounded length of time."
But certainly I can make a discrete decision in a constant amount of time, just choose to always go left. I must therefore be missing something here.
The next possibility is that a discrete decision procedure that guarantees success is not possible in an unbounded amount of time. This is more sensible and I think the idea is that there will be some kind of infinite regress. For example a donkey will starve to death if it doesn't eat in exactly 100 seconds. The donkey can choose to walk left X meters, or walk right Y meters to some food.
There are certain values of X and Y where the donkey dies no matter what, if X and Y are sufficiently far away that the donkey can never get to them in 100 seconds then the donkey dies.
There are also certain values of X and Y where the donkey can pick either of them and survive since the donkey can get to X or Y in well less than 100 seconds, so the donkey lives.
The above two scenarios are boundary conditions of sorts, where it's fairly trivial to come to a decision, the principle is about what happens when there is a value of X where it takes almost exactly 100 seconds to get to X, (and assume way more than 100 seconds to get to Y), but in order for the donkey to come to that realization the donkey needs to spend some time thinking about it and by the time the donkey has made a decision the donkey won't have enough time to carry it out. So the donkey has to take into account not only the amount of time to get to X, but also the amount of time it takes to come to a decision to get to X, but that too takes some finite amount of time... so the donkey has to take time to come to a decision about how long it takes to come to a decision to get to X, so on so forth... and you end up with an unbounded amount of time.
I could be way off here but I feel there is some subtlety in the way this principle is described that I'm missing.
> "A discrete decision based upon an input having a continuous range of values cannot be made within a bounded length of time."
"A discrete OPTIMAL decision...."
The range must be discrete and at least 2 possible values.
There is nothing about optimality at all here. Even if both possible outputs are equally "optimal", there is no procedure to pick one in a finite amount of time.
If an algorithm takes an input, continuous or discrete, discards that input and returns a constant value, that algorithm is just as "legitimate" as any other algorithm in so far as being an algorithm is concerned. It may not produce the optimal output for some given cost, but it is a formal and rigorous decision procedure that manages to output a discrete value given a continuous input in a bounded amount of time.
At any rate, reading the comments when this was last posted it looks like the issue has to do with producing the optimal output in a given time frame. If an optimal discrete decision must be made in the next T seconds on the basis of a continuous input X, then there always exists some value of X that requires more than T seconds to compute. This will be true for any value of T regardless of how large.
Looking at it macroscopically, highly complex decisionmaking such as the one that the brain produces, has continuity. In chaos theory, a double pendulum may, after a minute, be either on the left or the right depending on tiny changes in initial state; but changing the initial condition continuously, will change the state at the minute mark continuously, and so, the decision.
Looking at it microscopically, even CPUs are continuous, as there is a slim chance of a transistor only producing half the current, because of the quantum behavior of electrons.
It is conceptually weird that there must be a chance for permanently indecisive animals. To me, the true resolve of the paradox is superdeterminism: the choice taken had to happen.
I am glad that paper got rejected.
The algorithm itself is possible. If implemented in a continuous machine, though, it is not guaranteed to reach either point 0 or point 1 in any given timeframe.
In practice, it usually does not matter. One application I can think of, though, is the surprisingly powerful fault injection vulnerabilities which can break the physical implementation of a cryptosystem: http://euler.ecs.umass.edu/ece597/pdf/Fault-Injection-Attack...
“Always go left” satisfies the first (a discrete decision) but not the second ([decision] is based upon an input having a continuous range). If the decision is preordained, it may be interesting but not relevant to the problem at hand.
?
It sounds like you could easily avoid this. You can use a microprocessor and lead in analog input on one pin and then set an output pin based off that. You will always output a valid voltage. Sensors have noise anyways so it's not a big deal if the output is slightly wrong.
At first, I had trouble to believe it, and tried to find counterexamples, until I realized that the composition of continuous functions is continuous. I still wonder sometimes whether an infinite composition of continuous functions has to be continuous (limit of a sequence) if it represents some real world evolving process. Can it produce a discontinuity in the limit?
Anyway, Buridan’s principle informs our consensus process in Intercoin Protocol[1] [2] — it is why the consensus groups are always of a bounded finite size, so we don’t approach the errors in Buridan’s priciple (I think this is also called unstable problems).
1. Beyond blockchains https://community.intercoin.org/t/intercoin-technology-conse...
2. Experimental improvements https://community.intercoin.org/t/experimental-improvements-...
Anyone who reads the above … I would be very interested in your feedback on our forum.
No, as you expected. Take f(x) = x^2 on [0,1]. Then f(f(f(...f(x))) converges to g(x) = { 0, if 0 <= x < 1; 1, if x=1 }
Consider a pencil that is placed almost vertically, and then later drops to a horizontal position. There is a period where it is "unsure" where to go, and the more perfectly balanced it starts out, the longer this period is (assuming no wind etc.)
That's what is being discussed.
Then A(t,x) is clearly not continuous in x, and we can easily bound the time required to make a decision. The nuance here is that we have to somehow be able to distinguish 0.5 - eps from 0.5 for very small epsilon.
Edit: on further thought, suppose we had a device which measured reasonably well. More precisely it tells us x < 0.5 if x is actually <= 0.5 - c, it tells us x >= 0.5 if x > 0.5 + c, and tells us it is unsure otherwise. We do not know c, but it is deterministic (and hopefully reasonably small). Then we can decide to go left if it tells us x < 0.5, and right if it tells us unsure or that x >= 0.5.
If you ignore the problem then the problem indeed goes away. The need for distinguishing very small epsilon exists because of the continuity assumption, and because of the continuity assumption you can't really solve it either.
> Then we can decide to go left if it tells us x < 0.5, and right if it tells us unsure or that x >= 0.5.
Now you just moved the problem to deciding at which point you are unsure. As long as there is a decision to take the issue persists, it's only if you always go left (or always go right) that the issue doesn't exist.
That's not true. There are two problems.
1. The math is set up to rule out a lot of obvious solutions. Write out A_t(x) for the decision rule "always walk right at a constant rate." For any rate, A_t(x) becomes a spike at 1 for t > some threshold that depends on the rate.
2. The math rules out randomness. A_t(x) can't be defined for the decision rule, "flip a coin and walk right half the time, left the other half," because you wind up at 0 with probability 1/2 and 1 with probability 1/2 for t > some threshold and the problem is defined so A_t(x) must resolve to a single point.
Once you do that, the only solutions left are kind of mind fucks. But you can't draw any general conclusions from it.
Now, an inverted pendulum can be stabilized with a control system, and this might be possible to relate with adversarial inputs of some system.
Reminds me of one of the Boeing 737 crashes where pilots were reading the manual but had not enough time to reach the relevant pages.
It comes up all the time in multiport memories, where several CPUs are accessing the same memory, and there has to be an arbiter to decide who gets access access for near-simultaneous requests. Requests can be very close to simultaneous. If two CPUs with 3 GHz clocks are contending for memory access, and the clocks are not synchronized but close, about once per second they will be within 10^-19 seconds of whatever difference is most troublesome.
This was a serious problem with some early multiprocessor mainframes. Early in my career we saw this happening with multiprocessor UNIVAC 1108 machines. They had an unsound arbiter, and, once in a while, every few hours, they'd botch a memory access. Early on, the software stability was so bad the OS crashed more often than the hardware did, but as the OS got better, it became clear there was a race condition at the hardware level.
[1] http://async.org.uk/David.Kinniment/Research/papers/IEE1976....
[1] http://www.async.org.uk/David.Kinniment/DJKinniment-He-Who-H...
[2] https://arl.wustl.edu/~jst/cse/260/glitchChaney.pdf
edit: typo
"The 74F786 is designed so that contention between two or more request signals will not glitch or display a metastable condition. In this situation an increase in the BRn to BGn tPHL may be observed. A typical 74F786 has an h = 6.6ns, t = 0.41ns and To = 5µsec."
"If two or more BRn inputs are asserted at precisely the same time, one of them will be selected at random, and all BGn outputs will be held in the high state until the selection is made. This guarantees that an erroneous BGn will not be generated even though a metastable condition may occur internal to the device."
So, usually, you get a win within a specified time, but sometimes it takes longer. The limit on the longest time is probabilistic, but the odds of settling increase rapidly with time, by orders of magnitude per nanosecond.
In this part, we get to see a simple standalone arbiter with its own data sheet. A logic diagram is given, so you can see how this is built out of simple gates. Note that diagram can't be read as abstract logic; propagation delay matters. While the arbiter is in a metastable state, an inhibit signal is generated which prevents any output from appearing. Once the arbiter has settled, the winning output is allowed out. The gate thresholds matter. The inhibit signal doesn't turn off until there's only one clear winner.
It's an old part, from 1991, because today this function is usually part of a larger function such as a CPU chip or a DRAM interface.
[1] https://www.digikey.com/en/htmldatasheets/production/96092/0...
I understand that this is true for deterministic decisions. But a limited amount of randomness/noise is generally an acceptable alternative to a requirement for unbounded precision.
Lamport's paper covers this in the interrupt example: a computer must decide whether an interrupt line is signaled or not in a finite amount of time (the next instruction). The interrupt is continuous: it passes through some halfway voltage level like 2.5 between 0 and 5, and so it can be called the wrong way.
Basically the whole result is a restatement of real numbers not being computable.
The discrete sample value you get from an ADC is not based on knowing the underlying real number absolutely, and then choosing the nearest value. It's a time limited process of discovering the value partially. Since the value is not partially known, it is not known whether the reported sample value is the closest one.
Paxos is so beautiful. It’s a piece of art. It’s way more elegant and beautiful than Raft (as it solves the smallest consensus problem instead of a log). I will die on this hill.
https://www.youtube.com/watch?v=p54W-XOIEF8
[suddenly in a clown costume] "What kind of clown am I, claiming I can make you think better? Why should you pay attention to me?"
[in a suit] "This is not the time to be modest. I have done seminal research in the theory of concurrent and distributed systems for which I won the Turing award. You can stop the video now and look me up on the web." [long pause]
Raft is more understandable, because they wrote a paper that was easy to understand. You are much better off reading Heidi Howard to understand Paxos, than Lamport.
In 2016 some researchers at Stony Brook published the full (Multi-) Paxos algorithm using TLA+, including a couple of nice extras:
https://arxiv.org/abs/1606.01387
The spec is succinct enough to commit to memory and way easier to comprehend than Lamport's prose descriptions (Lamport has never specified the Paxos algorithm in TLA+, although you can find his TLA+ spec for Paxos consensus). The paper also includes a mechanically checked proof and an interesting overview of other related work.
In my career I have seen that people who are true geniuses are also very humble!
My favorite is on "Time, Clocks and the Ordering of Events in a Distributed System", where he applies the lessons of special relativity to understand computers, and he says:
> Jim Gray once told me that he had heard two different opinions of this paper: that it's trivial and that it's brilliant. I can't argue with the former, and I am disinclined to argue with the latter.
I came to the conclusion that the designer of a new system must not only be the implementor and the first large-scale user; the designer should also write the first user manual. - Donald Knuth, 'The Errors of TeX' (1989) https://yurichev.com/mirrors/knuth1989.pdf (4.8MB)
I also used LaTeX heavily in the 80s so was surprised to see him pop up as a genius of distributed systems later (although that work was published much earlier it didn't get much exposure until the 90s). Like "oh that guy must be _really_ smart to excel in two quite different fields".
e.g.
https://en.wikipedia.org/wiki/Join_and_meet
https://math.stackexchange.com/questions/1775926/when-is-a-j...
https://www.microsoft.com/en-us/research/group/research-soft...
https://assets.amazon.science/07/6c/81bfc2c243249a8b8b65cc21...
Nice...
I don’t think someone at his level becomes dull at such a young senior age.
I, on the other hand, am amazed by the progress being done today, and the research paper released on an almost weekly basis by companies like Google, Microsoft and co. Just look at the language and image models that are released, the progress on computer vision etc.
I played with the playground of OpenAI GPT-3 and I am blown away at the result. It's not perfect, but it's orders of magnitude better than what we had easily access to just few month ago. I have started using it in my day-to-day life on some specific tasks, and I can't wait to see the next versions. I can't even start to fathom the amazing products people are going to build around it in the next decade.
It's absolutely true that these models/research serve the purpose of the company building them... but even Leslie says in the video that he did his entire career in the industry because that's where he saw many interesting challenges, and that his whole research was a means to an end.
R&D is still alive and kicking, and is going faster than ever. You might just be looking at the past with rose tainted glasses, and needlessly undermining the present.
Now the next step is for somebody who understands their writings to teach us unwashed masses :-(
It's really interesting because it only uses hash function and doesn't rely on trapdoor function! I think quite many people (me included) only learn about digital signature after public key cryptography, so it's quite a surprise to know about Lamport signature. Also, IIUC, because it doesn't rely on trapdoor function it also resistant against quantum computer in the future
I am not sure what he means by maths? Exactly which topics in maths?
That's very Djykstra. Read Djykstra's "A Discipline of Programming" (1976). Programming is hard and programmers should suffer.
The trouble is, suffering doesn't scale. Academic computer science was a tiny field before the mid-1980s.
Alt-x [package-install] (press intro) [geiser-guile]
Alt-x [package-install] sicp
Alt-x [geiser]
Alt-x info (press Intro) Ctrl-s SICP (press Intro).
(Press Ctrl-x b) to switch between the buffers, or just use the menu entry in Emacs.