arnioxux

Pretty sure the answer is no as a math problem where f is pure. If f returns the same answer each time, the square ending at N,N contains [1, N * N]. Then the rectangle ending at N,N+1 must contain (N * N, N * (N+1)] in the nonoverlapping strip. Ditto for N+1,N. But then for N+1,N+1 all other locations must contain low numbers and there's only a single corner cell left which can contain the rest of the missing numbers between (N(N+1), (N+1)(N+1)]. Unless I made a mistake I don't think any function would work here for N > 1. What were the examples you found?

OTOH as a programming problem you can just cheat and store state somewhere to count the row width. This satisfies your interface requirements (python):

    lastJ = None
    def f(i, j):
        global lastJ
        if i != 0:
            return i * (lastJ + 1) + j
        else:
            lastJ = j
            return j


    N = 3
    M = 5
    seen = set()
    for i in range(N):
        for j in range(M):
            q = f(i, j)
            seen.add(q)
    assert sorted(seen) == list(range(N * M))

> My favorite was the magnetically levitated sphere in a bowl

I think you might be talking about CMU's Magnetic Levitation Haptic Interfaces: https://youtu.be/cMi75SrDbsk?t=12

I've used it before and it's pretty cool. It's strong enough such that when you hit a hard virtual surface it actually feels hard, and precise enough that when you drag across a brick-like surface it actually feels rough. Unfortunately I remember it having a tiny range of motion.

For your jointed arm "handle", I think they've managed to find a niche in surgical training simulations. I see a lot of similar products come up searching for 6DOF haptic devices (e.g. Phantom Premium 6DOF, Geomagic Touch, Force Dimension Omega 6)

Is your "winning" strategy so different from other social data / sentiment analysis approaches? There is some novelty with how you weigh the sentiments (based on how much of an insider or expert they are) but I am sure existing trading strategies weren't just taking a dumb average of a twitter firehose either. Shouldn't it be easy for some large firm to replicate your approach and make the alpha disappear?

The most damaging advice I ever got in my life was to talk to myself.

I have always thought in mostly visual/symbolic terms. This made me really good with math early in my life, but it started failing when I began proof writing. Then I got the advice to "explain the problem to yourself and the solution will come naturally" from a friend who is really gifted at mathematical proofs. (And also later received similar advice in programming in the form of "rubber duck debugging").

Surprisingly, it works remarkably well and probably increased my problem solving skills tenfolds, or at least my ability to communicate the answer. I started using it for everything.

But it comes with the side effect of gaining a voice in your head. And that voice is a fucking vicious asshole to me. In the recent years the thoughts even started subvocalizing and I have to forcefully remember to make it stop. It's super embarrassing when there are other people around you.

I am sure this problem is unique to me because I have never heard someone describe it (other than partially matching symptoms of schizophrenia or tourette). But I am certain in my case when the voices started and I am not sure the superpower problem solving skills are worth the trade.

There are firsthand accounts from women who did compete successfully on why it's harder for them (for example first answer is from a female IOI silver medalist):

https://www.quora.com/Why-dont-more-women-participate-in-pro...

I watched these lectures a few years ago and they were really good.

Since this is meant to be a graduate level research course, I wonder if any cutting edge topics have been added since? Were there any major advances or new ideas in fundamental data structures in the past few years?

I wonder how they are going to handle security/abuse? Once the training is on end user devices, can't the user give fake results?

For example back when recaptcha was introduced, trolls tried to transcribe everything as the n-word (since you only had to get one of the two words correct).

These cases can obviously be noticed and fixed but it's harder now that the training data is opaque right?

I write a lot of scientific code with python and in those cases you really want your equations and matrix columns aligned for readability. These matrices appear so often adding fmt:off destroys the readability/elegance gain from the clever formatting.

Here's an example snippet by Peter Norvig that is beautifully formatted:

  def neighbors4(point): 
    "The four neighboring squares."
    x, y = point
    return (          (x, y-1),
            (x-1, y),           (x+1, y), 
                      (x, y+1))

  def neighbors8(point): 
    "The eight neighboring squares."
    x, y = point 
    return ((x-1, y-1), (x, y-1), (x+1, y-1),
            (x-1, y),             (x+1, y),
            (x-1, y+1), (x, y+1), (x+1, y+1))

For example Peter Norvig does an exemplary job with jupyter notebooks for expository writings: https://github.com/norvig/pytudes

Using ln as our utility, at time N we get:

  E[ln(X_1*...*X_N)] = sum E[ln(X)]

So kelly happens to maximize this expected utility by construction since it was derived by maximizing expected log of one round of betting (E[ln(X)]).

But if we use square root as our utility instead:

  E[sqrt(X_1*...*X_N)] = prod E[sqrt(X)]

We would maximize this expected utility by maximizing E[sqrt(X)]. Going through the same calculus we can see that we don't arrive at kelly.

Where did I go wrong?

I think the log is just the utility function. You could substitute it with any non-linear utility function and it would've gave you another answer that makes sense for that utility.

The main takeaway is that a linear expected utility doesn't make sense.

It would've told you to bet all your wealth every game, which does result in a higher linear expected value, where you win (1+1.1)^N with probability 1/2^N at time N, but 0 otherwise. But no real human would take the bet of extreme high payoff at extremely rare chances with ruin otherwise.

Also see St. Petersburg paradox for a similar "paradox" resolved with expected utility theory.

Tangentially related, this reminds me of the spy recordings[1] between incarcerated German nuclear physicists and their reactions to the atomic bomb dropping.

At first they were incredulous and thought it must be a bluff. The bomb should be "impossible", with Germany failing at their own nuclear project a few years earlier. Then they slowly worked out how it could have been done, and realized that the americans must've had hundreds of thousands of people working on it. "Which is a hundred times more than we had" This was follow by a lot of regret over what they could've done better and the various implications of a world where such a bomb now exists.

I originally found that link from a tweet[2] by someone working at OpenAI. I am sure AI scientists are feeling similar anxiety about their research.

It's easier than ever for someone with a hundred times your computation resources to achieve things that are supposed to be "impossible", at least to the unsuspecting public who haven't grasp the rate of progress in AI.

And I am not even talking about some massmurdering AGI. It's the boring stuff like astroturfing chatbots whose sole purpose is to psychoanalyze individuals to manipulate voting behavior that scares me. This asymmetric power might already be available to those who are willing to throw a few hundred GPU years at the problem and I am not sure how the common man can defend against it.

[1] "Transcript of Surreptitiously Taped Conversations among German Nuclear Physicists at Farm Hall, August 1945" http://germanhistorydocs.ghi-dc.org/pdf/eng/English101.pdf

[2] https://twitter.com/karpathy/status/778286393441169408

I wrote a similar comment before, viewing redux as a realization of the event sourcing pattern: https://news.ycombinator.com/item?id=17061827

You can go even further and use the exact same "events" or "actions" for event sourcing on the server side too. You just need to save the list of actions to the server. This lets you do realtime sync and multiplayer really easily!

One thing people don't realize is that a lot of web analytics software already do "session recording". So it can playback every wiggle of your mouse, every scroll, every keystroke, etc.

You can see some creepy demos here:

https://www.inspectlet.com/

https://www.fullstory.com/

> So they trained it using 100 GPUs (100 GPUs dear lord!), and got no difference until fourth decimal digit! 100 GPU's to get a difference on fourth decimal digit!

That's hilarious!

But I found the criticism on their toy task less convincing. Algorithmic toy tasks can always be solved "without any training whatsoever".

For example in RNNs, there's a toy task that adds two numbers that are far apart in a long sequence. This can be solved deterministically with a one liner, but that's not the point. It's still useful for demonstrating RNN's failure with long sequences. Would you then call the subsequent development to make RNNs work for long sequences just feature engineering with no universality?

In that sense, I think their choice of toy task is fine. They're just pointing out position is a feature that's currently overlooked in the many architectures that are heavily position dependent (they showed much better results on faster r-cnn for example).

His previous post was probably the best overview on program synthesis paradigms (circa 2015):

https://homes.cs.washington.edu/~bornholt/post/synthesis-exp...

It's a much broader overview of the field so read that first if you need an intro!

I saw something similar before (from gifs.com's sticker editor of all things lol) where you annotate the segmentation of the first frame of the video and it will propagate that segmentation to the rest of the frames:

https://medium.com/gifs-ai/interactive-segmentation-with-con...

If you're already using react/redux, there's one really nice pattern to make eventsourcing easy to implement. I learned it from the boardgame.io google project[1].

Basically redux is more or less already doing eventsourcing on the client. You have actions("events") that tells your global state what the next state should be, and replaying the history of all actions will deterministically get you to your current state. The only thing you need to do is persist that history of actions in your state also (which is already commonly done for implementing undo and time-travel).

Then all that is left is to make your server use these same events too! This means whenever you save to the server, rather than doing GraphQL/REST-style per resource updates, you just sync any unsaved redux actions from the client. The server will save these actions and optionally replay them to get a snapshot of current state (or don't - that's just an optimization. The queue of actions is the first class citizen now).

Afterwards realtime collaboration, undo/time-traveling, history/audit logging, etc all come for free.

This is a really nice way to unify event shape design that is consistent on client and server.

[1] https://news.ycombinator.com/item?id=15946425 or https://github.com/google/boardgame.io.

(I am making it sound a little simpler than it is. There's a lot of details to get right with state design, tagging actions as relevant to be persisted, multiplayer conflict resolution, app versioning issues, event queue compaction, serverside permissions validation, etc. But to get a prototype up and running quickly what I said above will work. Boardgame.io didn't worry about those details but it's still a nice codebase to study)

This visualization is really cool and innovative! It's useful for seeing the changes in the stream of values even if they are small relative to absolute magnitude (e.g., 1000001 -> 1000005 -> 1000003 etc).

The tradition way to solve this is to rescale the y axis (for example google stock price chart ranges from 1000-1200). But this doesn't work for graphing IO because it spikes up/down too often in a huge range.

Your visualization plots the base-1024 digits so whenever the IO is sustained at a certain range, the top few digits will be held constant and the next digit will show the variation within that scale.

Unfortunately I don't think this solves the spiking problem completely. It's mentally hard to parse since two adjacent values are only comparable whenever their top digits match.

For example it's only meaningful to compare (1gb+2mb+3kb) with (1gb+2mb+5kb) on the kb graph. If it spikes down to 0gb+0mb+0kb, you need to switch to reading the gb graph (since comparing the kb value to one before is pretty meaningless unless for some crazy reason you actually care about the mod 1024 value of two very different numbers).

To fix this, you should color code the "runs" of values that are meaningful to compare! In other words, kb values have the same color only if their gb/mb values match. And mb values have the same color if their gb value matches.

I hope http://fabiensanglard.net will do a code review for this!

That's a really elegant way to put it. Yay for information theory.

> Robust? Yes. It would take virtually forever to run out of space through repeated list reordering.

I think the author missed a pathological case with using the Stern-Brocot tree.

The goal is to sort each item with a score. Then to insert something between two items, you calculate a new score that is "between" their existing scores.

- Option 1) Take the average of the two score a/b and c/d:

  (ad+bc)/(2bd)

- Option 2) Take the mediant(https://en.wikipedia.org/wiki/Mediant_(mathematics) ):

  (a+c)/(b+d)

We don't want to use the first option, the average because almost by definition you would require at least another bit of precision in the denominator for each insert due the multiplication by 2. This limits you to a meager ~64 repeated inserts before overflowing the denominator.

So the mediant is better because the precision requirements is limited by the number of times you can add instead of multiply. This might seem to grow a lot slower at first. And the author even showed that if you're repeatedly averaging 0/1 with the new score, you would get the pattern 1/2, 1/3, 1/4, 1/5, 1/6, .... Which means you can handle up 2^precision number of inserts which is much better than before.

But this only applies to inserting to front (medianting with 0/1) or back (medianting with 1/1) because this will only add a 0 or 1 each time.

If you go down a zigzag path down the middle of the tree instead, you can get the two numbers added to be much closer in magnitude and explode. For example: 1/1, 1/2, 2/3, 3/5, 5/8, 8/13, 13/21, 21/34, ... (it's fibonacci!).

So essentially you get the same thing as the averaging case where the precision required roughly doubles(1.6180339887...) at each step and it will overflow quickly.

This isn't a purely theoretical edge case either. The example from before can be realized by repeatedly inserting between the last two items inserted, which arises naturally if you're repeatedly adding to the middle of a list.

In the first few lectures of the course you get a pretty good history of deep learning and you'll see it didn't really take off until around ~2012. And the reasons for it taking off is mostly because people are getting better at the black magic of training a deep network.

So these grad students are exactly the people you want to learn from because they have done the dirty work of fiddling with parameters to know what tricks work and what doesn't. It's probably preferable to a more theory-heavy course because very few people (not even the more experienced professors) understand why those tricks work.

Note: I took an older version of the course which was started by Andrej Karpathy who was a grad student at the time but is now the Director of Artificial Intelligence at Tesla.

I second his recommendation of CS231N: http://cs231n.stanford.edu/

You can probably go through the whole thing including assignments in under a week full-time.

When I teach others, I usually rely on a lot of metaphors/analogies.

For example if I had to describe gradient descent to someone, I wouldn't start with a definition from calculus. I would instead describe it as climbing down a hill.

But that assumes prior knowledge of what a hill is (it has gradients) and what climbing is (you wanna go to the lowest point) and what the physics is like (go towards steepest direction with some momentum). This isn't something that I expect a blank-slate AI to be able to produce.

In short, explaining things intuitively to another human requires knowing about common human experiences and mode of thought. It's the "transfer learning" for humans. This seems like the realm of AGI and not something I expect to see solved well anytime soon. I hope I am wrong!

(Note: even in this example I assumed you understand the experience of trying to explain gradient descent in order to explain my point)

I think there is someone working on a webgl version: https://tenso.rs/

There are a lot of rapid prototyping tools for computer vision usually done via some dataflow interface:

http://imageplay.io/

https://www.adaptive-vision.com/en/software/

http://www.adrianboeing.com/improvCV/

https://github.com/utwente/parlevision

I don't use any of these but I see a few of these reinvented every year with not much differentiating them from each other. I wonder why none of them caught on yet?

I think they are useful compared to the tedious workflow I am stuck on (prototype with python notebook to dump visualizations and play with hyperparameters, then port to C++ later). But the fear of getting trapped into some unsupported edge case and having to deal with a messier port usually stops me from changing.

Are the language/syntax differences documented somewhere? It's difficult to tell if something is an obscure early stage ecmascript proposal or a language feature specific to observablehq. For example in https://beta.observablehq.com/@mbostock/introduction-to-note...

  import {canvas as flood} with {height} from "@mbostock/randomized-flood-fill"

or how you define generators

  i = {
    let i = 0;
    while (true) {
      yield ++i;
    }
  }

etc

> 3: Arguments-from-extreme-upside sometimes do work

I have seen a bunch of less wrong articles on pascal's wager and variants so I assume they are familiar with them:

- https://en.wikipedia.org/wiki/Pascal%27s_Wager

- http://lesswrong.com/lw/kd/pascals_mugging_tiny_probabilitie...

- http://lesswrong.com/lw/h8k/pascals_muggle_infinitesimal_pri...

With that fallacy in mind, is bitcoin still a good call? It could've been any other cryptocurrency winning and they all satisfy the same tiny probability with astronomical returns. It doesn't make sense to dump all your resources onto every new coin that pops up. Just like how you wouldn't adopt every religion to guarantee that you will never burn in any hells.

I don't really get it either so here's an attempt to justify it with gambler's ruin: http://mathworld.wolfram.com/GamblersRuin.html

> Let two players each have a finite number of pennies (say, n_1 for player one and n_2 for player two). Now, flip one of the pennies (from either player), with each player having 50% probability of winning, and transfer a penny from the loser to the winner. Now repeat the process until one player has all the pennies.

> The chances P_1 and P_2 that players one and two, respectively, will be rendered penniless are

> P_1 = n_2 / (n_1 + n_2)

> P_2 = n_1 / (n_1 + n_2)

So if the current order book looks like:

- Sell 5 for $99.00 (best offer)

- Buy 10 for $98.75 (best bid)

And you chomp up orders randomly, flipping a completed order to the opposite position:

- the probability that the price is above $99.00 (all 5 sell got filled first) is 10 / (5 + 10)

- the probability that the price is below $98.75 (all 10 buy got filled first) is 5 / (5 + 10)

So expected value is (99.00 * 10 + 98.75 * 5) / (10 + 5) = 98.9167.

I am not happy with this explanation. It seems like after you execute an buy for $98.75 you need to immediately put a sell for it at slightly above $98.75 (and vice versa) to fit the random walk model described above. And then I took the expected value using the price at two different points in time. Overall I am still confused myself.