cscheid

In terms of "in general", there's mapd, a startup doing something on a very similar space http://mapd.com/

It's definitely the case that our work is not ready to be shrink-wrapped and sold. But the need for it is there. I'm no longer at AT&T Research and can't answer for their current needs, and I unfortunately cannot talk about the usage scenarios when I was there. There's the academic paper you can read, http://nanocubes.net/paper.pdf, which has a few examples.

In practical terms, the difference in a system that has latency of a few seconds to one in which the queries come back in 60fps is significant, and enables a bunch of interesting stuff that might not be obvious at first sight.

(author here, sorry I didn't see this earlier)

One of the principles we tried hard to follow is that after building the indices, the query times should be proportional to the resolution of the plots you're generating.

So the difference here is building the data structure that's fundamentally well-suited to interactive visualization. So we push hard to get low latency, and we take trade-offs that will let us generate query results well-suited to create histograms , heatmaps, etc. These are typically multiresolution (sometimes you want mile-wide pixels, sometimes you want yard-wide-pixels, or day-wide pixels vs second-wide pixels) and require some index-building tricks that we hadn't seen combined in the way we did it.

(Author here, sorry I didn't catch this earlier) The 3.5 minutes is the preprocessing time (think of it as index-building time).

After the index is built, the goal is to be able to generate outputs at a time bounded by (up to a poly-log loss) the size of your screen, and not of your data.

After the index is built you get sub-millisecond answers for the 2-D histograms over reasonably-sized screens, and that turns out to be pretty useful for the use cases we're going for.

You can write composable WebGL, but it takes more work. Here's my Loop-Blinn example: http://cscheid.github.io/lux/demos/loopblinn/loopblinn.html

The bits that do Loop-Blinn are here, https://github.com/cscheid/lux/blob/master/src/lux/text/outl..., while the bits that decide the color of the text go elsewhere.

Look at 'unconed's MathBox for this kind of thing taken to incredible heights, or https://github.com/kovasb/gamma for the ClojureScript analogue.

That turns out to be exactly what I did, but that type of opportunity is now much rarer than it was even 10 years ago. MSR is the stalwart "you-can-publish-here" industrial research lab, and look at what happened to MSR SV -- to say nothing of other smaller places, also shrinking.

Myer and Sutherland wrote a classic 1968 paper on what they came to call the Wheel of Reincarnation: simple displays accrue progressively more complexity until someone comes along and throws the whole thing away with a new, clean, design.

Then someone finds that they can add a bit of processing to that display to make it go just a bit faster...

http://cva.stanford.edu/classes/cs99s/papers/myer-sutherland...

The graph you linked to. I'm comparing the 1984 numbers to latest available, since 1984 is what the linked piece talks about (and which I mentioned in the original comment)

> drop off was nearly identical for men

> women did seem to drop off more

These two statements cannot be both true at the same time.

Read the graph again. The dropoff for women is much larger.

$ python

>>> 20000.0 / 17000.0

1.1764705882352942

>>> 12500.0 / 7000.0

1.7857142857142858

CS enrollment for women in 1984 was 78% larger than in 1996; For men, it was 17%.

David Deutsch also proposed a similar resolution for closed timelike curves, which say that if you have a probability distribution of the present, all that's necessary is that when you travel back in time, you get the same probability distribution. In short, if you flip a (true!) random coin to before shooting your grandfather, no paradox exists:

http://journals.aps.org/prd/abstract/10.1103/PhysRevD.44.319...

This is described with more intuition and background in one of (always excellent) Scott Aaronson's lectures: http://www.scottaaronson.com/democritus/lec19.html

That's very much not what quantum computing is about, sorry.

It does not do exponential parallelism by trying many things at once. What it does do is create exponentially many paths, while some of which interfere with one another destructively. When you observe the system, then, some paths will have probability zero even though they have non-zero complex amplitudes associated with it.

Scott Aaronson's "Limits of Quantum Computers" to the rescue: http://www.cs.virginia.edu/~robins/The_Limits_of_Quantum_Com...

This is really nice.

Minor nit #1: https://www.dropbox.com/s/2meqa8hhen9ztba/Screenshot%202014-... Seems like the graph visualization is sticking to the wrong coordinates (dragging it to the left doesn't help; it moves back to the center)

Minor nit #2. I'd love to see a visualization of the "probability mixing" interpretation of markov chains and stationary distributions, which is what PageRank is really about. That is, it'd be really nice to have a visualization of the fact that Markov chains are ultimately memoryless (it eventually doesn't matter in which state you start for the distribution of events). I think it could be done by exchanging "probabilities conditioned on the past", which is most easily done by multiplying the entire probability vector by the stochastic matrix and visualizing that.

Wait: the entire point of Bayes's idea was that under very sensible and straightforward rules, "plausibilities" behave exactly like probabilities. So why multiply concepts?

What is the difference, in your proposal, between "plausibilities" and "probabilities"?

The paper is from 2005, so it might be worth saying that on the submission title.

And it's also worth mentioning that because Gelman and Shalizi have recently written a really nice paper about the philosophy of Bayes: http://www.stat.columbia.edu/~gelman/research/published/phil... (PDF) Gelman is a well-known "bayesian", and Shalizi is a well-known "not all that is interesting is bayes"-ian.

Heh, this is far from the worst problems in this dataset. Wait until you find the clusters of lat-long points coordinates that are off by 1 degree in either direction (or 0.1 degrees)

Then there are some pickups clustered around weird areas of NJ and CT (NYC cabs are not supposed to pick up passengers outside NY). Lots of weirdness to sink your teeth into.

But it's a really fun dataset. You can see hurricane Sandy, big city events, construction, etc.

Here's a recent paper about how to compensate for this very illusion, by Heike Hoffman and Marie Vendettuoli: http://users.soe.ucsc.edu/~pang/visweek/2013/infovis/papers/...

I've been using Synthesia on an old macbook air hooked to my digital piano on and off for a few years. It's really astonishing how quickly you can learn a song when you have instant feedback.

Although this tool is obviously not going to teach me dynamics and expression, I can't recommend it strongly enough for people looking to learn to play the piano by themselves.

Scott Aaronson's review of the Tegmark's book is pretty great: http://www.scottaaronson.com/blog/?p=1753

Reminds me of the Bret Victor's "Alligator Eggs!": http://worrydream.com/AlligatorEggs/

I suspect your downvotes are coming because you haven't mentioned http://www.scons.org/. Letting you know in case you're not aware of it. (I would have emailed you out-of-band, but there's no contact info in your profile. Sorry for the noise, everyone else)

On the first day of the first course as a CS major, the prof said something like "in this course you'll learn how computers work from the bottom up", turned to the blackboard, and wrote Maxwell's equations. We laughed, and he said, "You're laughing because I got it wrong, no? You need quantum mechanics for transistors".

We actually ended up starting at logic gates, flip-flops, and latches. But the joke was a great reminder that your floor is someone else's ceiling.

In the US, the http://en.wikipedia.org/wiki/Home_mortgage_interest_deductio... on your first home makes a huge difference when making rent-vs-own plans. It wasn't obvious to me when I first started making rent-vs-own plans (I'm not a US citizen).

In 30-year mortgages you are realistically paying more than half your monthly payment as interest, and doing that with pre-tax money means you're conservatively looking at 15% total reduction in monthly payments compared to renting.

Does anyone know of a version of http://theyrule.net with updated data?

A fast sorting algorithm in Haskell degrades gracefully to a (faster) selection algorithm when you don't ask for the whole thing.

In other words, "sort lst" gives you a sorted list, but "take 5 (sort lst)" gives you the first five elements, without sorting the rest of the list.

> functions are functions in the algebraic sense

This is a minor nit, but there are effects in pure Haskell functions, namely partiality and non-termination. (In other words, the sense in which "functions are functions" is actually a deep question)

There's plenty of academic discussions on how to solve this problem. See stuff like this: http://lambda-the-ultimate.org/node/2003

The set of (say) 4 motors consumes much more power than the on-board computers. Brushless controllers these days handle 20 amps at around 5 volts for a total wattage of 400W(!) . The on-board computer power doesn't even count.

Of course, you're right that powerful computers are needed. But it's more about miniaturization for weight purposes.

Yes, one must be careful not to define a new standard, but rather to observe a strict subset of both for which that "equation" holds.

Yup, the way to do it is via a small variant of http://en.wikipedia.org/wiki/Expectiminimax_tree. Backgammon has "min", "max" and "expectation" nodes. The "correct" solver in this case only has "max" nodes (for the human moves) and "expectation" nodes (for the computer 'moves').

Heh, I did something like this for the original Threes games via computer vision and screenshots :)

http://www.youtube.com/watch?v=Sn2o2hb1bi0

Mine is using a minimax variant that replaces the minimum nodes with expectation (given that the choice of next tile is uniformly at random). This is sort of the algorithm used by backgammon solvers. The fun thing is that when expectation factors in, the branching factor is quite wide, but the necessary depth for the algorithm to beat humans is much smaller (with 8-ply on Threes this thing is miles ahead of me, no contest)

Now someone needs to start a project to find a large, documented subset for which ANGLE(ToGL(x)) = x so we can all get rid of this historical two-standards-that-do-exactly-the-same-thing nonsense.

If you have access to a GPU (or can evaluate quadratic functions of barycentric coordinates) Loop and Blinn's 2005 algorithm is brilliant http://research.microsoft.com/en-us/um/people/cloop/loopblin...