cscheid

I found the Acela north of New York (NY-Boston) to be great, but south of New York (NY-DC) my experience was a bone-rattling ride from hell. It was shaking so bad it threw my laptop off the table not once, but twice.

If you can get your hands on an emulator, you owe it to yourself to try "The Castle" and "Castle Excellent", two of the best puzzle games ever created.

But do play them on an MSX; the NES version is nowhere near as good.

Not resolution. detail.

But look at the variance on those distributions. "Late 30s" is a poor description. How about "half of the winners are between 28 and 45"? Not so exciting then, I would guess.

Better yet: look at distribution of "age when paper was written", modeling the generating process is scientist X at age Y writes a paper with Z=0 if no award is won, Z=1 if award is won. Is it obvious that this distribution conditioned on Z=1 is different from the unconditional one? Not to me.

Or, like Scott Aaronson put it in his blog, "of course it's a computer. But the interesting question is: what kind of computer is it?"

That's because your short statement is not really representative of good statistical practice. For example, people spend a lot of time researching http://en.wikipedia.org/wiki/Generalization_error, and models like http://en.wikipedia.org/wiki/Probably_approximately_correct_... worry a lot about things like VC dimension exactly because they characterize the behavior of statistical models to unseen data. Or maybe you don't think of prediction as "behavior under unseen data"?

Um, https://xkcd.com/385/

Watch out for the fibonacci flim-flam http://www.lhup.edu/~dsimanek/pseudo/fibonacc.htm

Related, a purely functional package manager, Nix, and a Linux distribution based around it: http://nixos.org/nix/

I think OP might have been thinking of s/efficient/dense

Antimatter is certainly the densest way we know to store energy, which is great when it works. But unfortunately, it's also the densest way we know to store energy. Which is not so great when it explodes.

Double-plus nitpick: in Vect, VxW can be seen as both a product (which is usually called cartesian product) and a co-product (which is usually called direct sum). So, I guess I would not say "rather than" but "in addition to".

First hit on google for "Haskell Wiki Monoid" was this: http://www.haskell.org/haskellwiki/Monoid

Lots of examples there, including a post by Dan Piponi that says, right at the beginning, "In Haskell, a monoid is a type with a rule for how two elements of that type can be combined to make another element of the same type. To be a monoid there also needs to be an element that you can think of as representing 'nothing' in the sense that when it's combined with other elements it leaves the other element unchanged. "

I don't know how you can construe that as "The Haskellians don't even help you out enough to point towards ordinary abstract but instead send you all the way to category theory with sign-posts along the way."

What about a group? Is that combinable too? Monoid is short, precise, and googlable. Which also happens to be the case with group, monad, functor, etc. Why invent new vocabulary?

Have you noticed how little sense the words "class", "struct", "union", "object" make to someone who hasn't programmed before? It's just that we got used to them.

But "Appendable" is a terrible way to describe what a monoid is. Some of the really cool tricks in the way abstractions in Haskell match mathematical notions come from extending the abstractions beyond what you would expect. For example, integers under sum are monoids; naturals under the maximum operation too. And so is the "over" operator from computer graphics. All of these find applicability as general monoids in Haskell, but you'd have a hard time calling them "Appendable". All of these just need a binary operation that's associative and with an identity element.

The same thing happens for functors and monads: their use come from the properties they respect. And if you accept that abstractions will be defined as a set, some operations, and some algebraic laws respected by those operations, well, then, you arrived at abstract algebra, and you might as well use the right terms.

You're right, it is :)

The version with online demos uses latitude/longitude for the spatial dimension, but a new version we're working on (under the master branch on github) allows arbitrary x,y addresses. With that, we've encoded IP addresses as locations in space-filling curves to play around with source IP/destination IP datasets. It's particularly nice because the hierarchical nature of the spatial addresses end up mapping to larger and larger subnets of the IPv4 space.

Heh, I had a conversation a few weeks ago with Jeff Heer, coauthor of a very similar paper where we talked about how to make this stuff work with hexes. I think it can be done, but the hierarchical part is a pain, and I'll need more than a HN comment to explain :) Look me up offline if you're interested and we can chat.

It's open source (http://github.com/laurolins/nanocube), so one could certainly try that. With that said, it is currently an in-memory store with no disk backing, so it would be a significant amount of work to get it going.

At the same time, I certainly believe that some sort of hierarchical, low-footprint data cube would be a great addition to postgis.

Memory usage gets worse as the number of dimensions increases (because there's more ways in which to build summaries), and also as the number of unique keys increases (because there's more of different things to store). The main advantage nanocubes has is that it's a sparse scheme: previous work on fast data cubes for visualization (immens, for example) use a dense storage scheme, and so memory usage goes up proportionally to the size of the address space instead.

> Hm, I see, so this can output the virtual miplevels quickly?

Yes: when you see the tiles on (say) http://www.nanocubes.net/view.html#twitter, what you're getting is not a precomputed tile: the server is actually visiting the datastructure every time (and I hope I can convince you that at that speed, it is not touching 210 million points :). The reason you can't precompute all choices is simply that there's too many of them and you'd run out of bits in the universe (the paper, http://www.nanocubes.net/assets/pdf/nanocubes_paper.pdf, has details).

> Also, would it work with a "full" dataset (i.e. no empty cells)?

Not exactly. We get away with an in-memory data cube exactly because in many practical cases the data is sparse over the address space. With dense data you'll take a much more significant memory hit (but it's one you'll have to take with any aggregation scheme. The paper, again, has details)

> Also consider miplevel interpolation, makes the transitions between levels much easier on the eye (doesn't work well with nearest sampling).

That's fair, although again we're hoping to keep the WebGL version feature-compatible with the leaflet.js version (that does canvas-only, and has no zoom transition at all); the smooth version looks nicer, as you can see in the (bit on the PR-heavy side, sorry) following video https://www.youtube.com/watch?v=8P9QA6TJwys#t=69

(Hey, I'm a big fan of your WebGL work!)

This is really not about interpolation or mipmapping; it's about creating a data structure that allows you to quickly answer queries with which to build visualizations. For example, you might want to see a heatmap of all tweets that happened in 2013, but you might want to see only the count of a particular week, or a month of tweets from Instagram. You might also want to see all of these things above over the entire world, or over a single metro region. What nanocubes enables is answering these queries in time essentially proportional to the size of the output (we take a polylog hit, but that's kind of to be expected). We don't even touch all of the input data, which, if you ask me, is kind of nice.

I wish I had bigger public datasets to show you, but we keep this fast experience with datasets in the billions of points.

About your interpolation point: yes, we could (and do sometimes) use interpolation and smoothing. We choose here not do it here so that it looks the same when you go to the webgl-less mode and run it on an iPhone or iPad, there isn't a big visual difference.

Author here, happy to answer questions.

You don't need to drive out all honest mining. All you need is not to have to pay for any costs.

The issue he's pointing out is that the rational, amoral actor will have to decide between a saturated market of ASICs with extremely long time to ROI, and a quick buck off of grandma's draining laptop battery.

Elsevier has been targeting academia.edu lately. HN discussion here: https://news.ycombinator.com/item?id=6882290

Sean Carroll talks a bit about this kind of problem here, (albeit when discussing the distinction between "coordinate space" and "momentum space") where we perceive the world in one way, but a lot of physics seems gears towards either one of the worldviews, and the question then is "why the hell?"

http://www.preposterousuniverse.com/blog/2010/11/10/against-...

In a tangential note, if P=NP, a sufficiently carefully crafted algorithm very much like that suffices to solve any NP complete problem in polynomial time! (Of course the constants are ridiculously, absurdly large)

http://lucatrevisan.wordpress.com/2010/03/07/on-the-necessit...

Unfortunately, ray tracing of general scenes is undecidable: http://www.cs.duke.edu/~reif/paper/tygar/raytracing.pdf

Wait, no, LU decomposition is not stable unless you're doing pivoting. And if you're doing pivoting, then it very much stops looking like an "exact formula", because it'll get filled up by ifs and variable renamings, etc.

There's plenty of cases in graphics and simulation where solving even a 4x4 matrix needs careful numeric consideration (see this paper about tetrahedral mesh simplification, section III.D, page 7: http://www.sci.utah.edu/~hvo/papers/tetstream.pdf)

If you care about accuracy at all, do not use the straightforward "exact formulas" like Cramer's rule. That's just asking for trouble.

And for numerical linear algebra, I'd go start with Strang and chapter 9: http://math.mit.edu/linearalgebra/

Eliezer Yudkowsky gave a talk at MIT recently that touched on this very subject:

http://lesswrong.com/lw/itp/meetup_talk_by_eliezer_yudkowsky...

Some commentary by Scott Aaronson:

http://www.scottaaronson.com/blog/?p=1558#comment-89317

Raw data, and the source code which you used to arrive at your conclusion, or it didn't happen.

In today's world of github gists and python PyPI's and R's CRAN, there's no excuse to not document the entire process, in addition to raw data.

Very cool! We've been working on a similar open-source project for collaboration via github gists of R scripts. Think iPython web notebook meets js-fiddle over a github backend: https://github.com/att/rcloud

One nice feature of RCloud is it knows how to make a Javascript-R bridge, so you can create d3 and webgl plots with R data and back.

The github page has more details: http://att.github.io/rcloud