896 karma · joined January 2, 2012
http://dumps.wikimedia.your.org/other/cirrussearch/
These are structured with a JSON string for each doc roughly like https://en.m.wikipedia.org/wiki/Foobar?action=cirrusdump
What is the shape of the database? Do you normalize each document into a single vector which is compared, or are you keeping per-word vectors? I'm imagining you probably don't have a database with a row for every word, but maybe you do?
How do you pre-filter the list of documents to compute L2 for? If no pre-filtering, can this approach scale into millions of documents?
> The approximate distribution in the Universe is 5% regular matter, 25% Dark Matter, and 70% Dark Energy. Half of that 5% was missing, and now found.
> Regular matter makes stars and visible galaxies, so it is "bright". Dark Matter is so named because it does not make things we can see with telescopes directly - it is "dark". We can see the effects it makes with gravity, such as the rotation curves of galaxies, and gravitational lensing. So we know something is there, just not what it is made of. Dark Energy was invented to solve a couple of mysteries. One is the geometrical "flatness" of the Universe, and the other is the apparent acceleration of the Universe's expansion. Like Dark Matter, we don't yet know what it is. But something is causing the flatness and acceleration, so we gave it a name as a place-holder for theories.
> A similar situation happened a century ago, with the precession (shift) of Mercury's orbit with time. We thought it was caused by a planet inside of Mercury's orbit that we hadn't found yet. It was named Vulcan, after the Roman god of fire (not Spock's home planet). It turns out relativity was the right answer - the Suns gravity bends space near it, and causes the orbit to shift. Vulcan was just "a name we gave to whatever causes the observed effect".
> Dark Matter and Dark Energy could turn out to be something entirely different than types of matter and energy, but in the mean time it gives them names we can attach theories about them to.
> I will announce locations for between two and four Gigafactories later this year – probably four.
This puts them in the range of around 175 (35 * 4 new + Nevada) gigawatt-hours annually. They are certainly looking to compete in this space