struct

Rewatching the films recently, the only explanation I've thought of for the machine's barmy plan is that they must be bound by the three laws or feel some kind of deep obligation to us, even after we made the entire world uninhabitable. Why else would they spend so much time growing us, feeding us, simulating for us, allowing the One(s) to save humanity repeatedly, and why else would they initially try to simulate paradise for us?

28nm was quite a cheap fabrication technology even in 2015, but it costs a lot to have a completely custom production run. My guess it that it approximately works out in savings of power and space over the lifetime of the chip. It probably doesn't make sense for them to move to something smaller (and thus more expensive) whilst the performance benefit remains so substantial. If I were Intel, I probably wouldn't lose too much sleep over it either, because you still need something to attach the highly-specialised TPU to, and that'll be a Xeon for the forseeable future.

I imagine that once they've trained the floating-point models, they'll then quantize them into integers to make inference faster. It's not something I've done, but I imagine that the limited range of the of integers may cause problems (though they say in the paper that the 16-bit product can be accumulated to something that's 32-bit). The features to do this will be coming fairly soon to regular TensorFlow too.[1]

[1] https://youtu.be/0r9w3V923rk?list=PLOU2XLYxmsIKGc_NBoIhTn2Qh...

Interesting points I took from the paper[1]:

* They actually started deploying them in 2015, they're probably already hard at work on a new version!

* The TPU only operates on 8-bit integers (and 16-bit at half speed), whereas CPU/GPUs are 32-bit floating point. They point out in the discussion section that they did have an 8-bit CPU version of one of the benchmarks, and the TPU was ~3.5x faster.

* Used via TensorFlow.

* They don't really break out hardware vs hardware for each model, it seems like the TPU suffers a lot whenever there's a really large number of weights and layers that it must handle - but they don't break out the performance on each model individually, so it's hard to see whether the TPU offers an advantage over the GPU for arbitrary networks.

[1] https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk...

It's the same situation in London, only about 25% of Underground stations are accessible[1]. This seems to be the cost of old infrastructure - when most of the network was built, accessibility simply wasn't a consideration. Neither was having a decent mobile signal, or air conditioning. Fortunately, we've learned from this and newer systems like the ones in Shanghai and Taiwan seem to be far more accessible.

[1] https://tfl.gov.uk/transport-accessibility/wheelchair-access...

Michael Hastings? [0]

[0] https://en.wikipedia.org/wiki/Michael_Hastings_(journalist)

My hobby involves free-form Internet text, which has similar normalisation problems. I found that for my purposes, character-level models where quite effective partly because they were easy to train on not much data and usually quite robust to minor misspellings[1]. I also developed a part-of-speech tagger[2] which might be useful to you, assuming your corpus has verb and noun tags available. If you've got some more questions, my email's in my profile.

[1] http://dracula.sentimentron.co.uk/sentiment-demo/

[2] https://github.com/Sentimentron/Dracula

How does this compare to ARM's VIXL?

[1] https://github.com/armvixl/vixl

Very impressive project! It's results seem pretty good - what's the size of the index?

I've used go-bindata[1] for embedding things like shell scripts and data files into my applications, it's a fantastic tool!

[1] https://github.com/jteeuwen/go-bindata

We are so screwed.

Edit: we've known about the greenhouse effect since Victorian times, why is this still so controversial?

I happened to bump into one of the people working on this a few weeks back, the 1/2 billion cores per system aspect is very fascinating and because it's SPARCv9, there's quite a lot of software that already works. It's definitely a project I'll be watching closely.

Looks neat, the API seems nice, and it'll be interesting to see a new framework now that Leaf[1] has been abandoned. Also fantastic to see a linear algebra library for Rust: that's useful for lots of applications other than just machine learning. I'll be keeping an eye on this :D

[1] https://medium.com/@mjhirn/tensorflow-wins-89b78b29aafb

The performance difference between CPU vs GPU is very substantial: the consumer-class Nvidia GTX 980 I use can deliver about 4612 GFLOPS [1] of raw performance, whereas the i7-6700k CPU I've got can only deliver about 113 GFLOPS. Whilst those numbers aren't really comparable (and hence the difference is not as large in practice) you still get a substantial speedup, maybe 10x-15x (this is only especially important during the training phase). The insight I've gained is that the kinds of APIs that Google/Microsoft/IBM give you are definitely not magic and they _can_ be replicated + tweaked to a reasonable degree, and actually I think the concentration of natural language understanding / visual recognition / machine learning and the datasets needed to make them work into the hands of a few very well-financed corporations is both a good thing (because it lets you get up and running quickly and the predictions are continuously updated) and a bit of a bad thing (since these types of applications are going to become a more essential part of computing, and for reasons of cost, performance and customisability). I've also written a bit about my experiences with Theano versus TensorFlow[3].

[1] http://techgage.com/article/intels-skylake-core-i7-6700k-a-p...

[2] https://en.wikipedia.org/wiki/GeForce_900_series

[3] https://medium.com/@sentimentron/faceoff-theano-vs-tensorflo...

To answer the spirit of your question: the really interesting thing about deep models is that once you've developed and trained them the forward pass is really easy to deploy. The forward-only version I did here relies on numeric.js[1] for the basic matrix operations (addition/dot-product etc). The really useful and amazing thing about both TensorFlow/Theano is that they can automatically differentiate the model so that training becomes tractable, meaning the actual linear algebra concepts aren't too hard to grasp. Aside from that, you'll need lots of time, a bit of money, and patience: I first started adapting Dracula from a tutorial[2] (although the structure of the model is different) about a year ago, but things didn't really get started until November last year after I'd bought a decent GPU (a GTX 980). Happy to answer any more questions :D

[1] http://numericjs.com

[2] http://deeplearning.net/tutorial/lstm.html

It's not on my radar at the moment but it might be a fun experiment! I've often thought that sentiment might be useful in the context of parental controls: whether you could de-emphasise links or messages which might contain distressing content.

Now also available as a Node.js module: https://www.npmjs.com/package/dracula-sentiment

Feedback welcome!

I've been working on making my deep NLP model Dracula work in the browser and on node.js. Here it is doing sentiment analysis: http://dracula.sentimentron.co.uk/sentiment-demo/

As much as I like the idea of living with only a few possessions, it strikes me as a constant struggle: where do you put the 50 pack of envelopes you had to buy to send one letter? Are scissors for opening packaging included in his possessions? Nail trimmers? Toiletries? A formal pair of shoes? Do you have to hire all this stuff when you need it, or just keep in a drawer out of sight? And this is what I didn't get about the KonMari method either: a formal suit doesn't "bring me joy" any more than a sink plunger, but both are necessary. How are minimalists able to get away without this cloud of objects following them?

Thanks for this! I recently ported something over from Theano to TensorFlow (I wrote it up at [1]) and I have to admit I generally enjoyed the experience a great deal, even if the single-GPU performance wasn't good enough to make me switch. The TFLearn library (especially [2]) looks very compelling for prototyping however, some I'm very excited to see how the project develops.

[1] https://medium.com/@sentimentron/faceoff-theano-vs-tensorflo...

[2] https://github.com/tflearn/tflearn/blob/master/examples/nlp/...

It seems hard to do it directly, but I've been analysing [1] for an upcoming project, and they do some clever tricks to amortize the cost. If you get to try it out, let me know what you think!

[1] http://dev.blockcypher.com/#microtransaction-api

Congratulations on the release! It'll tale a while to digest the truck-load of new stuff you've got :D

The realtime database is IMO very cool and it's definitely worth it just for that, but integrated analytics and cloud notifications takes it to the next level.

Looks neat, let's hope Google leads in 3rd party applications too and not just in appearance. Also interesting that they specifically gave a shout out to the Alexa team.

Not to sound overly critical, but I don't enjoy the tone of this paper: presenting the concepts (e.g. graph operations) and features (e.g. automatic differentiation) of TensorFlow as new and novel when in fact very similar systems like Theano have existed since at least 2010 (which they finally get round to mentioning 14 pages in). They really should have shorn the paper in half and just focused on the distributed bit, which is the really novel and exciting bit.

Addendum/Edit: With that said, I don't want to disparage the amazing technical achievement of the Google Brain team and the way TensorFlow really smartly reuses and improves on the concepts that make Theano really powerful. I'm following the project very closely and TensorFlow has a very high chance of being the foundation for my future projects :)

For anyone interested in this topic, I can really recommend "Radiation: What It Is, What You Need to Know" (by Robert Peter Gale and Eric Lax)[1] which includes some very thorough discussion on Chernobyl and Fukushima. An interesting point they raise about Chernobyl is the psychological fallout of people thinking they were exposed: there was a notable rise in abortions and quite a few plant workers drank themselves to death after the accident. It's all very fascinating.

[1] http://www.amazon.co.uk/Radiation-What-You-Need-Know-ebook/d...

1) I'd be very interested in such a service. 2) Yep, it's a lot, but that query is quite a lot bigger than most. Assuming some constraints on the layout of the index, I estimate you'd spend roughly $70 plus taxes and compute time retrieving the indexed documents from S3 for that query. You'd always be able to reduce or expand the keywords to and only retrieve as much as you could afford. I think there's value in both allowing people to tackle querying the index by themselves and providing a paid-for managed service that automates much of that.

This is basically my use-case too, in that it's more important to get access to _every_ document which contains that keyword and less important to rank them in a search-engine order. I think that it may be possible to do fairly inexpensively[1] but I'm still benchmarking to pick the right mix of technologies and data structures.

[1] https://www.getguesstimate.com/models/4225

Some of my primary interests are building websites that can analyse other websites and for that, I need a keyword index and access to the underlying crawl data (as in, a response that can point me to the exact file offsets that contain the pages). Think [1] but with keywords instead of URLs.

[1] http://index.commoncrawl.org/

Neat, I was working on a project to give a full programmatic keyword index to the contents of the common crawl, but I guess there's no need! It's very exciting to consider what kind of applications you can build with this.