andrioni

It might be more about bureaucracy than about cost. At my last job even small expenses would require printing forms and getting the CFO to sign them, so in the end it was pretty much not worth it for small tests.

Do you think you could ask someone and find out the cluster sizes they used for those sorts? They mention "With the largest cluster at Google under our control", but it would be more interesting to have an idea of actual numbers, even if just an order of magnitude.

Yeah, I do this every day: I treat my work laptop as a desktop, and just remote desktop into it from a Chromebook (or a Macbook, depending on where I am) whenever I want to work remote.

Well, they still have the instance types where instance storage matters most, IMO: d2 and i2 (which are great for running things like Cassandra and HDFS).

Also, complementing toomuchtodo's comment, there's also now EBS types focused on almost all kinds of workloads: sc1 for cold data/archiving, st1 for workloads optimized for sequential scans (a Confluent benchmark even reported that Kafka ran better with st1 EBS than instance storage), io1/PIOPS for databases and gp2 for everything else.

Orkut was always part of Google, it was created by a Google employee (Orkut Büyükkökten) as his 20% project.

They already offer spot p2 instances, although I think the main spot pricing history widget isn't up to date yet. The current minimum spot price seems to be 0.09 USD/hour for a p2.2xlarge, and there have already been steep price fluctuations for p2.16xlarges in the past hour.

2TB of RAM isn't close to a supercomputer nowadays, you can even get one of those at AWS now (the appropriately named x1.32xlarge instance).

>• It's possible to know how to say a word, but have no clue how to write it. This phenomenon is called character amnesia, and it affects most native speakers.[1] Phonetic languages allow you to write out a misspelled word, which readers can understand (or autocorrect can fix). > >• Likewise, it's possible to know what a symbol means, but have no idea how to pronounce it. This is extra-fun in Japanese, where most kanji have multiple pronunciations.

I don't think English is much better in these cases. In fact, the writing can be so divorced from speech that spelling bees are a thing.

Given that games tend to be deployed on a wide range of computers, it's hard for them to use newer x86 extensions (e.g. AVX), while most scientific computing applications tend to be compiled for a specific machine. To get to see the larger improvements, you must use the newer instructions and registers, like you can see in BLAS benchmarks[1].

[1]: https://www.bountysource.com/teams/openblas/fundraiser

And most of the ports that weren't done by icculus were done instead by flibitijibibo (Ethan Lee), who is also pretty amazing.

Do you have any numbers on queries that return millions of rows? If it's still fast, asyncpg could be nice for some ETL-type workloads.

In the spec[1] there is a definition of an "object container file" which includes the schema, and is the default format used whenever you save an Avro file. You can even use it whenever sending Avro data through the wire, if you don't mind paying the extra space cost.

[1]: http://avro.apache.org/docs/1.7.7/spec.html

That's the way Julia does multiple dispatch, and it works well.

It shouldn't be hard, given that most of the work is being done by a shared C++ library.

(I'll probably take a stab at writing one this weekend, though)

Most people (in my experience) don't use the old S3 block filesystem protocol anymore, and just use object storage instead (with either S3n, S3a or the proprietary EMRFS).

The Hadoop FileSystem interface doesn't really force an specific underlying implementation, and you can even use "local" filesystems without any issue, in fact, IIRC, MapR-FS is just an optimized NFS drive, i.e. a shared network drive.

What do you think about Cloudera's Spark backend to Google Cloud Dataflow pipelines[1]?

[1]: https://github.com/cloudera/spark-dataflow

I second the recommendation to use CloudFormation + packer + ELBs + auto scaling groups for web applications whenever possible, it just makes everything so easy and automatic. Of course, there's a learning curve and you pay a premium for all that automation, but in my experience it has been usually worth it so far.

The Russian phrase means "[They] call me X". "My name [is] X" would be "Моё имя — X".

>Despite the fact that list 'n' hashes are very common, there are times I wish I was using a thoughtful tree representation. Such a model can provide richer navigation operations.

Specter[1] provides something like this for Clojure, allowing rich navigation and transformation abilities to deeply nested structures.

[1]: http://nathanmarz.com/blog/functional-navigational-programmi...

> Heck, even Go programmers have to think about compile cross-platform.

Not that much in Go 1.5 :) On the last project I worked, just setting GOARCH and GOOS worked out of the box.

Weirdly enough, this goes even to the database:

- I've seen my fair share of columns typed as fixed-width NCHAR. Yeah, a "89.99" order total you see on the screen might be stored internally as `000000000089.99`.

- Booleans? Why not use `x`, ` ` and `#` instead of true, false and null?

I thought F# and Haskell were at least on the same level of performance, if not faster. Is it not the case?

The author is using the scalaz syntax, you could substitute it for the "vanilla" syntax:

    var h: Option[String] = Option("Hello")

or import scalaz (in case you have it in your classpath etc):

    import scalaz._, Scalaz._
    var h: Option[String] = "Hello".some

I'd guess it's this system: https://cmsweb.cern.ch/das/

Oh, I'd love to have something similar, running etcd (for Kubernetes) and ZooKeeper (for the rest of the services) can be a PITA, and it feels like an unnecessary duplication of infrastructure.

While I'm not using Samza, Spark Streaming also works pretty nicely in this case, although it is not so focused on keeping state (it can, though, using the checkpointing system and the `updateStateByKey` transformation) and thus might not perfectly stable if you require to handle failures without reprocessing.

Yeah, I'd guess it depends mainly on what type of instances you run. Only one of the 28 instances I'm running right now (all in us-east-1) is going to be rebooted, and it is the only old generation instance I still run (it's a hi1.4xlarge). None of my M3s, C3s or R3s are affected, even though some are still on PV.

Zalando is already giving profits since Q2 2014.

Sorry, but did you mean one-dimensional data represented in two dimensions, as in scalars represented through areas?

Isn't compiling nginx from source just a `./configure; make; make install`? I mean, any developer who has ever used a *nix-like probably knows the classical three-command incantation.