arhyth

> Databricks is often managing a million-ish Spark-sub tasks for various users. They couldn't do that using traditional operating system scheduling techniques: they needed something that could scale. The obvious answer was to put all scheduling information into a database. That's exactly what the Databricks guys did: they put it all in a PostgreSQL database, and then started whining about Postgres performance," says Stonebrake.

anyone know of a talk/paper discussing this scheduling "hack" in more detail

and these data structures are themselves able to span hundreds/thousands of disks? eg. what happens when a customer's bucket/object are too many such that the skip list "indexing" them cannot fit in a single harddrive?

kind of agree but also disagree. the runway for Go to go (pun unintended) from easy to difficult to use because of it consisting few parts is far longer than for a developer using a feature rich language to eventually shoot themselves in the foot. let me throw in another language in the mix to describe what i personally think is _almost_ the best tradeoff between this difficult and easy simplicity. Elixir. it has almost the same amount of keywords (as a proxy measure to API surface) as Go but on the other hand it also exposes metaprogramming and so, if you really want to, you can easily shoot yourself in the foot. but in both languages, as I said for Go, the runway for language usability to go from easy to difficult due to their limited features is very very long.

just new to Golang (in programming in general, actually) and this bug me. in the slide, it's indicated they are semantically the same. but how?

(1) var thing Thing json.Unmarshall(reader, &thing)

(2) var thing *Thing = new(Thing) json.Unmarshall(reader, thing)

in (1) thing is a variable for a value of type Thing, while in (2) thing is a variable for a Thing pointer

shouldn't thing (2) need to be dereferenced first?

thanks for the outline. article was behind a paywall from my end.

can someone please give this outsider-noob an eli5 explanation why golang do not adopt a same policy/attitude/implementation towards package management?

i love elementary OS. but gahd Calibre is just ugly and for some (probably noob) reason, can't get it to work on my machine. i'll try this one out.

+1. even for "non-believers", there are practical gems that will benefit anyone who cares to really read it.