makmanalp

Even the name "bad news", depending on the situation, can come off as a bit of a deflection - if you're causing the bad news (e.g. firing someone), then it's not bad news, it's just you! Not to say that that it's your fault per se, but the wording is a bit weaselly, which often doesn't do much more than add to the frustration.

I think this a good interim "solution", but it won't work for everyone. On a larger scale it's rife with issues: e.g. browsers often have a limit to how many downloads they'll do from a single domain in parallel, so stuffing API, assets and pages all into the same domain can limit performance. The other thing is that using DNS to load balance between servers is much more lightweight than using an HTTP proxy / load balancer to direct traffic.

I can't believe I'm saying this but it would be neat to have a browser native BLAS, or absent that perhaps cross-compiled to WASM from a RUST implementation or something! Makes me think of all the demos like the tensorflow playground (dunno if all that happens in the browser or not).

Slightly tangential but dang, this is the best graph and accompanying explanations of consistency models I've ever seen: https://jepsen.io/consistency

> They then tested against only different configurations of themselves for configurations that hit disk.

In the case of Redis, AFAIK it can't support larger than memory use cases, right? And in fact they do compare to RocksDB for larger-than-memory, see Fig 10. Granted, it could be more detailed, but I think it makes their point.

You wouldn't - first and foremost, this is probably most useful as a storage layer for some more higher-level data store.

Second, think of software artifacts from papers as a proof of concept, or a prototype. It's intended more to demonstrate some architectural innovation, and less to serve customer needs. Their ideas revolve around supporting a high level of concurrent access and removing the locking overhead that would normally stem from this, and an interesting way to handle spilling over to disk in larger-than-memory scenarios. This might evolve into a customer-oriented product eventually. Or perhaps get retrofitted into the guts of something like MS SQL server.

On top of that, it'll compare complex structures to each other, so you could have a very large list of dicts that contain a custom data structure and it'll show you a diff and exactly where the mismatch is. Then it can auto-drop to the debugger and you can inspect previous variables to inspect the root cause, if not immediately apparent. Going back to C land, as much as I enjoy it, the tooling feels like the stone ages. I get that it's a lot harder for a language that compiles down to machine code, but dang.

Whoops, I see my mistake, thanks.

> The only limitations come from the platform itself (JavaScript/web), which do not allow for side channel resistance or reliable constant time operations. Overall however this is an exceptional library for JavaScript cryptography.

How would this compare to something like WebCrypto, which assume would be implemented in a way that would allow for side channel resistance etc? It does seem surprising that we don't have something like a browser API version of libsodium in widespread use already.

Hey! Thanks for taking my semi-snarky reply at face value. I appreciated your kind response, and learned a few things. I'll pass on your hello to Glaeser if I ever run into him :-)

> finding a new CEO is harder than it looks

I support your side of the argument, but why is it the case that finding a CEO is hard? It seems a bit like becoming president: there aren't many people in the pool who have matching previous experience anyway, and it's not even clear if "similar" experience even applies (Does one company resemble the other, even in the same industry? Different stages of growth, corporate structure, business model, region of operation etc), and much of the job seems to be adapting to new conditions. It seems more about the CEO's reputation and network, is that really just it? You pay for the CEO's access to the Old Boy's Club? Wouldn't the company be better served by spreading the role across multiple people, to avoid the stigma of changing CEOs?

I think what causes most people to stumble when learning Turkish is vocabulary - with romance languages and to some extent germanic ones, you can get away with a lot just because of similar word roots. With Turkish, a lot of people have to contend with entirely new words for pretty much everything. The rest is easy.

To expand upon this, in PCA, different components have "loadings" not on single variables but on many variables (a.k.a. columns), which are often highly correlated with each other.

In case anyone else is wondering why you'd not just use LXC:

https://cloud.google.com/blog/products/gcp/open-sourcing-gvi...

That said, they don't quite go into the details of what type of isolation is missing from standard containers - I'm curious. It does seem like it would have been ideal for everyone if LXC would have had better isolation, rather than having to run a userspace kernel emulator thingy for each container, but c'est la vie!

> Question: If what makes AMP fast is the restrictions on size, JS, and CSS, and you know this and want to conform to this, why do you need to use AMP? Why not just develop your site like this anyways?

I think aside from the icon and the special treatment mentioned above, don't AMP pages get served directly from google and get preloaded on the search results page? So that's the other benefit, I guess. Whether a few less milliseconds from preloading on an already fast page really buys anything is another question.

> That suggests that the life of Java as Android's primary language might be really coming to an end

Huh - how did you get to this conclusion? What would be the alternative? Do you mean Java was essentially "better" because it had access to all the undocumented APIs while other languages didn't?

It's more common than you'd think - I remember sitting in a talk in college by someone from Scaled Composites (they did SpaceShipOne etc) and they had said that they use inches everywhere.

> It's not like printed photos can't be burned, scratched, lost, discolor, soak, stained and many other things besides...

Not that I disagree with you, but this is an interesting point. I've heard archivists argue that with simpler analog storage, the curve for corrupt bits vs interpretability is smoothish, but with digital storage it's often a step function where it's like "whoops, a few bits flipped, can't read at all anymore"

> the fact that data can be copied perfectly, as many times as you want

Tangentially, this makes me wonder: While microfilm can't be copied perfectly, isn't it true that the source quality (e.g. the crispness of the print) is vastly lower than the resolution of the medium itself? Which seems like it would imply that the "lossy" transformation might in some sense still lossless, at least for a while - like converting a high bitrate mp3 recording of a radio broadcast into a lower bitrate one. Or perhaps the kinds of loss are different. I'm sure I'm confusing a lot of ideas and butchering some terms here.

It's better to think of tuple spaces as a concurrency / communications model than an implementation. So it's more like "the actor model" rather than Erlang's or Java/Akka's specific implementation of it. It's more about "if we had this type of system with these constraints and these features, abstracting away these details, what would we gain or lose?". You're right that in the end a good or bad implementation can make or break things (take a look at this paper: https://arxiv.org/abs/1612.02979), but that's not the point, at least with the original paper.

The interesting thoughts from the paper as far as I can see were: 1) Tuple spaces are programming language or architecture or program independent, and vastly different programs can communicate with each other 2) You don't communicate directly to other agents by address, you write to a topic and read from a topic, which is a form of decoupling producers and consumers 3) The "block when nothing to read in this topic" idea, which makes programming coordination SO easy. I guess it's a bit like unix pipes.

If tuple spaces don't seem that interesting and novel, it's probably because of the benefit of hindsight and that a lot of these ideas are so subsumed into the tools of today. I can't definitively make the claim that Linda is the cause of this, but I suspect it had some effect. I think the original author also had a lot of wacky ideas around "cyberspace" and all that, but that's another deal and I don't think it's why people find the Linda paper interesting now. The closest useful descendants of Linda to be seem to be modern Pub / Sub systems or coordination databases like RabbitMQ, Kafka, Zookeeper.

> they start by, without evidence, asserting that the main thing fork() does is to insert and remove processes from a list of processes

Since fork() as far as I understand is effectively copy-on-write in modern architectures on linux, that's probably not a terrible model at least for the initial step but it does seem kinda over-simplified if you think about the rest of the lifetime of the process and the different execution models with processes on a computer vs a supercomputing cluster.

What looks like it happened is that they had a cool enough idea: with supercomputers, the scheduler is kinda like the operating system. The scheduler becomes the bottleneck in some workloads. So why not replace the core data structures and operations with a DBMS which is kinda built for this sort of highly distributed, high contention state management situation. Also the larger general idea of comparing it to an OS sounds curiously close to the mesos / kubernetes worldview of things in some ways: "the operating system of your datacenter".

Then some annoying but probably correct person was like "hey we need a basis for comparison and a benchmark because this is an academic paper and that's how systems papers must be because academia in 2018" so they shoehorned in some apples and oranges comparison and we got this.

This is exactly my point. Saying google solves the discoverability problem in this manner is obtuse - it's not the same thing as looking for a specific thing. If you want to make discovering new content that you might like in a decentralized manner easy, then great! I'm sure people will be excited about that. Currently though, medium does that to the first approximation.

Also, social news websites require effort to post on, and often the content that gets upvoted is the lowest common denominator of the interests of the people who currently happen to be staring at the site, rather than each person getting their own set of new content.

I also loved that era of blogs, but unless the distribution problem is solved, we're probably never going back. Self-hosted personal blogs end up mostly being ghost towns these days unless you get somewhat famous. Medium on the other hand gets your foot in the door and draws your niche crowd and conversations for you with much less effort.

See also: nbdime to diff / merge notebooks https://github.com/jupyter/nbdime

Just as a note, breakfast in the rest of Turkey is also similarly diverse and excellent :-) You'll notice that each region has its own small specialty additions of spices, jams, pastries, eggs, etc. And often different fruits and vegetables depending on what's seasonally available in the area. It's true that breakfast in Istanbul doesn't quite compare to breakfast on the beach or in the mountain with tomatoes and cucumbers picked freshly from the garden, but still, quite a great breakfast can be had in the city.

The deployment is /way/ more simple, especially if you already have a python environment (even more so with conda), which is one of the main attractions for me. The other is that the API is way richer - spark dataframes tie your hands down in so many ways and require you to write tons of custom code for routine stuff, while pandas (and dask) has builtins for almost everything imaginable these days.

The tradeoff is that spark and hadoop in general have invested /serious/ efforts into resilience, while dask only really provides protection against worker failure, but not really scheduler failure. In practice is this really an issue? shrug it depends. "Works for me". How many tasks do you run in parallel? If you're doing ad-hoc analysis on a very large dataset, dask might be a great fit. If you have a data warehouse use case and have tons of people running analytics queries, then you have uptime requirements, and spark might be a better fit. That's where I ended up.

> Dask requires users to keep in mind the computation graph and persistence

I saw this statement in the official documentation too ... in practice this isn't a big burden - if you're going to reuse something, persist it. That's about it. "Keep in mind the computation graph" doesn't mean anything - the docs and guides really gloss over why I don't have to do this. Not thinking about these is effectively a decision made one way or another.

The papers offer a bit more insight: Looks like the novelty is in the scheduler that was built for low task latencies and high throughput: wonderful for all kinds of compute-intensive workloads (they talk about Reinforcement Learning, for example) and definitely fulfills a need. They especially talk about how some tasks are dynamic in nature and spawn new tasks unpredictably and how you might not know the whole task graph in advance, or how some work can be done either locally or globally in a transparent manner, or how if you have many tiny tasks, the task overhead can kill you. But don't see what this buys for traditional table style data-heavy analytic workloads which have none of these problems.

Don't get me wrong - I'm excited about more contenders. The neat stuff about ray seems to be the scheduling model they have, the fact that they built in fault tolerance by storing all state globally in a sharded data store (redis in this case), but also they use Apache Arrow and the plasma store, which means zero-copy data transfer between workers and other data stores - awesome. Dask should probably also consider some of this. But pandas + ray just seems like an odd couple - it seems like ray's scheduler was built for the opposite use case.

Dask is already this! They have a dataframe replacement, a numpy array replacement, and some lower level primitives like dask.delayed too. Plus, the nice thing is that it's already being used with large amounts of data, and the warts (which were plentiful two years ago) are rapidly reducing.

http://matthewrocklin.com/blog/work/2018/06/26/dask-scaling-...

To add, I think perhaps the levain makes it develop to a good state faster such that 2 days is fine as opposed to the 3+ I seem to need with no levain, if you want the nice spotted and bubbly dark and light pattern as opposed to the even golden brown.

> not that bubbly crispy crust you get in pizzerias

Yes, aside from high heat the cold ferment step seems to play a big role: https://slice.seriouseats.com/2010/09/the-pizza-lab-how-long...