Optimism vs. Pessimism in Distributed Systems

Yes,distributed systems problems and multi threading problems tend to have very similar solutions. Contention is the critical factor indeed - if there's no coordination needed, then we can keep scaling forever.

Yes, the main difference would be that distributed systems have more failure modes to be concerned with and that dictates a lot.

Even a naive sharded approach requires coordination if you want a scalable solution. Where do you send keys when there are 4 shard buckets? What happens when we want 5, or 8, or 32 shard groups? Just yolo turn off all writes, toggle some config and turn everything back on? Not great. You've changed the shard selection of a key from 2 to 11? What happens to all the previously persisted keys that were written to 2? The answer is unsurprisingly complicated, and depending on the system you're writing, this can be a really hard to solve against if you haven't prepared to grow (or shrink).

As you noted, there are a lot of similarities between parallel programming and distributed computing.

The coordination of shared resources is the key problem in both fields. Distributed computing is more complex since you're adding in the concept of communication which may be unreliable.

I think by "something is wrong" you mean that data has changed and if data hasn't changed then the coordinating method of a distributed system isn't very relevant