azurelake

MySQL has DDL replication.

> Like most things, it's more complex than that,

Sure, no doubt. My point wasn't really about the particularities. It was around the mistaken idea that I see sometimes where people believe that TrueTime allows for synchronized global writes without any need for consensus.

> who doesn't want instantaneous global writes

I want to gently note since I see a lot of misunderstanding around Spanner and global writes: Global writes need at least one round trip to each data center, and so they're still subject to the speed of light.

I mean, he definitely knew about the concept of edit distance: https://en.wikipedia.org/wiki/Edit_distance before writing this code, which is the most difficult part. Here's a practice problem: https://leetcode.com/problems/edit-distance/.

So in this case, the special "something" he has is quite literally the amount of hard work he put in to learn this stuff. And if you take this opportunity to learn about edit distance, you'll be one step closer to that special someone you want to be.

Cool, thanks for the clarification!

When you say per user, you mean per user of your app?

Let's say the price was 12. If this idea is compelling enough for you at that price point to venmo me an advance for your first month, I have some ideas to explore that could actually make this viable. FWIW, I've worked professionally on a PaaS and a managed database offering.

Let's say the following product existed:

1. You can only run one instance of your app.

2. There is a small window of downtime each deploy.

3. Your app has access to 10 GB of storage. The storage is persisted across deploys and can be used for sqlite.

4. Your sqlite data is automatically backed up and can be restored / downloaded as needed.

How much per month would you pay for that product?

If you want to read about an option that doesn't suffer from fragmentation (no pedantic replies please), check out LSM Trees which are what RocksDB uses.

Designing Data-Intensive Applications has a VERY clear and understandable chapter about them. Once you've finished reading it, you'll have the tools to whip together a toy implementation for something like this.

You can use `>>>` from `Control.Arrow` to switch the order if you don't mind make your code more esoteric.

Interesting, I’ll have to dig more into this. Thanks for the super detailed response!

I don't think what you're saying is correct, especially your last sentence (it definitely DOES work at least in terms of not being a table scan). Here's a stackoverflow answer that agrees with me and 8n4vidtmkvmk: https://stackoverflow.com/a/9537675. Do you have any sources for what you're claiming?

edit: Here a quote from Percona as well: "In the simplest case, we want to put the most selective column first in the index, so that the number of possible matching rows is the smallest, i.e. we find the rows as quickly as possible." - https://www.percona.com/blog/2009/06/05/a-rule-of-thumb-for-...

The concept you're looking for is linearizability. Basically, it's the idea of being able to pretend that your distributed system is a single node with a single source of truth. An example system that has this property is etcd: https://jepsen.io/analyses/etcd-3.4.3

If you want to learn more about this, take a look at Designing Data-intensive Applications from Martin Kleppmann.

You can already do this with MySQL/Postgres/etc., and there's no special magic in any of these newer tools that allow you to avoid the speed of light. The reason people don't typically run WAN synchronous setups is exactly because of that (and network reliability).

The only way to a "free lunch" is via CRDTs.

edit: This is in no way a knock on FDB by the way. I think it's a really interesting piece of technology and I want to explore it more. Just that it doesn't have a way to do consensus faster than the speed of light :)

You’re living in the past my friend! https://old.reddit.com/r/unclebens/comments/el1d8q/part_2_in...

Laziness by default makes it extremely hard to reason about performance, at least for me. So Rust is actually far closer to C/C++ than Haskell lite in terms of being able to think about how the code you write maps to the physical machine.

The parent comment said "C lineage", not C.

Yeah, and a real problem is how they "poison" the ecosystem. If the community was one that strongly encouraged libraries and frameworks to stick to Haskell2010, I might still consider using it.

I have a similar experience and I agree. I think there is a solid, relatively practical language hiding under all the language extensions, but the never ending desire of the community to solve problems of abstraction vs. real user problems makes it a dead end in terms of time spent.

And funnily enough, the time I've spent really learning shell/awk/sed (pretty much the exact opposite of Haskell!) was one of the best investments of my career!

It's just a fancy way of saying you can use generics for the "outer" type of a container. e.g. List<T> vs. U<T>. That's literally all there is to it.

MySQL has query hints: https://mariadb.com/kb/en/index-hints-how-to-force-query-pla...

It's not. For the niche it has found success in (infra type backend stuff), it's by far the best option available. `if err != nil` is a godsend in that world.

You can make the type itself public without making the fields public. This means someone could hypothetically create one of your structs without using the "New" function you provide, but it's not an issue in practice. And if you really want to avoid exporting your struct at all, you can always just return an interface.

The effort to use something like SQS is almost nothing.

There's a pretty big spread between google-scale and needing to use a "real" queue instead of a RDBMS though. I honestly think you're better off using something like SQS to start with and taking the very minimal ops burden of the extra dependency.

Sure, but this post is specifically about a tool that relates to that weak story. In the context of the use cases PgCat is trying to address, Postgres is most definitely not "the greatest". Apparently, that's a controversial statement :)

Not only is not the greatest, it's literally the worst database technology that's in use for the things PgCat is trying to solve. And that's not a knock on PgCat all (or on Postgres!), it's a knock on the "golden hammer" type worship of Postgres that's been the zeitgiest for the past few years.

The state of the art for sharding, connection scaling, and failover for Postgres is far behind everything else. MySql has Orchestrator and Vitess, the NewSQL systems are doing lots of interesting stuff with replication and sharding, etc. etc.

edit: Look at the work that Notion had to as of just 3 months ago to shard Postges: https://www.notion.so/blog/sharding-postgres-at-notion. Maybe they would make the same choice over again, and that's fine, but doing stone age level work to shard a database in 2021 doesn't jive with the whole "just use Postgres" idea to me.

I see, guess I'm wrong then! What's the disadvantage of Paxos Commit vs 2PC? Presumably there's a reason that the Spanner team didn't use it for distributed transactions.

edit: Looking at the paper, it looks like the tradeoff is increased amount of coordination needed.

"The Two-Phase Commit protocol is thus the degenerate case of the Paxos Commit algorithm with a single acceptor."

I suppose in Spanner, having more than one acceptor is redudant since each shard is a Paxos group anyways.

Yes, see https://en.wikipedia.org/wiki/X/Open_XA

That being said, I don't think it's a good idea for reasons similar to this: https://wiki.c2.com/?DistributedTransactionsAreEvil vs. using the Saga pattern.