radub

Hi, regarding number 3, I have filed https://github.com/cockroachdb/cockroach/issues/58032. Would love to get your feedback on the proposal; does it address your need? do you have a different idea for how it would work?

There are quite a few case studies / customer stories on the website: https://www.cockroachlabs.com/customers/

We plan to support cardinality estimations for multiple columns. Other than that there are no plans for multivariate stats in the near future.

[Cockroach Labs engineer here, I worked on statistics]

All table statistics are stored in a system table (system.table_statistics). CREATE STATISTICS scans the table, calculating statistics on a column, and adds a new row to system.table_statistics.

We still have a ways to go in making statistics automatic (right now it requires a manual call to CREATE STATISTICS) and more useful to the optimizer (which currently only uses basic stats). We're working on that right now for the 2.2 release.

I don't have a specific timeline but it is something we will be focusing on in the following releases.

Regarding DDL statements, this blog post [1] has details. In a nutshell, online schema changes are possible; the changes become visible to transactions atomically (a concurrent transaction either sees the old schema, or the fully functional new schema).

[1] https://www.cockroachlabs.com/blog/how-online-schema-changes...

Thanks! Yes, I expect we will be moving in that direction before too long. It will work well with log tags which are already key-values. Plus most of our messages make it into traces and OpenTracing is switching to a KV interface as well.

In the majority of cases, the current layer would have added a log tag to the Context, so looking up the log tags structure will be immediate. On the other hand, we also use OpenTracing and we do have to go through the hierarchy to find the current Span, which will be slower with deep hierarchies.

The other thing to consider is that logging a message carries other costs, so we can't go crazy with very frequent log messages anyway. In our case the message usually also makes its way into an OpenTracing span and that involves memory allocation and/or data copying. My guess is that going through a few linked Contexts is not a big deal, comparatively speaking.

I don't have the kind of benchmark that you are looking for (with vs without using Context), but I can tell you that in the workloads we looked at, tracing was taking a few percentage points (1-2%) of CPU usage, and I don't think a significant part of that was in Context code.

Finally, Context is an interface, so if need be, we can make our own Context implementation that always stores the frequently looked-up things (like log tags, tracing Spans) to short-circuit that cost.

It could be great exposure therapy :)

[cockroach labs engineer here] Where are you getting the "ignoring stability" part? Or that we "skipped correctness and stability" (in another comment of yours)?

Correctness and robustness are the main factors behind the design and implementation choices we make. A significant part of the overall engineering effort has been on stability for a long time. We're now making it closer to 90% for a while.

None of this is unexpected in the development of a complex system, especially when there are many factors involved in deciding how much to focus on various aspects. I have worked on a few unrelated systems which turned out to be stable and successful when released and - at a comparable stage in their development - they were much less stable. So personally, I am very optimistic about CockroachDB.

The relevant tests fall somewhere between unit tests and integration tests. Many are intended as unit tests, but we prefer to test them on top of the "real" implementations of the layers underneath (instead of mocks) so they are integration tests as well.

> If it's a feature or integration test, why is it a problem to pull in the other packages?

The compiler does not allow the cyclic dependency (even if it's only caused by testing code).

Thank you!

Making everything public runs the risk of other modules using internal implementation details, and that entanglement makes changing things a lot harder.

For a small project where you can easily control this, it's likely not an issue. But we are big enough that no one person knows all the code changes that are going on in all modules.

Exported vs non exported also serves as implicit documentation, it makes understanding things easier when you can tell right away what is internal and what is not.

The sql module needs other aspects of the server set up to function (e.g. the KV layer). You are free to browse the code at https://github.com/cockroachdb/cockroach if you want a more in-depth look.