jacques_chester

"Compatible" isn't mentioned on the homepage, though, despite multiple opportunities to do so -- "Create, manage and query serverless PostgreSQL", "Run history, status, and metadata live in Postgres", "Full Postgres. Fully typed.".

This lack of detail may cause folks to form the incorrect impression that this is PostgreSQL, or a fork of it, or some module or plugin for it. Folks will be upset to learn that they were misinformed. Some will assign deception as the cause, whether that is true or not.

I think your interests would be best served by trying to make that distinction clear and prominently so. So for example "A PostgreSQL-compatible, fully serverless database", or similar.

I hope I have explained better.

> db9 is a PostgreSQL-compatible distributed SQL database. Your data is stored in a distributed TiKV cluster, and each database (tenant) gets its own isolated keyspace. [0]

I feel like the lede is a bit buried here, bordering on deceptive.

That or the architecture doc is wrong. Both plausible I guess, in this day and age.

[0] https://db9.ai/docs/sql

Quick aside, some dutch folks did a more language-y DSL for tax codes, which might be of interest. I don't know if it is still being used, though.

https://resources.jetbrains.com/storage/products/mps/docs/MP...

I would be interested in a layperson summary of how this code deals with whacky distributions produced by computer systems. I am too stupid to understand anything more complex than introductory SPC which struggles with non-normal distributions.

My guess is that it's a response to "Chainguard are growing so fast that VCs have fought each other to give them hundreds of millions in 3 years despite having no AI play".

I should add, to clarify: I don't work at Shopify anymore and I'm not speaking for them. Purely a personal view.

Glad to hear it's all so simple. So you'll have no problem setting it up and finding thousands of volunteers to help, right?

At Shopify I was the person who first proposed that we needed to stump up $$$ for RubyGems (and only by implication Ruby Central).

This is not what I had in mind and now I'm embarrassed that I helped make it possible.

Oh, this old chestnut. "Just do what the distros do".

OK, sure, let's pencil this out.

Debian has ~1k volunteers overseeing ~20k packages. Say the ratio is 20:1.

npm alone -- not counting other ecosystems, just npm -- has 3 million packages.

So you'd need 150k volunteers. One hundred and fifty thousand unpaid individuals, not counting original authors.

For one repo.

"Nonsense", you riposte. "Only maybe 100k of these packages are worth it!"

Cool, cool. Then you'd need "only" 5 thousand volunteers. Debian maxed out at 1k and it is probably the source of the most-used software in history. But sure, we'll find 5 thousand qualified people willing to do it for free.

Oh, but how do you identify those 100k packages? OK, let's use download count. Or maybe reference count. Network centrality perhaps? Great, great. But some of them will be evicted from this paradise of rigorous repackaging. What replaces them? Oh, shoot, we need humans to go over up to 3 million packages to find the ones we want to keep.

What I need distro boosters to understand is that the universe of what is basically a package manager for large C libraries is at least two orders of magnitude smaller than everything else, bordering on three if you roll all the biggest repos together. The dynamics at language ecosystem scale are simply different. Yelling at the cloud that it should actually be a breeze isn't going to change things.

I disagree with theses in this piece.

1. "2FA doesn't work". Incorrect. MFA relying on SMS or TOTP is vulnerable to phishing. Token or device based is not. And indeed GitHub sponsored a program to give such tokens away to critical developers.

In 2021.

2. "There's no signing". Sigstore support shipped in like 2023.

The underlying view is that "Microsoft isn't doing anything". They have been. For years. Since at least 2022, based on my literal direct interactions with the individuals directly tasked to do the things that you say aren't or haven't been done.

I have no association with npm, GitHub or Microsoft. My connection was through Shopify and RubyGems. But it really steams me to see npm getting punched up with easily checked "facts".

Most of the biggest repositories already cooperate through the OpenSSF[0]. Last time I was involved in it, there were representatives from npm, PyPI, Maven Central, Crates and RubyGems. There's also been funding through OpenSSF's Alpha-Omega program for a bunch of work across multiple ecosystems[1], including repos.

[0] https://github.com/ossf/wg-securing-software-repos

[1] https://alpha-omega.dev/grants/grantrecipients/

Trunk-based development, by itself, is a fool's errand.

But combine it with TDD & pairing and it becomes a license to deliver robust features at warp speed.

It's not for everyone. Some people have excellent reasons why it isn't workable for them. Others have had terrible experiences. It takes a great deal of practice to be a good pair and, if you don't start by working with an experienced pair, your memories of pairing are unlikely to be fond.

However.

I paired full-time, all day, at Pivotal, for 5 years. It was incredible. Truly amazing. The only time in my career when I really thrived. I miss it badly.

Interesting watching this part of the landscape heating up. For repos you've got stalwarts like Artifactory and Nexus, with upstart Cloudsmith. For libraries you've got the OG ActiveState, Chainguard Libraries and, until someone is distracted by a shiny next week, Google Assured Open Source.

Sounds like Pyx is trying to do a bit of both.

Disclosure: I have interacted a bunch with folks from all of these things. Never worked for or been paid by, though.

It's a great feature, but GitHub's parser chokes on it.

Compare:

https://github.com/jchester/spc-kit/blob/eb2de71d815b0057e20...

To:

https://github.com/jchester/spc-kit/blob/main/sql/02-spc-int...

Basically the original rendering makes me look incompetent to a casual skimmer. Plus tools like JetBrains IDEs can suss out what comments belong to what DDL anyway.

> Brian Kernighan and Rob Pike

Most of us aren't Brian Kernighan or Rob Pike.

I am very happy for people who are, but I am firmly at a grug level.

This triggered some associations for me.

Strongest was Cells[0], a library for Common Lisp CLOS. The earliest reference I can find is 2002[1], making it over 20 years old.

Second is incremental view maintenance systems like Feldera[2] or Materialize[3]. These use sophisticated theories (z-sets and differential dataflow) to apply efficient updates over sets of data, which generalizes the case of single variables.

The third thing I'm reminded of is Modelica[4], a language where variables are connected by relations (in the mathematical sense). So while A = B + C can update A on when B or C change, you also can update just A and B, then find out what C must have become.

[0] https://cells.common-lisp.dev

[1] https://web.archive.org/web/20021216222614/http://www.tilton...

[2] https://www.feldera.com

[3] https://materialize.com

[4] https://modelica.org

Grids! I knew I forgot something in my list. Updated.

Fair enough. If it makes you feel better, I have a very different project where I have deliberately avoided any AI-generated code: https://github.com/jchester/spc-kit

I'm using "control" in the statistical process control sense, where it means "we can tell if variation is ordinary or extraordinary".

To me it seemed clear that the paper is about detecting regressions, which is control under my definition above. I still think of that as distinct from optimization.

SPC gets used on complex dynamic systems all the time. It takes more work and more nuance, but it's doable. I don't see a categorical error here, it's about fine-tuning the details.

I think the confusion arises because of the difference between optimization and control, which are superficially similar.

Having control lets you see if things changed. Optimization is changing things.

This team seems to be focused on control. I assume optimization is left to the service teams.

I don't agree. This is basically an elaborate form of statistical process control, which has been proving itself useful and effective for nearly a century. We can quibble about the thresholds and false positive rates, but I think the idea of automating regression detection is perfectly sound.

The important difference is that sigstore enables a "single click" signing procedure with no faffing around with key material. How it works is much less important than the user experience, which is vastly better.

Nobody said it was. The point is that it's better.

Maintaining this capability isn't free, it is of dubious benefit and there are much better alternatives.

On a cost benefit analysis this is a slam dunk.

9 months sounds like plenty, but don't be hasty.

I've been unemployed for almost a year. I would not take a buyout in this engineering market. It is brutal.

I performed a similar analysis on RubyGems and found that of the top 10k most-downloaded gems, less than one percent had valid signatures. That plus the general hassle of managing key material means that this was a dead-end for large scale adoption.

I'm still hopeful that sigstore will see wide adoption and bring authorial attestation (code signing) to the masses.

1. SPC kit [0]. Once made it to the front page! [1]

It's an SQL library for doing statistical process control (SPC) calculations.

This has been a labour of love for about 2 years now. I work on it sporadically. Recently I got more disciplined about what I am working on and I am slowly closing the gap on a first 0.1 release.

2. Finding work. As much fun as it is to tinker, I am nursing the standard crippling addiction to food and shelter. I am also nursing an increasing loathing for LinkedIn and wish to be free of having to check it.

[0] https://github.com/jchester/spc-kit

[1] https://news.ycombinator.com/item?id=39612775

I remember seeing a VMware-internal presentation on the DDlog work which led to Feldera and being absolutely blown away. They took a stream processing problem that had grown to an hours-deep backlog and reduced it to sub second processing times. Lalith & co are the real deal.