Time between events is an approach Montgomery (8th EMEA) discusses in 7.3.5. The application there is for dealing with very low error/defect rates. I am not familiar with Wheeler's approach.
23,573 karma · joined May 28, 2008
I am frequently wrong. I hope you will not spare my feelings when I am.
For my CV: https://www.linkedin.com/in/jacqueschester/
For personal email: hn@jacques.chester.id.au
For sure -- but these are surely special or assignable causes of variation. That's one of the key insights in SPC: that some variations are ordinary and some out of the ordinary. A "fix the bugs" campaign is out of the ordinary.
> Which is to say, different managers will get different means and standard deviations even if the devs change nothing about their work habits.
If they have a bad measurement rubric, certainly. But that in turn goes to questions of gauge reliability, a topic about which SPC has a lot to say.
I should say by way of agreement that I'm not convinced that SPC can be safely and effectively applied to software development processes. But I'm also not convinced it can't. I don't have a firm, final position on the matter. I would need to apply it for a while and see for myself.
This causes confusion in my experience. "Stable" doesn't mean what it does in casual usage. It means that you have a mean and variance that are not varying over time (and there are ways to alert if they are). A stable process can lurch around like a drunken sailor and still be considered stable for SPC purposes.
I think bugs should be stable if you normalize it by a size metric. So bugs/SLOC or bugs/story points.
In the past I've also tried the trick of taking the logarithm of values to convert a power distribution into a more normal-ish distribution. My project was to see if I could alert on an increased rate of uploads to RubyGems, signaling a possible malware campaign. Time between uploads follows a power law. I used the logarithm trick to convert it -- though not successfully. There are actually multiple distributions -- the "random" one which is prominent, but also other peaks centered around 1 second, 2 seconds, 3 seconds etc. Basically the detection was made difficult because of mass uploads being rate-limited on the uploader's side.
Two reasons come to mind. First, for whatever reason, it might not be economical to measure every individual in the population. There might be too many, or the thing being sampled is a continuous flow (eg liquid product in a chemical processing plant) or perhaps the sampling is destructive (imagine a pressure failure test, or a reagent test).
Second, you might just naturally have a sample. For example, each YJIT benchmark is run multiple times because there's variability in the measurement not due to changes in YJIT. Since you have n runs of the benchmark, you naturally have a sample size of n. It doesn't make sense to think of them as consecutive measurements.
The rest of your comment goes to the business of selecting what the grouping of samples is (the term is "rational subgrouping"). A lot of pages are given to this question in the SPC books I've read, because it's not difficult to unintentionally mask signal by combining things that shouldn't be combined. An example is checking the precision of piston machining. It might be you take all the pistons in a given engine as your sample. But later you discover that one of your four machining stations is off-center, which was masked by the ordinary variation of the other three. In this case the sampling should have been per-station, not per-engine.
I can't think of a reference for the different charts off the top of my head. Certainly working SPC-kit helped me to better understand the differences and how they relate to each other. But that's not a very scalable way to transmit information.
That Grafana plugin looks awesome. If I can shave off some time from endless leetcode grinding I might play with it.
Unfortunately the GitHub Linguist grammar for SQL doesn't recognize PostgreSQL comments, so highlighting will be wonky in places.
The background is that I've thought about applying SPC to software behavior for a long time[0]. While I was at Shopify I began to think about using it for detecting regressions in the YJIT benchmark suite[1]. My first thought was to code a Ruby library (Shopify is a Ruby on Rails outfit), but I decided that it would be more accessible to more folks if it worked inside a database instead. I got caught in a 20% layoff at Shopify and that somewhat took the wind out of my sails, which is why it's been dormant for a year.
There are some limitations to be aware of in a software context.
First, sample sizes have to be constant. In a world of unreliable systems and networks this is a pretty tough constraint. It could be achieved by taking fixed subsamples at random from a varying larger sample (eg, normally we get 100, pick 50 at random). Switching to variable sample size would be possible, but it will be a lot of work and I haven't had the motivation to tackle it yet. It's also only lightly treated in the main textbook I've worked from, because the usual focus is manufacturing operations where fixed sample sizes are common.
Second, it doesn't deal with non-parametric distributions. Classical SPC is rooted in the Normal distribution, and continues to work quite well with any distribution that has some kind of centrally-located mass that tapers off to the edges. But a lot of software behavior follows power laws, especially Pareto distributions. There is SPC literature to deal with non-normal distributions using non-parametric statistics. But I haven't bought and digested the relevant books.
It's worth noting that SPC is as much a system of problem solving as it is a bunch of statistical tools. Most major metrics vendors now offer some kind of anomaly detection, but then what? What are your out-of-control action plans? What is your concept of an acceptable amount of staying close to the mean? What is your target, and why is it your target? Can you even achieve such a target with the system as-is, or are you dreaming? SPC attends to these questions as well.
I can definitely recommend kqr's introduction[2], which is linked from the repo.
[0] https://theoryof.predictable.software/articles/what-is-predi...
[2] https://two-wrongs.com/statistical-process-control-a-practit...
It's the basis for YJIT, which has landed big speedups for the C Ruby implementation over the past few years.
Disclosure: I used to work in a team adjacent to Maxime's.
I recall reading an interviewing.io blog post[0] in which the dominant considerations interviewers weighed were (my interpretation):
(1) Did they solve the problem optimally? (2) How fluid was their coding?
With "communication" turning out to be basically worthless for predicting hire/no-hire decisions.
Perception of coding fluidity seems like it would be affected by how often the candidate stops and looks up things like library functions or obscure syntax.
For that reason I've been investing time in committing a lot of library functions to memory, so they instantly flow from my fingers rather than spending a minute looking it up.
It's dumb that I need to do this, but I don't make the rules. I'm just at the bottom of the information cascade that led to how things are done now.
[0] https://interviewing.io/blog/does-communication-matter-in-te...
I'm not asking LLMs to write code for me in this scenario -- I need to get by in interviews without AI assistance. But as a personal tutor it is turning out to be helpful.
DuckDB has an implicit "SELECT *" if you just give it "FROM tablename", which fits your example.
> SELECT 1;
Something like Oracle's "dual" table would handle this corner case. Or just selecting a value from itself:
FROM 1 SELECT 1;
FROM ::integer SELECT 1;
> Intelligent tooling can and does already solve this.
It can progressively guess as you type, but it can't provide an upfront list of alternatives for autocompletion. That is an annoyance.
What it produced was more of an essay written by a talented undergraduate about Borges, rather than Borges.
I'm sure Borges would've found LLMs fascinating and one can only dream what stories and essays would've been written.
For me at least this hasn't been true.
Right now I can't access any Slack channels, I can't access Google Drive/Docs, I can't send an email to an external address. I can only DM people and email people in the company.
In about 15 minutes I'll be shut out entirely.
As far as such things go it seems like a reasonable balance of humane consideration and legal prudence.