crdrost

So, there is a reason that CRDT researchers would not like this response that you have given, but down-thread from you it's not why the author jakelazaroff didn't like it, but it's worth giving this answer too.

The reason CRDT researchers don't like the sync server is, that's the very thing that CRDTs are meant to solve. CRDTs are a building-block for theoretically-correct eventual consistency: that's the goal. Which means our one source-of-truth now exists in N replicas, those replicas are getting updated separately, and now: why choose eventual consistency rather than strong consistency? You always want strong consistency if you can get it, but eventually, the cost of syncing the replicas is too high.

So now we have a sync server like you planned? Well, if we're at the scale where CRDTs make sense then presumably we have data races. Let's assume Alice and Bob both read from the sync server and it's a (synchronous, unencrypted!) last-write-wins register, both Alice and Bob pull down "v1" and Alice writes "v1a" to the register and Bob in parallel writes "v1b" as Alice disconnects and Bob wins because he happens to have the higher user-ID. Sync server acknowledged Alice's write but it got lost until she next comes online. OK so new solution, we need a compare-and-swap register, we need Bob to try to write to the server and get rejected. Well, except in the contention regime that we're anticipating, this means that we're running your sync server as a single-point-of-failure strong consistency node, and we're accepting the occasional loss of availability (CAP theorem) when we can't reach the server.

Even worse, such a sync server _forces_ you into strong consistency even if you're like "well the replicas can lose connection to the sync server and I'll still let them do stuff, I'll just put up a warning sign that says they're not synced yet." Why? Because they use the sync server as if it is one monolithic thing, but under contention we have to internally scale the sync server to contain multiple replicas so that we can survive crashes etc. ... if the stuff happening inside the sync server is not linearizable (aka strongly consistent) then external systems cannot pretend it is one monolithic thing!

So it's like, the sync server is basically a sort of GitHub, right? It's operating at a massive scale and so internally it presumably needs to have many Git-clones of the data so that if the primary replica goes down then we can still serve your repo to you and merge a pull request and whatever else. But then it absolutely sucks to merge a PR and find out that afterwards, it's not merged, so you go into panic mode and try to fix things, only for 5 minutes later to discover that the PR is now merged. And if you've got a really active eventually consistent CRDT system that has a lot of buggy potential.

For the CRDT researcher the idea of "we'll solve this all with a sync server" is a misunderstanding that takes you out of eventual-consistency-land. The CRDT equivalent that lacks this misunderstanding is, "a quorum of nodes will always remain online (or at least will eventually sync up) to make sure that everything eventually gets shared," and your "sync server" is actually just another replica that happens to remain online, but isn't doing anything fundamentally different from any of the other peers in the swarm.

"When data subjects exercise one of their rights, the controller must respond within one month. If the request is too complex and more time is needed to answer, then your organisation may extend the time limit by two further months, provided that the data subject is informed within one month after receiving the request."

Backup retention policy 60 days, respond within a week or two telling someone that you have purged their data from the main database but that these backups exist and cannot be changed, but that they will be automatically deleted in 60 days.

The only real difficulty is if those backups are actually restored, then the user deletion needs to be replayed, which is something that would be easy to forget.

Upvoted for the cute proof- without-words geometrical diagram of the Legendre transform, but the fact that you defined the inverse map as (x, y) to (\hat y, \hat x) I found impossible to keep my head straight. Probably it's easy if I slow down and stop skimming the article.

IMO the easier derivation — may just be personal tastes as someone more on the engineering side — is just another integration by parts.

So with f(g(x)) = g(f(x)) = x, define y = g(x) for U-substitution with x = f(y):

    ∫ g(x) dx = ∫ g(f(y)) f'(y) dy
              = ∫ y f'(y) dy
              = y f(y) – ∫ f(y) dy
              = g(x) x – F(g(x)) + C

The more interesting thing is that this is a really basic integration by parts which means that this diagram of yours that I like, is more universal than it appears at first? I'd have to think about that a bit more, how you can maybe graphically teach integration by parts that way, is there always a u substitution so that you can get u f(u) or so and get this nice pretty rectangle in a rectangle... hmm.

So for example the capital O on license plates in California is only distinguished from the zero by being slightly more squarish, the capital G is mostly distinguished from six by six being slightly more smooth and diagonal in its top arc. I and one are a bit further visually, as are B and 8, but it would probably fool a traffic camera that was taking down plates automatically.

In addition, all-numbers-plates, I believe, are reserved by California exempt plates (emergency vehicles, police), and vanity plates are absolutely a thing, much more likely to start and/or end on a letter, so that's why you see numbers at the beginning and end. Like you can kinda see “6EIC023” and say “oh yeah my car looks like an ad for Geico” but because the start and end are numbers it doesn't occur to most people.

The enum idea is often wise; also: for just an example that has probably occurred a hundred thousand times across the world in various businesses...

Original design: store a row that needs to be reported to someone, with an is_reported column that is boolean.

Problem: one day for whatever reason the ReporterService turns out to need to run two of these in parallel. Maybe it's that the reporting is the last step after ingestion in a single service and we need to ingest in parallel. Maybe it's that there are too many reports to different people and the reports themselves are parallelizable (grab 5 clients, grab unreported rows that foreign key to them, report those rows... whoops sometimes two processes choose the same client!)... Maybe it's just that these are run in Kubernetes and if the report happens when you're rolling pods then the request gets retried by both the dying pod and the new pod.

Alternative to boolean: unreported and reported records both live in the `foo` table and then a trigger puts a row for any new Foos into the `foo_unreported` table. This table can now store a lock timestamp, a locker UUID, and denormalize any columns you need (client_id) to select them. The reporter UPDATEs a bunch of rows reserving them, SELECTs whatever it has successfully reserved, reports them, then DELETEs them. It reserves rows where the lock timestamp IS NULL or is less than now minus 5 minutes, and the Reporter itself runs with a 5 minute timeout. The DB will do the barest amount of locking to make sure that two UPDATES don't conflict, there is no risk of deadlock, and the Boolean has turned into whether something exists in a set or not.

A similar trick is used in the classic Python talk “Stop Writing Classes” by @jackdied where a version of The Game of Life is optimized by saying that instead of holding a big 2D array of true/false booleans on a finite gameboard, we'll hold an infinite gameboard with a set of (x,y) pairs of living cells which will internally be backed by a hashmap.

I believe _Algernon_ was making a joke about how you could take this problem and make the code even worse, not proposing a solution to the problem.

It's also not even a problem with slow computers or insufficient memory, __init__ does I/O here, it connects to ZeroMQ, so it could have arbitrary latency in various circumstances exceeding the 100 milliseconds that we would be sleeping for. So the joke is, this fixes the problem in your test environment where everything is squeaky clean and you know that ZeroMQ is reachable, and now you have bugs in prod still.

I don't think it's totally crazy to, say, open a DB connection in __init__() even though that's not an instantaneous operation. That's not a hill I would die on, you can just say “open a connection and hand it to me as an argument,” but it just looks a little cleaner to me if the lifecycle of the connection is being handled inside this DB-operations class. (You can also defer creating the connection until it is actually used, or require an explicit call to connect it, but then you are also writing a bunch of boilerplate to make the type checker happy for cases where the class is having its methods called before it was properly connected.)

So in this case that would be, a FooBarWidget is not a subclass of FooWidget but maybe still a subclass of AbstractWidget above that. It contains a thread and config as its state variables. That thread instantiates a FooWidget with the saved config, and runs it, and finally closes its queue.

The problem still occurs, because you have to define what it means to close a FooBarWidget and I don't think python Thread has a “throw an exception into this thread” method. Just setting the should_exit property, has the same problem as the post! The thread might still be initing the object and any attempt to tweak across threads could sometimes tweak before init is complete because init does I/O. But once you are there, this is just a tweak of the FooWidget code. FooWidget could respond to a lock, a semaphore, a queue of requests, any number of things, to be told to shut down.

In fact, Python has a nice built-in module called asyncio, which implements tasks, and tasks can be canceled and other such things like that, probably you just wanted to move the foowidget code into a task. (See @jackdied's great talk “Stop Writing Classes” for more on this general approach. It's not about asyncio specifically, rather it's just about how the moment we start throwing classes into our code, we start to think about things that are not just solving the problem in front of us, and solving the problem in front of us could be done with just simple structures from the standard library.)

So when you've got an interval you usually mean two sounds that are separated in time. So like the iconic Jaws Melody dun-dan-dun-dan-dun-dan, those notes are separated by an interval that could be called one semitone, 100 cents, or a minor second, depending on who is talking.

Or in “Oh when the Saints Go Marching In,” the ‘Oh-when’ interval is two tones (four semitones), 400¢, or a major third, the ‘when-the’ interval is another minor second, and the ‘the-Saints’ interval is one tone or a major second. Adding those up we find out that “oh-Saints,” if you just omit the other words, is 700¢ or a “perfect fifth”, so “saints-Go” is a descending perfect fifth, -700¢.

Now you can play all four notes at the same time and you would still refer to these distances between the notes as intervals, but nobody is likely to describe this sound as a bunch of intervals. It is a “I(add 4) chord” in that context and the +100¢ interval between the major third and the perfect fourth is what gives it its spiciness.

So then you have to clarify whether you mean that we are playing one note first and then two notes together second, or are we playing all three notes at the same time, or are we playing all three notes separately.

If it's one and then two, or two then one, the higher note of the dyad will sound like the melody usually, and you'll reckon the interval between those two. People who have really well trained musical ears, instead hear the shift on the lowest note, but it requires training.

If you mean that all three are separated by time, then it's a melody. In this case these first four notes of “Oh When the Saints Go Marching In” would perhaps be described maybe as an arpeggiated major chord with a passing tone, same as I said earlier as “I(add 4).” I'm not actually 100% sure if that's the right use of the term passing tone or whether passing tones have to lie outside your scale or something.

If the three notes are played at the same time, that's a chord, specifically it's a triad chord. You might talk about the stacked intervals in that chord, a major chord stacks a minor third on a major third, a minor chord stacks a major third on a minor third, stacking major on major is augmented, stacking minor on minor is diminished, and there are suspended chords where you don't play either third, so sus2 stacks a fourth atop a second and sus4 stacks a second atop a fourth. So a lot of those have their own names, and some of those names get weird (like to stack a fourth on a fourth you might say “Csus4/G,” which treats the lowest note G as if it were the highest note but someone decided to drop it down an octave).

If you just used lead the billet would be 31lb, it'd be procurable and shippable for under $100.

Osmium is about twice as dense so yeah that would still be shippable at around 60 lb... I don't think that's even tens of thousands at that point? Isn't it going to be in the millions? I'm just thinking that like if you went for gold instead, 50 lb is 800 oz, at $3300/oz these days doing this with gold is $2.64M, no? And surely osmium blows gold out of the water by like 10x, right?

Jokes on you when the $100 of insurance is all you get back from the post office...

Edit: a sibling comment points out that you can probably do this under $100 with tungsten too and get up into that 50lb+ range.

So you are right that the random number becomes "password-like", but of course if you really care about that you don't need to store the exact random number, just as you don't have to store a raw password.

However, if your random login code is easily typable then it's usually drawn from a small enough keyspace that any such code is trivially brute-forceable. Like if it's decimal digits you need more than ten.

So in practice people just trust that you are on good terms with your email provider and anyone else with access to your email, and use other mechanisms to limit access to these codes from insiders.

Yeah it's especially a bit humbling when you realize that the vast majority of critters are some form or another of carnivore/omnivore and there's only a trace few herbivores and they're all the "pests" like aphids that we are constantly doing battle with... Except when they are things like cows that we ourselves eat.

Plants are naturally a bit protected from animal predators by a simple economics of scale: proteins require nitrogen, we can't get it from the air even though it's abundant there, we need to get it from nitrogen fixing bacteria but those are farmed by plants in their plant roots... But plants are mostly cellulose and don't have much nitrogen by proportion. But then they are also actively protected by being extremely effective poison manufacturers, using their rigid cell walls to put things at high pressures, growing around pests, cutting off resources to leaves and regrowing them after the predator moves on, and attracting predators of their predators like ladybugs.

But the networks of interdependence that we eventually form should perhaps not be put into a traditional exploiter/exploited juxtaposition. “How can I tweak myself so that you find me indispensable?” is a question both the humans and the plants are asking of themselves vis-a-vis each other.

This is the list they compiled, with a little hash if it was in the DSM-V before this study:

• Persecutory/paranoid #

• Reference #

• Family/relatives

• Grandiose #

• Passivity

• Schneiderian

• Neighbours/friends/associates

• Spied on/watched

• Paranormal

• Sexual

• Poisoning

• Religious

• Control #

• Mindreading

• Special powers/skills

• Infidelty

• Bizarre #

• Perception

• Thought broadcast #

• Somatic #

• Police/secret agent/army

• Religious leader/God/Prophet/Saint

• Thought insertion #

• Jealousy #

• Guilt/sin

• Made affect

• Hypochondriacal

• Fantastic delusional memories

• Thought withdrawal

• Erotomanic heterosexual #

• Possession

• Primary

• Catastrophe/world catastrophe

• Poverty

• Nihilistic/negation

• Spy/surveillance

• Made impulse

The authors’ primary goal appears to not have been to shed light on the diversity of delusional experience, but to indicate some evidence that it is culturally sensitive, so right after this chart, we get into the discussion section and it discusses how in East Asia you can find slightly more delusions of jealousy, in Eastern Europe you can find slightly more delusions of guilt/sin, in their dataset.

Reinterpreting it as this press release does, as a survey of the variety of delusional experience, makes for some interesting food for thought, but a lot of the categories identified kind of are just who are you being paranoid about, and what do you think they are doing. So you might have thought that people are afraid of strangers, this data set points to them being more delusional about family members, or thinking strangers are government spies, it's perhaps surprisingly not a stranger as a stranger that is terrifying? (Compare e.g. with the widespread worry among parents that a stranger might abduct their child.)

The passivity delusion (“I’m not in control of my body, someone else takes over”) and Schneiderian delusion (which I think is “someone is narrating my life in a Stanley Parable-esque way”?) seem like solid omissions on the DSM-V's part, but after that I feel like I am scrolling down to hypochondriacal and “Fantastic delusional memories” (which is presumably something like “I actually went to Narnia when I was a kid, it's a real place”?) before I see things that genuinely strike me as, I didn't realize some folks have those delusions in their schizophrenia or whatever.

Yes! But it's not a true quine because it returns “something functionally equivalent to myself,” not “my own source code.” so then I debated what else to call it before deciding that I was already taking too long writing an HN comment and it didn't matter what it was called, haha. Well spotted.

No, although you do see that in set theory, but this is a specific idea about the type signatures and encodings.

List type:

    type Stack<x> = null | {first: x, rest: Stack<x>}

The church encoding of a data structure, is to go through all of these cases separated by |, and define a handler for each one, and then define a function which accepts all those handlers and calls the appropriate one for its case:

    type StackF<x> = <z>(
        ifNull: (n: null) => z,
        ifItem: (first: x, rest: z) => z
      ) => z;

The Scott encoding is actually a little more practical, and has `rest: StackF<x>` in that second handler, but that's not what we have here.

Now the point is that a unit type like `null` that only has one possible value, that first handler can actually be simplified to just `ifNull: z`. Similarly if we consider a list of nulls, we would get

    type NullStackF = <z>(
        ifEnd: z,
        ifItem: (rest: z) => z
      ) => z;

And after swapping the arguments and currying, you get precisely the type of the Church numerals.

The Church encoding of a list is effectively replacing that list with its reduce function, so that's why I am saying that it's basically Array prototype.reduce on an Array<null> type. (I actually think it's reduceRight but this type is symmetric under .reverse() so who gives a darn). So it's

    const zero_ = f => [].reduce(f)
    const one_ = f => [null].reduce(f)
    const two_ = f => [null, null].reduce(f)

in JS. But if you had just Scott-encoded the numbers instead, you would have had no problems with pred() but you need the Y combinator to actually use the numerals for something interesting:

    type ScottNum = <z>(next: (rest: ScottNum) => z) => (z: z) => z;

    const zero: ScottNum = _f => x => x
    const one: ScottNum = f => _x => f(zero)
    const two: ScottNum = f => _x => f(one)
    function succ(n: ScottNum): ScottNum {
      return f => _x => f(n)
    }
    function pred(n: ScottNum): ScottNum {
      return n((x: ScottNum) => x)(zero)
    }
    function church<z>(n: ScottNum, f: (z: z) => z): (z: z) => z {
      return z => n(m => f(church(m, f)(z)))(z)
    }

But surely you have seen that places which practice design reviews and code reviews and double down on building tests, tend to be slower shippers and deadline missers?

Not saying that you in your ideal company suffer this problem, just the vast majority of the people who have taken your advice...

I come to you with a major refactor, +400 lines -600 lines, so like if you printed it out it's a 25-page diff, if someone is now reviewing my design or coding, they potentially have half an hour to an hour's work trying to fully understand this and make sure that I didn't break anything in that refactor. Post-hoc ain't the way here. I sometimes feel like I'm the only person who will read through such a long diff and understand it and come up with useful feedback. Everybody else is going to lean on testing, which to be fair to you is one of your bullet points, but consider that it's one of the major points.

And why did the refactor get that big in the first place? Because of the merge latencies, because of the review process. The more resistance you place here, the more gets buffered into a feature branch, and the more you have to review later.

Some shops do well with that, I worked at one once, we actually worked ourselves out of a job by finishing everything ahead of deadlines. (We were making a game that wasn't very fun at a company that was not principally interested in making games, so we were forcing the issue to a head of whether they wanted to keep taking this risk with a dream team of programmers or cut their losses and double down on their core.) But the major thing that we were doing different, has nothing to do with your bullet points. In the abstract, it was that, we didn't have performance review hanging over our heads. And therefore we didn't have every single developer working on their own separate thing in the system, so that they could call it their own and claim it on paper at PerfTime. Which meant that we could pair program organically, “hey you mind if I look over your shoulder?”. We all merged into dev, everyday, multiple times a day, and that's how we saw that someone else was working on the same part of the code that we were working on, and then we talked about like how are we going to not step on each other's feet? We had also decided early on that we were going to have a centralized place to push out feature toggles, it was kind of janky but it was available for QA department, yes we had a separate QA department, so that they could tweak which things were going to go out versus which needed to be baked more.

Yes.

Lambda calculus is about template substitution. It is the abstract mathematics of substituting templates into other templates. Alonzo Church got interested in this when he realized that it had numbers,

    type NumF = <X>(f: (x: X) => X) => (x: X) => X
    const zero: NumF = f => x => x
    const one: NumF = f => x => f(x)
    const two: NumF = f => x => f(f(x))

Addition is not too hard to define, multiplication is actually just function composition, and decrement is an unexpected motherf$@&er. To understand this last point, it may help to understand that this encoding of numbers is basically zero as [], one as [null], two as [null, null], etc and you are trying to use Array.prototype.reduce to compute the array with one fewer element. So you can do it with a (last, curr) pair or a null initial value, you have to know what those look like in the lambda calculus to translate,

    // predecessor and successor
    const succ = n => f => x => n(f(x))
    // Maybe fns: in λ-calc you'd uncurry
    type MaybeF<x> = <z>(ifNil: z, ifJust: (x: x) => z) => z
    const nil: MaybeF<any> = (ifNil, _ifJust) => ifNil
    function just<x>(x: x): MaybeF<x> {
      return (_ifNil, ifJust) => ifJust(x);
    }

    const pred = n =>
        n(maybe_n => maybe_n(just(zero), k => just(succ(k))))(nil)
      

Now you asked for a technique to remember the Y combinator. The basic thing to remember is the infinite template substitution discovered by Haskell Curry,

    (f => f(f))(f => f(f))

Forgetting everything you know about programming languages, and just thinking about template substitution, this is a template on the left, being substituted with a value on the right, and the reason it's an infinite loop is that after the substitution you get the exact same expression that you had before the substitution. [On the right we could have also written g => g(g) if that helps, the two variables are defined in separate scopes and only happen to have the same name f to make it clearer that this will loop infinitely if you try to eagerly evaluate it.]

Terms like this, were why Curry wanted a type theory. He immediately understood that this was kind of a weird expression because if you tried to express the idea of f(f) it was secretly holding on to a sort of paradox of self-reference or an infinity or whatever you want to call it. It is a function type which takes only the same function type as an argument, and returns God-knows-what. If you just forbid self-reference in the typing judgments, you don't get this sort of weird f(f) pathology.

So if you are remembering f(f) being somehow the key to the Y combinator, then you immediately see that you can kind of step it by not immediately evaluating it,

    const Q = (f => () => f(f))(f => () => f(f))

so as a matter of algebra Q() = Q although a programming language struggles to prove equality between functions so as a matter of programming that might not be equal. So this is a function you can call Q()()()() and it's just infinitely callable without producing any other values.

But why () =>, why not x => ? Well, you can introduce that but Q will just ignore it, it doesn't know what to do with that. So you need to write f(f)(x) to plumb it through to do something.

Note briefly that there IS a difference between f(f) and x => f(f)(x). The latter delays execution of f(f) until needed and so is still stepped, the former is an infinite loop.

And the last thing is that you need to hand this internal function to someone else, some decider, to decide when to call the recursion. This leads to the full form,

    const Y = decider => (f => decider(x => f(f)(x))(f => decider(x => f(f)(x)))

Usage with an example decider:

    const factorial = Y(recurse => n => n <= 1 ? 1 : n * recurse(n - 1));

Satvik gave you a fine link in a sibling comment, but I like to add something that I call "shell services" so let me give maybe the smallest example I've got of what it looks like, this is a little Python lambda. First you have a core module, which only holds data structures and deterministic transforms between them:

    core/app_config.py   (data structures to configure services)
    core/events.py       (defines the core Event data structure and such)
    core/grouping.py     (parses rule files for grouping Events to send)
    core/parse_events.py (registers a bunch of parsers for events from different sources)
    core/users.py        (defines the core user data structures)

(there's also an __init__.py to mark it as a module, and so forth). There is some subtlety, for instance events.py contains the logic to turn an event into a slack message string or an email, an AppConfig contains the definition of what groups there are and whether they should send an email or a slack message or both. But everything here is a deterministic transform. So for instance `parse_event` doesn't yet know what User to associate an event with, so `user.py` defines a `UserRef` that might be looked up to figure out more about a user, and there is a distinction between an `EventWithRef` which contains a `list[UserRef]` list of user-refs to try, and an `Event` which contains a User.

Then there's the services/ module, which is for interactions with external systems. These are intentionally as bare as possible:

    services/audit_db.py      (saves events to a DB to dedupe them)
    services/config.py        (reads live config params from an AWS environment)
    services/notification.py  (sends emails and slack messages)
    services/user_lookup.py   (user queries like LDAP to look up UserRefs)

If they need to hold onto a connection, like `user_lookup` holds an LDAP connection and `audit_db` holds a database connecction, then these are classes where __init__ takes some subset of an AppConfig to configure itself. Otherwise like for the email/slack sends in the notification service, these are just functions which take part of the AppConfig as a parameter.

These functions are as simple as possible. There are a couple audit_db functions which perform -gasp- TWO database queries, but it's for a good reason (e.g. lambdas can be running in parallel so I want to atomically UPDATE some rows as "mine" before I SELECT them for processing notifications to send). They take core data structures as inputs and generate core data structures as outputs and usually I've arranged for some core data structure to "perfectly match" what the service produces (Python TypedDict is handy for this in JSON-land).

"Simple" can be defined approximately as "having if statements", you can say that basically all if/then logic should be moved to the functional core. This requires a bit of care because for instance a UserRef contains an enum (a UserRefType) and user_lookup will switch() on this to determine which lookup it should perform, should I ask LDAP about an email address, should I ask it about an Amazon user ID, should I ask this other non-LDAP system. I don't consider that sort of switch statement to be if/then complexity. So the rule of thumb is that the decision of what lookups to do, is made in Core code, and then actually doing one is performed from the UserLookupService.

If you grok type theory, the idea more briefly is, "you shouldn't have if/then/else here, but you CAN have try/catch and you CAN accept a sum type as your argument and handle each case of the sum type slightly differently."

Finally there's the parent structure,

    main.py        (main entrypoint for the lambda)
    migrator.py    (a quick DB migration script)
    ../sql/        (some migrations to run)
    ../test/       (some tests to run)

Here's the deal, main.py is like 100 lines long gluing the core to the services. So if you printed it it's only three pages of reading and then you know "oh, this lambda gets an AppConfig from the config service, initializes some other services with that, does the database migrations, and then after all that setup is done, it proceeds in two phases. In the first ingestion phase it parses its event arguments to EventWithRefs, then looks up the list of user refs to a User and then makes an Event with it: then it labels those events with their groups, it checks those groups for an allowlist and drops some events based on the allowlist, otherwise it inserts those events into the database, skipping duplicates. Once all of that ingestion is done, phase two of reporting starts, it reserves any unreported records in the database, groups them by their groups, and for each group, tells the notification service, "here's a bunch of notifications to send" and for each successful send, we mark all of the events we were processing as reported. Last we purge any records older than our retention policy and we close the database connection." You get the story in broad overview in three pages of readable code. Migrator.py adds about two printed pages more to do database migrations, in its current form it makes its own DB connections from strings so it doesn't depend on core/ or services/, it's kind of an "init container" app except AWS Lambda isn't containerized in that way.

The test folder is maybe the most important part, because based on this decoupling,

- The little pieces of logic that haven't been moved out of main.py yet, can be tested by mocking. This can be reduced arbitrarily much -- in theory there is no reason that a Functional Core Imperative Shell program needs mocks. (Without mocks, the assurance that main.py works is that main.py looks like it works and worked previously and hasn't changed, it's pure glue and high-level architecture. If it does need to change, the assurance that it works is that it was deployed to dev and worked fine there, so the overall architecture should be OK.)

- The DB migrations can be tested locally by spinning up a DB with some example data in it, and running migrations on it.

- The core folder can be tested exhaustively by local unit tests. This is why it's all deterministic transforms between data structures -- that's actually, if you like, what mocking is, it's an attempt to take nondeterministic code and make it deterministic. The functional core, is where all the business logic is, and because it's all deterministic it can all be tested without mocking.

- The services, can be tested pretty well by nonlocal unit "smoke"/"integration" tests, which just connect and verify that "if you send X and parse the response to data structure Y, no exception gets thrown and Y has some properties we expect etc." This doesn't fully test the situations where the external libraries called by services, throw exceptions that aren't caught. So like you can easily test "remote exists" and "remote doesn't exist" but "remote stops existing halfway through" is untested and "remote times out" is tricky.

- The choice to test stuff in services, depends a lot on who has control over it. AuditDBService is always tested against another local DB in a docker container with test data preloaded, because we control schema, we control data, it's just a hotspot for devs to modify. config.py's `def config_from_secrets_manager()` is always run against the AWS Secrets Manager in dev. UserLookupService is always tested against live LDAP because that's on the VPN and we have easy access to it. But like NotificationService, while it probably should get some sort of API token and send to real Slack API, we haven't put in the infrastructure for that and created a test Slack channel or whatever... so it's basically untested (we mock the HTTP requests library, I think?). But it's also something that nobody has basically ever had to change.

Once you see that you can just exhaustively test everything in core/ it becomes really addictive to structure everything this way. "What's the least amount of stuff I can put into this shell service, how can I move all of its decisions to the functional core, oh crap do I need a generic type to hold either a User or list[UserRef] possible lookups?" etc.

Yes! I call this sort of top-down programming "wishful thinking." It is these days much easier to explain to people, because machine learning tools.

“if you can just trust that chat GPT will later fill in whatever stub functions you write, how would you write this program?” — and you can quickly get going, “well, I guess I would have a queue, while the queue is not empty I pull an item from there, look up its responsible party in LDAP, I guess I need to memoize my LDAP queries so let's @cache that LDAP stub, if that party is authorized we just log the access to our S3-document, oh yeah I need an S3-document I am building up... otherwise we log AND we add the following new events to the queue...”

It is not the technique that has most enhanced what I write, which is probably a variant on functional core imperative shell. But it's pretty solid as a way to break that writers block that you face in any new app.

So that was a quote from a piece from the same author as the original article.

I'm trained in physics, not medicine, so I am somewhat reluctant to give my own take. I have not been on SSRIs myself but I have been the responsible one for making sure that folks take them. But I would note these points about what I see in the general discussion:

• Nobody is saying the SSRIs aren't psychoactive compounds that could maybe be helpful for at least short-term intervention. Everyone agrees on at least those two things. So, nobody is telling you that you didn't feel something helpful with that second one.

• The psychiatric practice is indeed to “shop around” not just different drugs, but different generics of the same drug. To my mind this basically proves the point: if you had had a serotonin problem, the first SSRI would have fixed it and the only question would be “can you tolerate the side effects?”. So actually what's fixing the problem is a side effect of the SSRI, it is not the main effect of boosting serotonin, but it's the other ways in which these particular drugs happen to be psychoactive.

• Your first person anecdotal experience obviously is not a disproof of a placebo effect and it cannot be—this is I think you applying a popular misconception that placebo effects “don't feel real” or “don't last” or “aren't real medicine” or whatever, but they do, and they can, and they are. In these tests there is never a control that didn't get either the medicine or the placebo, which I actually find kinda frustrating—so “placebo” also means “control group.” It's just a measure of all the things that you didn't have control over. But it's called that because they give the control group a placebo. But like I don't have control over air quality, air quality can affect sleep quality, sleep quality can affect depression, if I start my experiment during a week of bad air quality and it gets better for the rest of the experiment, that generates both a “placebo” signal and a “test” signal, even though that's not the placebo effect. If you're peeking in as a physicist you've got to remember that the body heals the vast majority of our medical problems on its own, and that psychiatric problems are even moreso because “I made a new best friend at the XYZ study, now we go for walks every day together and complain about all the things that are making us more depressed this week” can fix a psychiatric problem much more easily than, say, putting cancer in remission. So like the body heals itself, the brain heals itself, the brain also has influence over physical context (“I am going to eat some broccoli and go on a walk today”), the brain also has influence over psychosocial context (“I’m gonna go no-contact with my abusive parent,” vs “I will get stuck curled in a ball in bed saying ‘I’m useless, I’m useless’ until my physics-major housemate has to remind me to get up and drink a sip of water and swallow my medicine.”) There's just so many ways the contexts are plastic, and they are all valid medicine in that context, going no-contact with an abusive parent can absolutely be therapeutic. So, placebo means control, it doesn't mean that the healing wasn't real, it just means that the healing was out of the control of the experiment, which was only looking at this particular drug.

• Finally, because these tests are done one drug at a time, this fact that they don't beat the placebo group with the test group, comes out even more nuanced. Remember that you are not being healed by the main effect of an SSRI but one of its side-effects. The question is, is that repeatable for others with depression or did it only work for your depression because it happened to take the edge off of this or that subsystem in your brain’s feedback loops, but that subsystem isn't critical to someone else's depression? Hypothesis: the failure of SSRIs to beat placebos in these tests, means that biologically there are 20+ different “depressions” and when you are “depressed” we don't know which one you have (or maybe you have even more than one!), and each of these drugs is only able to make an impact on say five of those different depressions, and whether it is worth the side effects depends on even more concerns. Meanwhile everyone in the control group gets to cure 4 of the depressions at random, say. Does your second medicine beat the placebo? Well, not for everybody. Not even for the vast majority. But for people in your particular circumstance it does. But we don't have the ability to isolate that circumstance.

In conclusion, if you are looking at medicine as a physicist, everything is f*¢#ed and it's so much nicer to play with my quantum dots and lasers and liquid nitrogen.

> Finally, Jauhar et al. argue that serotonin must be involved in depression because drugs which target the serotonin system are effective and other authors also argue that antidepressants ‘work’. However, whether antidepressants produce a genuine and useful pharmacological effect that is independent of the placebo effect, has not been established. Antidepressants show marginal differences from placebo, which do not fulfil criteria for clinical relevance, and may represent amplified placebo effects due to unblinding [31,32,33]. It is hard to reconcile even the most generous appraisal of their efficacy with the vast numbers of people now taking them. Contrary to Bartova et al’s claims, the idea that antidepressants reduce suicide has not been established, and evidence from randomised trials suggests they increase the risk of suicidality in some age groups [34, 35].

-- Monicreff et al. (2023), https://www.nature.com/articles/s41380-023-02094-z

This is the sort of long-form comment that I'd write and love to read, signed in just to upvote it, well-put!

Nit: chromosomes mostly don't look like little X-shapes except during cell division, which was historically a useful time to image them (I think because the dye couldn't get through the cell-nucleus wall but during cell division it has dissolved?). But then you are seeing _two_ chromosomes that happen to be bound at a centromere before that centromere gets torn into two by the cytoskeleton dividing the cell in half.

Just to add to this, the reason that the things are different is, stochastics as a subject is trying to do calculus in the presence of noise, and what noise does is, it makes your function nondifferentiable. You would think that you cannot do calculus, without smooth curves! But you can, but we have to modify the chain rule and define exactly what we mean by integration etc.

So the idea is “smooth curves do X, but non-smooth noisy curves do Υ(χ) where χ in some sense is the noise input into the system, and they aren't contradictory because Y(0) = X. (At least usually... I think chaos theory has some counterexamples where like the time t that you can predict a system’s results for, is, in the presence of exactly 0 noise, t=∞, but in the limit of nonzero noise going to zero, it's some finite t=T.)

That is a legitimate need, but it should not be the default.

Another example of when you need it, is games. I want to store in the game state, that on Day 32 of being in this cave, there is randomly a cache of diamonds if you mine the east wall. If I generate this random number dynamically, then I permit save-scumming behaviors which may distract from the artistic points of my game. So I basically want a random seed and a hash function, hashing (seed, day, activity, location, attempt_number) to see if something is successful.

So hens don't usually have to be force-fed. Some of that color can come from having a diverse source of proteins--like the bugs and insects that pasture-raised hens get access to--but farmers "in the know" will also add paprika and marigold to the usual soy-and-grain supplemental feed, to try to encourage it to come out a bit more.

At the most extreme end of the other side, Buddhist monks, in addition to not eating meat (or, often, aliums like onions and garlic), also don't generally believe in dinner—they have to eat all their solid food for the day before noon, so you could view this as fasting for half the day every day.

(There are some caveats... At least for the Tibetan monks I knew, morning prayer is early at like 5am and comes with a sort of pita bread and tea, bedtime is closer to 9pm, and during these 9 hours there will be more tea. With a little googling I am able to confirm that some of the "pita" (pao balep) is consumed at the lunch tea, and I think this is after the lunch meal, so it might be 1pm. I think there's none at the evening tea that you'd have around sundown? Also in terms of calories the Tibetan tea is “bulletproof,” consisting of a very long steep for the leaves to extract maximum bitter flavors, that get mixed with a bunch of yak butter and salt. So liquid calories are very much a thing for them.)

I just want to semi-hijack this thread to note that you can actually peek into the future on this issue, by just looking at the present chess community.

For readers who are not among the cognoscenti on the topic: in 1997 supercomputers started playing chess at around the same level as top grandmasters, and some PCs were also able to be competitive (most notably, Fritz beat Deep Blue in 1995 before the Kasparov games, and Fritz was not a supercomputer). From around 2005, if you were interested in chess, you could have an engine on your computer that was more powerful than either you or your opponent. Since about 2010, there's been a decent online scene of people playing chess.

So the chess world is kinda what the GPT world will be, in maybe 30ish years? (It's hard to compare two different technology growths, but this assumes that they've both hit the end of their "exponential increase" sections at around the same time and then have shifted to "incremental improvements" at around the same rate. This is also assuming that in 5-10 years we'll get to the "Deep Blue defeats Kasparov" thing where transformer-based machine learning will be actually better at answering questions than, say, some university professors.)

The first thing is, proving that someone is a person, in general, is small potatoes. Whatever you do to prove that someone is a real person, they might be farming some or all of their thought process out to GPT.

The community that cares about "interacting with real humans" will be more interested in continuous interactions rather than "post something and see what answers I get," because long latencies are the places where GPT will answer your question and GPT will give you a better answer anyways. So if you care about real humanity, that's gonna be realtime interaction. The chess version is, "it's much harder to cheat at Rapid or Blitz chess."

The second thing is, privacy and nonprivacy coexist. The people who are at the top of their information-spouting games, will deanonymize themselves. Magnus Carlsen just has a profile on chess.com, you can follow his games.

Detection of GPT will look roughly like this: you will be chatting with someone who putatively has a real name and a physics pedigree, and you ask them to answer physics questions, and they appear to have a really vast physics knowledge, but then when you ask them a simple question like "and because the force is larger the accelerations will tend to be larger, right?" they take an unusually long time to say "yep, F = m a, and all that." And that's how you know this person is pasting your questions to a GPT prompt and pasting the answers back at you. This is basically what grandmasters look for when calling out cheating in online chess; on the one hand there's "okay that's just a really risky way to play 4D chess when you have a solid advantage and can just build on it with more normal moves" -- but the chess engine sees 20 moves down the road beyond what any human sees, so it knows that these moves aren't actually risky -- and on the other hand there's "okay there's only one reason you could possibly have played the last Rook move, and it's if the follow up was to take the knight with the bishop, otherwise you're just losing. You foresaw all of this, right?" and yet the "person" is still thinking (because the actual human didn't understand why the computer was making that rook move, and now needs the computer to tell them that the knight has to be taken with the bishop as appropriate follow-up).

This sort of problem also occurs when you're trying to do CRDTs, which can roughly be described also as "design something that does Git better."

So e.g. to frame this, one approach to a CRDT is to just treat the document as a list of facts, "line 1 is 'foo', line 2 is 'bar'", and each fact has a number of times it has been asserted, and to "merge" you just add together the assertion counts, and then you can detect conflicts when a fact has been asserted more than once or fewer than zero times. So a patch says "change line 2 to 'baz'", this becomes "unassert that line 2 is 'bar', assert that line 2 is 'baz'" and it conflicts with a patch that says "change line 2 to 'quux'" because the fact "line 2 is 'bar'" has an assertion count of -1.

But anyway, in this context you might want to allow inserting lines, and then you have the list-labeling problem, you don't want the patch to unassert lines 4,5,6 just to insert a new line after line 3. So then an obvious thing is to just use a broader conception of numbers, say "line 3.5 is <X>" when you insert, and then we hide the line numbers from the user anyways, they don't need to know that internally the line numbers of the 7 lines go "1, 2, 3, 3.5, 4, 5, 6".

So then you need a relabeling step because you eventually have some line at 3.198246315 and you want to be able to say "yeah, that's actually line 27, let's have some sanity again in this thing."

This also maybe hints at the fun of adding randomization, consider that one person might add line 3.5, then add line 3.75, and then remove line 3.5; meanwhile someone else might add a different line 3.5, add a line 3.25, and then remove their line 3.5, and these patches would both amount to "assert line 3.25 is A, assert line 3.75 is B", and would merge without conflict. This means that in general if two people are messing with the same part of the same document asynchronously, this model is not able to consistently flag a merge failure in that case, but will sometimes instead just randomly order the lines that were added.

We can then just make that a feature rather than a fault: you don't insert at 3.5, which is 3 + (4 - 3) / 2, rather you insert at 3 + (4 — 3) * rand(). And then when two people both try to insert 12 lines between line 3 and 4 independently, when you merge them together, you get 24 lines from both, in their original orders but interleaved randomly, and like that's not the end of the world, it might be legitimately better than just declaring a merge failure and harrumphing at the user.

It's interesting that even this gives me an uncanny valley feeling, something about the fact that the motion of the stalk is "bottom-up" rather than "top-down" and that all of the stalks are trying to move in unison rather than wind kind of gently cascading over the institution.

I imagine the piece was supposed to be haunting in a completely different way -- so the idea would be that you'd be embedded in a space which is experiencing a "real" wind that you can't yourself feel on your skin, and so you'd feel like you took a step outside of reality. In that sense this would be an exhibit that a video probably can't do justice to? But I think the uncanny valley thing still sells it pretty well.

This is an impressively high sick-burns-to-words ratio, well done!

“I don't need to make an LLC” also pings way too hard here.