Comment by dgb23 | Hacker News Reader

Comment by dgb23 | Hacker News Reader

Parent Full thread

This is very good advice and a great article. It comes up on this site now and then because of it.

For those who don't necessarily program in statically typed functional languages:

The idea transcends paradigms.

You'll find very similar notions in 80's/90's OO literature, for example in Design by Contract. I'm sure one can dig deeper and find papers, discussions and specifications that go further back.

I think TypeScript is often written in such a way where you refine the types at runtime. I assume Design by Contract has influenced Clojure's spec (Clojure is a dynamic language).

Fundamentally this is about assumptions and guarantees (or requiring and providing). Once an assumption is checked and guarantees can be made, then other parts of the program don't need to check overlapping assumptions again.

In fact I think one of the most confusing things when you read code is seeing already guaranteed properties being checked again somewhere else. It makes code harder to reason about and improve.

> is seeing already guaranteed properties being checked again somewhere else

Because at some point those "already guaranteed properties" might "disappear", to be more exact, the process/procedure that implements and runs them might not do its thing anymore for one reason or another.

When that happens, because statistically speaking it will happen, then all the other processes/scripts/pieces of code depending on that "original" validating process would be in a very rough place.

Yes, and that's what the approach from the article is trying to help with. If you're coding in a statically typed language, the "already guaranteed properties" are propagated by types, so they can't just "disappear" - there's no way to get from e.g. String to Username without going through a String->Username function that does the checks, so functions accepting Username can rely on it having the relevant properties.

Of course, this stops being true when you start hacking around the checks, side-stepping the parsers entirely (say, type-casting String to Username without any check). There's nothing a language can do to stop you when you really want to do this[0] - but then, you're an adult; if you see restrictions and safety interlocks, and put effort to hack around them, then any problem is really on you.

This gets more difficult in dynamically typed languages, as you have to rely more on naming conventions and programmers not being idiots, but even without a typechecker, it will be rather obvious when you're doing something that could break the chain of guarantees.

--

[0] - Some can try; I've seen a Haskell paper about this idea that does very complex type magic to try and truly ensure that only the parsing function can actually construct the result type. I tried reproducing that in C++ once, but C++ just can't give such guarantees.

> Some can try

Rust does this with standard types even. "Parse, don't validate" is the answer to the question: why does Rust have so many string types? Of course you can hack around this using `unsafe` but, as you say, in that case you have to put visible effort into deliberately subverting the type system.

Any language that allows private constructors makes this pretty easy, no?

The constructor doesn't have to be private, it just has to be a constructor.

For example take this library: https://github.com/google/uuid

It represents UUIDs as `type UUID [16]byte`.

It would be trivial to circumvent a constructor: `var uuid UUID`. Go values are zero initialized.

(Aside: You might actually have a reason to do this, but often you would rather use https://pkg.go.dev/github.com/google/uuid#NullUUID).

But it's obvious that you would use one of the constructors in the library to get an actual valid UUID.

In C++ it's less obvious, so while a private constructor does most of the job, it's still tricky to come up with a way of designing those types so they feel natural, are easy to construct in the right way (via appropriate parsers), impossible to construct the wrong way by accident, and fulfill all the roles people expect, and don't devolve into impenetrable mess of template metaprogramming that can only be debugged by the author of the type definition framework.

I understand where it comes from, but I can't ignore the feeling that this is anxiety driven development.

Sure, the farther away from your control and trust circle, the more you're inclined to check assumptions that are already guaranteed. It's a valid reason to do this as you explained, but I think the tradeoff has to be considered:

Generally speaking, checking assumptions that are already guaranteed is a _bug_. It violates the DRY principle[0], and will break your program, when those guarantees relax or the assumptions become tighter at one place or another, because they diverge.

And again, it can be confusing for maintainers and makes it harder to reason about a program and make sensible changes. The anxiety that drives the checks will leak right into the reader who might be wary of breaking Chesterton's Fence. Now you have someone testing everything in order to figure out if there are code paths that only hit one or the other of the checking code and stuff like that.

Needless to say it can also make performance worse, because you're doing more work than needed, especially if the checks require fetching data from disk or the network etc. This type of performance degradation is quite common.

[0] The real/actual one, not the one where people factor out superficially repetitive code.

When you have a language with a strong type system, this is one of the practical things that ultimately gives you freedom as your program gets bigger and more complex.

But you have to use it. E.g. by having a Class UncheckedEmail, a Class ValidEmail and a Class VerifiedEmail and ensuring that the conversion from one to the other has to involve your email-verification process.

That way you never have to guess whether the email adress is unchecked, valid or verified and there is no need for "is_email_verified" booleans that you may or may not forget to update/check. If you use the wrong thing in the wrong place your type checker yells at you, while you can focus on actually important stuff.

Better yet, skip the Valid* classes and only make classes for valid objects.

'Int' is fine. We don't need ActualInt and IPromiseItsReallyAnInt.

This was of course an example to illustrate the concept. So whatever makes sense in reality will differ depending on the usecase.

I assumed there is a typical registration process. Future users of your application input a text in a field that is meant to be a reachable email address. So the first step would be to filter out garbage or typoed text that your mail-sending function would not be able to handle. And because you don't know how long people need to click on a link in their mail you need to store and work with that validated-but-unverified email address till they do. And for this having your own type that cannot be mixed up with the verified email addresses can make sense. Depending on how complex your application is.

The idea is take even further in DDD. A core idea is to allow the creation of data-graphs which are always in a valid business state.

These so called aggregates can only be created from a factory to avoid bad initialization. This way you never need to make ad hoc validations to be “ really really reallysure”.

Int's are a very poor analogy.

Somewhat less bad would be IEEE 754 floating point numbers - where your floating-point "numbers" can include both +0 and -0, sub-normal numbers, infinities, NaNs, and other miseries.

> Int's are a very poor analogy.

Then program using very poor analogies, like Int.

I think you've really missed the point.

I regret shooting from the hip, and answering the comment directly rather than tying it back to the article.

String->(valid)Email is 'Parse', (invalid)Email->ValidEmail is 'Validate'.

> That way you never have to guess whether the email adress is unchecked

Now you just get to guess whether, for every class, there is actually a ValidClass that you were supposed to be using, instead of Class.

  authenticate(User user);

^ This is a bug. We just let a malicious User object in because we invested extra time, effort and sloc into making both a User and a ValidUser class, disregarding the first "simple idea" from the article:

>> 1. Use a data structure that makes illegal states unrepresentable

Or as I wrote:

> only make classes for valid objects

> Now you just get to guess whether, for every class, there is actually a ValidClass that you were supposed to be using, instead of Class.

The type system stops this. You can't pass, e.g. a User to authenticate because it only accepts a ValidUser.

> only make classes for valid objects

Using primitives (String, Int, etc.) for unvalidated input, and custom classes for validated input is fine in many cases. However, sometimes you need to represent data that is in the process of being validated (e.g. when validation takes time, like waiting for a user to validate their email address) and then these intermediate classes arrive.

I think, for most cases with partial validation, you should parse the fields first into valid types for those fields and store them in a hashmap or something. When all the fields have been collected, parse the hashmap into your aggregate type.

But your colleage just implemented it with User - because he could

No tool can fully protect against user incompetence (pun intended). You can have the fanciest type system in the world and still implement everything as a String, for example.

I'm curious how would look like a real example with different Email classes. It seems silly. You don't often need a VerifiedEmail. But VerifiedUser is more useful from VerifiedEmail. You could get an implicitly verified email without introducing a new type.

> You don't often need a VerifiedEmail. But VerifiedUser is more useful from VerifiedEmail. You could get an implicitly verified email without introducing a new type.

Be careful about how the email is implicitly verified though, and how different login methods and assumptions interact when for example migrating users from one system or platform to another.

https://krebsonsecurity.com/2024/07/researchers-weak-securit...

> analysis released by security experts at Metamask and Paradigm finds the most likely explanation for what happened is that Squarespace assumed all users migrating from Google Domains would select the social login options — such “Continue with Google” or “Continue with Apple” — as opposed to the “Continue with email” choice

> Squarespace never accounted for the possibility that a threat actor might sign up for an account using an email associated with a recently-migrated domain before the legitimate email holder created the account themselves

> since there’s no password on the account, it just shoots them to the ‘create password for your new account’ flow. And since the account is half-initialized on the backend, they now have access to the domain in question

This kind of mistake makes me think that it’s better to be super explicit about the state of everything when it comes to accounts.

So even if it may seem excessive to have VerifiedEmail as a type instead of marking the account as verified or not. I will prefer being explicit.

And in medium to big size systems with multiple pieces of user account data that require separately keeping track of verification state it will be necessary anyway. For example even something as simple as having one email and one phone number associated with a user. Or beyond that one user having multiple email addresses or multiple phone numbers etc.

Thank you for beinging up this excellent display of hidden dangers to my very crude example about why type systems should be used if they are there.

My point was specifically about complexity. Using a string for all kind of emails of course works, tying guarantees to other things like VerifiedUser works as well. These implicit guarantees just fail to safe your ass once things get more complex and there is an edge case you didn't think of.

So in the end users were in state "not verified" after migration. Seems email is a secondary property.

But you don't want "implicitly verified", when there exist any code to which the difference between "verified" and "not verified" matters in some way.

In your own example, if you happen to use e-mail verification as proxy for user verification, then the very code that creates VerifiedUser instances would want to have VerifiedEmail as input!

You don't need VerifiedUser in the moment user click verification link. It's a state changing action. You get verified user from db and work with it later.

Fine. It still has merit that the code pulling verified user data from verified users table returns VerifiedUser objects, so the code processing them is clear about its assumptions, and won't accidentally get reused or refactored in UnverifiedUser flows.

Abstracting the email example, if you have a finite state machine that verifies some fact, you can model each state as a type. Their example goes Email -> ValidEmail -> VerifiedEmail. You could equally go String -> Email -> VerifiedEmail, where String is what the user submits as their email address. If you are verifying emails in the context of verifying users you could go Map<String, String> -> User -> VerifiedUser, where Map<String, String> is what you receive from an HTML form submission.

Getting hung up on the details of which type you use is missing the overall point.

Good point, the names really don't matter and the first type could as well have been a string. I just wanted to illustrate the concept of using the type system to add important information and guarantees to certain variables.

Your state machine analogy is a good one, because the conversion between different types is a bit like the transitions between states: you have to make explicit how exactly they are meant to happen (or whether they are actually possible).

This is a good thing, having less degrees of freedom may seem like it makes it harder to code, but in fact it allows you to reason better about what the system is doing at any given point in your code.

State machine is always a trivial part. Email verification is stupidly simple task. But User propagation through the system and available actions based on verified email are not.

Wouldn't UncheckedEmail be just string?

All of these are just strings. But if you have a function that requires that string to be of type VerifiedMail, your type checker or compiler will either warn you about the misuse or not even let you compile the code (like in Rust).

This is good, because you as the programmer can rely on the fact that wherever you see a string of type VerifiedMail, that it is indeed a string containing a verified email adress, you don't even need to check, because you know the conversion between the different types had be done explicitly.

You can of course extend the whole thing and have a OnboardUser with a ValidatedMail and only convert the OnboardUser into an actual User once there is a VerifiedMail etc.

You get the idea. Whenever you find yourself wondering if a variable is actually holding the expected information, it is a good idea to leverage the type system to replace wondering with knowing.

But you are right: unchecked email could as well just be a string.

In context, there's presumably a form in a web page containing a claimed email address and other personal data that "evolves" a bunch of strings into valid Email, Address, Name etc. objects or into a collection of form fields with a list of error messages attached, depending on how validation goes.

In representation yes. But if you have UncheckedEmail, you can't accidentally validate a Username, or use an UncheckedEmail as a Password

I've been going through the comments on previous posts. I think one of the biggest issues with the article is the title. It seems to act as an anchor for a lot of people to the point where they'll argue against things that aren't in the article, just implied by the title (without context). So there'll be people arguing that she doesn't want to validate at all and just wants to parse, when really the article is about where you validate your data (and what you do with the result). It is not about getting rid of all validation.

This is true of all famous essays. People remove the nuance included in the body and over-apply the title without really understanding it.

For example "Goto considered harmful". I remember working with a very good programmer who'd used a "goto", and a much less senior one[1] rejected their PR by linking to the article.

[1] I'm ashamed to say it was me, a long time ago.

That's precisely why I love to only comment things with questions, i.e. "isn't go-to considered harmful? Is this really the best way?".

It gives the person making the change the opportunity to give context for their decision - and either change or spell out their reasoning for it, potentially giving me an opportunity to learn from them.

Another good benefit is that it doesn't "attack" the PR author in the "non-violent communication" way

Doesn't that seem a bit passive aggressive? See what I did there?

Surely it depends on both your phrasing and how you usually interact with your colleagues, no?

You can totally do this with non-passive-aggressive phrasings ("Is this possible using structured programming instead of GOTO? What went wrong when you tried, how messy does it get?")

Reply on news.ycombinator.com