Comment by taeric | Hacker News Reader

Comment by taeric | Hacker News Reader

Strong static typing, where most of your data is going across the wire as JSON, is a battle that is largely fought in incoherent ways.

Yes, you should try and use all tools at your disposal. But you should also find that most "data" is far more squishy than you think it is. People don't fall back to phone numbers being strings because they are lazy, they do it because too many of them made the mistake of thinking they could make them a stronger type at some point. Same for names. Or addresses. Or postal codes. All of these things need to be taken from a user, and the only real way we have to do that is by parsing text. And if you make the mistake of making your system so it doesn't store the preparsed text, you are almost certainly going to regret it at some point.

Now, is it best to have a layer that keeps the original user entered text and offers it as a typed set of data to the user in the backend? I certainly think so, but there will be ROI considerations as to how much that matters for your little part.

More, if you are doing evaluations of any heavy sort, you probably want a translation layer to a SAT or other numerical model for calculation. And in that world, the numbers are the abstraction. Trying to do it another way will almost certainly lead to pain. And again, you will do well to have translations from problem to formulation, and from solution space to domain. All of these can largely be helped with types, but far too often the "types" that are focused on are not these.

Author here. One thing that we do at Svix that I alluded to in one paragraph but I should probably have elaborated on further: thanks to libraries like Serde and Pydantic, we actually follow deserialization is validation (is that a term?), which means that we validate all of the JSON data before even creating the structures in our code.

I guess that's similar to the redis example I gave, but it essentially means that even though we get sent JSON over the wire, we validate it fully and when it gets to our code we know it's a well formatted type. So our code can assume an email type is a valid email, an ID type is a valid ID, etc.

Right, but you should also take pains to keep the original. All too often the deserialization/validation can also alter the data. In many places, this is fine, I think. In many others, it leads to a ton of confusion.

Edit: I wrote "but you should", I really should have worded that as "and you should". I am wanting to add to your point, not contradict it.

Oh yeah, I agree, and I learned it the hard way. In many cases you want to keep the customer format rather than the canonical format. This is why I stopped using Postgres's JSONB for customer data, and exclusively use String/JSON nowadays.

It’s absolutely a term, and the way to go

> deserialization is validation (is that a term?)

“Parse, don’t validate » is a common Googleable idiom

What I like that you're doing and what I used to do with Python annotations is that you strongly define types for your higher level business logic - which may be encode quite complex formats.

In many ways, your point is correct and kind of difficult to argue against (i.e. I agree!).

What catches alot of people is they say how important types are but then just use 3 or 4 primitive types and never build custom types on top of them so they don't get much value out of those types aside from preventing the absolute worst runtime errors. The real value of types is having lots of them and evolving them over time as your systems and capabilities mature (US street address, US state, latitude, ... the list goes on for any piece of software).

Huh! That's given me a lot of insight into why I've had to argue about typing in the past. Thanks for encapsulating that for me.

I've had teammates "agree" and use types only to use the most basic possible types and then confuse themselves later (like, days later). But to me, because I like static typing, it comes very naturally to define types based on business logic, which makes things very easy to work with. And they're just not thinking in that way. Now I wonder how I can bring them around.

This seems to be a common practice in Go and I've seen it cause requests to be rejected because one unimportant field is wrong. Were we to do this for truly important requests, like lead traffic from third parties, we would lose a lot of business.

If the field is unimportant you'd represent it as such. `Option` or `Maybe` or whatever the equivalent wrapper is in your language of choice.

> we actually follow deserialization is validation (is that a term?)

Perhaps you are thinking of this great blog post on the subject https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...

This is correct - if it's going to error or panic, it should do it at the boundary, not at a random place in the program. It's also much faster to debug where the issue is this way.

> deserialization is validation

I saw similar, catchy phrase: “parse, don’t validate”.

https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...

In my opinion this is the single biggest benefit of GraphQL, forced validation of data server-side. Sure you can do that with swagger in REST apis as well but nobody ever bothers to do it strictly and correctly.

Technically you can have more complex and custom validation (email is a valid email address) with annotations but the client-side will not have this information.

You should still use name and address as types not strings, even though when you look deep you discover they are both strings. It is [almost?] always an error to mix up name and address fields and the type system can enforce this.

This is why I said it will be an ROI of where you are. In your javascript just pulling the data out of form fields to send to the backend? Probably not worth it. Somewhere between taking it from the user and processing it? Probably worth it. And sending it to another system? You almost certainly need a translation between your types and theirs. But making a layer that converts between every one of your types to every possible external system? Probably not worth it.

> Strong static typing, where most of your data is going across the wire as JSON, is a battle that is largely fought in incoherent ways.

Is it really? Map<String,String> is a totally reasonable type to use in certain cases.

Not to most of the debates you will see online. For one, they will be quick to throw that json is closer to Map<String, JsonAtom>, where JsonAtom can be JsonNumber, JsonArray, String, or Map<String, JsonAtom>. Then the clojure/lisp crowd will start to ask why can't they use a myriad of long standing tools that work on lists with this data. (And for more fun, the lisp crowd will point out that json is closer to an A-LIST, not a Map, since they allow duplicate keys... At this point, most of the crowd will rightfully tell us to shut it. :D)

Though, to be clear, I think I largely agree with you. I don't necessarily think typing is a curse. I just don't think most of the typing debates we see online are really telling us much. I'm reminded of a while back when everyone was writing what a TODO app would look like with their favorite abstractions. When, realistically, all applications get dirty after you throw enough users and existing data at them.

Yeah we are on the same page, lol. Practical usage isn't as sexy as diehard ideological arguments.

That said, what JSON corresponds to doesn't matter -- you're parsing it into a data structure that makes sense for your use case. You do not need to represent arbitrary JSON, and should not -- only valid values. So maybe that means something like

struct Person {

String name;

int age;

enum Country birthplace;

Map<String,String> other attributes;

}

I am baffled by the purists who act like this is somehow not allowed.

Or if you genuinely are just building a generic JSON parsing library, that's easy enough as you described. The pseudo BNF definition of JSON is simply mapped directly to sum and product types. I have worked on (very lightly) a popular java Json library (Jackson) and it was no problem...

Another problem is lately there are novice people whose only exposure to type systems is, of all things, typescript (or pep484 python) which is a horrible example of a practical type system because (much like pep484) it's been bolted on poorly onto an existing typeless language.

Reply on news.ycombinator.com