Hacker News Reader

Comment by tasn | Hacker News Reader

Author here. One thing that we do at Svix that I alluded to in one paragraph but I should probably have elaborated on further: thanks to libraries like Serde and Pydantic, we actually follow deserialization is validation (is that a term?), which means that we validate all of the JSON data before even creating the structures in our code.

I guess that's similar to the redis example I gave, but it essentially means that even though we get sent JSON over the wire, we validate it fully and when it gets to our code we know it's a well formatted type. So our code can assume an email type is a valid email, an ID type is a valid ID, etc.

View on HN

What I like that you're doing and what I used to do with Python annotations is that you strongly define types for your higher level business logic - which may be encode quite complex formats.

In many ways, your point is correct and kind of difficult to argue against (i.e. I agree!).

What catches alot of people is they say how important types are but then just use 3 or 4 primitive types and never build custom types on top of them so they don't get much value out of those types aside from preventing the absolute worst runtime errors. The real value of types is having lots of them and evolving them over time as your systems and capabilities mature (US street address, US state, latitude, ... the list goes on for any piece of software).

Huh! That's given me a lot of insight into why I've had to argue about typing in the past. Thanks for encapsulating that for me.

I've had teammates "agree" and use types only to use the most basic possible types and then confuse themselves later (like, days later). But to me, because I like static typing, it comes very naturally to define types based on business logic, which makes things very easy to work with. And they're just not thinking in that way. Now I wonder how I can bring them around.

> deserialization is validation (is that a term?)

“Parse, don’t validate » is a common Googleable idiom

> we actually follow deserialization is validation (is that a term?)

Perhaps you are thinking of this great blog post on the subject https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...

Right, but you should also take pains to keep the original. All too often the deserialization/validation can also alter the data. In many places, this is fine, I think. In many others, it leads to a ton of confusion.

Edit: I wrote "but you should", I really should have worded that as "and you should". I am wanting to add to your point, not contradict it.

Oh yeah, I agree, and I learned it the hard way. In many cases you want to keep the customer format rather than the canonical format. This is why I stopped using Postgres's JSONB for customer data, and exclusively use String/JSON nowadays.

In my opinion this is the single biggest benefit of GraphQL, forced validation of data server-side. Sure you can do that with swagger in REST apis as well but nobody ever bothers to do it strictly and correctly.

Technically you can have more complex and custom validation (email is a valid email address) with annotations but the client-side will not have this information.

This seems to be a common practice in Go and I've seen it cause requests to be rejected because one unimportant field is wrong. Were we to do this for truly important requests, like lead traffic from third parties, we would lose a lot of business.

If the field is unimportant you'd represent it as such. `Option` or `Maybe` or whatever the equivalent wrapper is in your language of choice.

This is correct - if it's going to error or panic, it should do it at the boundary, not at a random place in the program. It's also much faster to debug where the issue is this way.

> deserialization is validation

I saw similar, catchy phrase: “parse, don’t validate”.

https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...

It’s absolutely a term, and the way to go

Reply on news.ycombinator.com