Hacker News Reader

Comment by mikece | Hacker News Reader

I get that the author is asserting that untyped Pythong is superior for infrastructure code but I don't understand why the author thinks this.

View on HN

Yup! For me it's precisely the opposite of that, the deeper in the systems code, the more types I write. Same with JS, the bulk of libraries IMO should be written in TypeScript.

I’m not sure I understand the “infrastracture” label, but certainly for the cases the author listed as “infra” typing can be cumbersome. Famously just json deserialization can be rather painful in statically typed languages, especially ones without complex compile time features.

Serialization/deserialization can be really gross in something like Java, but TypeScript gives you some really good tools (zod, typebox, etc.) to validate and type-guard your inputs that I'm sure could be implemented in Python--if they're not already.

Pydantic will do that for Python.

Yeah, I assumed as much. I've only looked at Pydantic in the context of FastAPI, as Python's not really my thing, but it looks really well-thought-out.

If serialisation is painful in Java, you're doing it wrong.

Libraries like Jackson make it entirely painless.

I would assume JSON parsing is a solved problem in any typed language.

I've written a lot of code using Jackson and I strongly disagree that it's a solved problem. If you're Java on both ends, it's probably fine, but then your serde style is an implementation detail. And that's great, but if you're not, you are in a bear pit. The lingua franca for JSON specification is JSON Schema, and there is significant impedance mismatch between the standard JSON tools folks use in Java and with JSON Schema as used by most other folks. (Jackson has a nice JSON Schema module for exporting it, but importing/codegen is pretty gross.)

I find Java pretty nice for handling JSON serialization/deserialization these days. Just define some records for the expected structure, and Jackson's object mapper handles the rest.

It's the "defining some records" part that's slow and gritty. JSON is usually an interchange format, and Java class construction is either slow, tedious, and manual, or involves massaging a code generator (which, at least the last time I checked a few months ago, weren't very good in Java).

I don’t see how declaring a data model class is anymore difficult in Java than another typed language.

  record Thing(String name, int size, LocalDate when) {}

The only way I could see this being easier is to forgo type safety completely.

You're assuming you only have to define it in one language. JSON is an interchange format. The canonical way to express it is JSON Schema, and the tools are unpleasant in Java. So is dealing with structural typing of polymorphic responses. In an ideal world we wouldn't have those, but we live in this one.

These things are just better when you aren't in a nominatively typed language with some pretty crusty assumptions.

Hum... Almost all of the problems of serialization/deserialization in statically typed languages turn into testing problems in dynamically typed languages.

That's not a gain in any way.

In this case it’s more about shifting where the static typing occurs, since the author is still arguing for a typed business layer.

Since I/O is usually untyped, you need to cram it into a typed schema somewhere. In strictly typed languages, that must occur upon ingestion. The author is saying you can delay it by having a layer on top of the I/O transform the data in untyped fashion before being typed later on.

What's the benefit of delaying typing? By typing it at ingestion, you guarantee that invalid (with respect to structure) untrusted data will not break your code down the line. Doing otherwise means that you're manipulating untrusted data with a strong risk of forgetting that it is untrusted.

I have seen large applications (think hundreds of millions of daily users) break because of such oopses.

> Famously just json deserialization can be rather painful in statically typed languages, especially ones without complex compile time features.

That's funny, because last time I had to write json-related code in Python, I really missed Rust's serde, which makes deserialization + validation much easier to write in Rust than in Python.

YMMV, of course.

Pydantic is that for Python

Thanks, you remind me why I wanted to try Pydantic :)

Not the writer, but based on what the author wrote, I think it's based on the author's experience with Java. It can be a hideous nightmare of java.util.Date versus java.sql.Date versus java.time.LocalDate (just to pick one very well known fiasco in the language- Java defenders please don't come after me) and all of them have to exist to avoid breaking things but you have to have different ways to handle each of them, so dealing at the boundaries between business logic and "infrastructure" is always a mess of converting types- oh this library is old enough that it needs a utilDate, so I need to convert it back down, then convert it back up at every boundary.

By contrast, duck-typing means that the intricacies of which exact type you are deserializing into is a lot less important in python.

Duck Typing always struck me as utterly absurd wishful thinking. Two different types built for the same task will almost certainly not happen to be API compatible.

Am I missing something?

No, in my experience you are trading off ease of coding for order of magnitude more production bugs.

So you do not write tests?

I absolutely do, and one of those effective tests is a simple type annotation.

I have bounced back and forth between typed and untyped languages in my two decades of employment history, and I prefer typed languages myself, even when I've been writing libraries nested deep in what I think the author would describe as the "infrastructure layer," but this is what I believe the original author of the linked piece is arguing.

Yes, interfaces aren't just bags of data and methods. They have semantic meaning, too. An Employee, a Gun, and a ClayPot might all have a method named `fire()`, but they all mean different things. With structural typing, we can load our Employees into a Kiln and give HR a Gun to downsize the company, and nothing will stop us.

There are cases where it can be useful. One thing that comes to mind is numerical code, where your algorithms are generic over things that have *,/,+,-,==,!=, conversion from 0 or 1 [or some subset thereof] do the "right thing". Trying to define proper interface types for what constitutes the "right thing" is somewhat tedious without much relative gain.

Its worked for us when dealing with local operations vs. cloud bucket operations. Protocols also help when you really don’t care to litter your code with some vendor specific object but instead just declare what functionality should be expected from the object passed in. It helps describe what you’re doing with it anyway, it’s why typing with Iterable can be helpful if you don’t care if it’s a list, set, or tuple.

In sequences, even just too calls like "more?" and "next" will let you entertain all kinds of strict and lazy possibilities.

> By contrast, duck-typing means that the intricacies of which exact type you are deserializing into is a lot less important in python.

How do you explain the popularity of type hinting in Python then? Type hinting arose _because_ duck typing is horrible for large systems.

Well, the article is the author's attempt to answer that question. I'm not sure that it's convincing to me, but then again I have a preference for strongly-typed languages so I'm probably not the correct audience. I'm just trying to help someone else understand the author's argument, as I understand it.

Not the OP but I explain it as former Java/C# programmers coming to Python for one reason or another.

I’ve come to Python 20 years ago after a short stint with PHP and I don’t really care if the incoming stuff I have to handle is a list or a tuple (let’s say) as long as it quacks and walks like the proverbial duck.

More generally, you could say Python the original language is the victim of its own success (the same goes for JavaScript in relation to TypeSctipt).

Might not be the best example since Python made quite a mess of dates and date times (in fairness, like most languages), and duck typing didn't help at all.

No. The people make the mess. For persistence, only UTC without timezone makes sense. For UI, with few exceptions, only local time without time zone in ISO 8601 makes sense.

The only not so easy decision is: is it better to store as Unix epoch or as ISO string. That is a tradeoff between compactness and easy searching of date/time ranges.

I think from the perspective that "infrastructure" code being that which rummages around in objects, where the only distinction that matters being dict vs primitive.

I think it would have been better to say "internal library code for foundational projects." That code that takes advantage of all the freedom in Python to mess with types, generate classes on the fly, use type annotations at runtime, add and remove attributes dynamically. Code that is basically a notch away from `eval` (or sometimes literately for dataclasses). There's no way you'll ever be able to fully type this stuff outside of Any Any Any Any. But to be useful, the boundary between your library and it's user should ideally be fully typed so the type checker can help your users use the library correctly.

It's funny that you typoed Python as Pythong. I do that a lot - in my case I know it's faulty muscle memory, but I don't know where it comes from.

And to my chagrin I didn't realize it until AFTER the time window for editing a comment passed. Oh well.

Reply on news.ycombinator.com