Hacker News Reader

Comment by reissbaker | Hacker News Reader

I work at Instagram, which is a O(millions) LOC Python monorepo with massive throughput and a large engineering team. It's actually quite nice — but our code is heavily, heavily typed. It would be miserable without the type system. Some of the older parts of the codebase are more loosely typed (although they're shrinking reasonably quickly), and those sections are indeed a huge pain to ramp up on.

Part of the success of IG's large Python codebase is FB's investment into developer tooling; for example, FB wrote (and open-sourced, FWIW) our own PEP-484 compliant type checker, Pyre [1], because mypy was too slow for a codebase of our size.

1: https://github.com/facebook/pyre-check

View on HN

That's my major complaint about Python...

For it's age and popularity - the tooling is abysmal.

I have to rewrite too many things, that I expected to just be there for the age of the project.

And some things were only fixed now! dict + dict only started to work with 3.9!

dict1 + dict2 is syntactic sugar, you had been able to do `dict1.update(dict2)` since forever.

No, it's completely different: dict1+dict2 creates a new object and leaves inputs unchanged, dict1.update(dict2) modifies dict1.

you've been able to do splat for forever! https://dpaste.org/yU7s

Yeah, you could use my hairline to tell when I’m working with Python... I still haven’t figured out a good typed framework for it yet.

I also struggle with weak(er) typing in Python, compared to strong(er) typing in C++, Java, or C# -- or even VBA(!).

Frustrated like you, I wrote my own open source type checking library for Python. You might be interested to read about it here: https://github.com/kevinarpe/kevinarpe-rambutan3/blob/master...

I have used that library on multiple projects for my job. It makes the code run about 50% slower, on average, because all the type checking is done at run-time. I am OK with the slow down because I don't use Python when I need speed. My "developer speed" was greatly improved with stricter types.

Finally, this isn't the first time I wrote a type checking library/framework. I did the same for Perl more than 10yrs ago. Unfortunately, that code is proprietary, so not open source. :( The abstract ideas were very similar. Our team was so frustrated with legacy Perl code, so I wrote a type checking library, and we slowly applied it to the code base. About two years later, it was much less painful!

Try using Pyre! It's open-source. I use it daily at IG.

Have you guys published any whitepaper on this subject? These last few years working on moderately large python codebases with dynamic typing have been less than idilic.

Here's a small collection https://pyre-check.org/docs/pysa-additional-resources

Reply on news.ycombinator.com