Hacker News Reader

Comment by vturner | Hacker News Reader

Agreed, modularity definitely helps us minimize the mental load of understanding all the objects at once. But how does it handle this situation:

Write function in module A that expects some complex data structure that comes in. In a static typed language, you put a type on the argument and thus guarantee, to some degree, the data in the object can be safely manipulated.

Over in module B we call that function. Because Python doesn't enforce types, I can pass in some type of data structure that might behave well in many situations, but in some that function is going to fail because I didn't properly construct the data structure.

How do other Python projects avoid this? Do they avoid large data structures of multiple fields?

View on HN

> Write function in module A that expects some complex data structure that comes in. In a static typed language, you put a type on the argument and thus guarantee, to some degree, the data in the object can be safely manipulated.

Let's question some of the assumptions. Do you NEED the entire thing to be passed around? If your functions are small and only do one thing, can't they be fed the portion of the data they need and only manipulate that? If they are getting unnecessary data, soon someone will like the convenience and that piece of data will now become necessary.

I have seen Java projects with humongous "DAO" objects being passed across all apps. Even if Java had Haskell's type system, at some point it breaks down.

So, you do not enforce that on Python itself. Rather, you have to apply due diligence and ensure your data structures are clean, and functions do what they are supposed to do and no more.

> I can pass in some type of data structure that might behave well in many situations, but in some that function is going to fail because I didn't properly construct the data structure.

Then you construct the structure in a single place, and that should have some sanity checks and sane defaults.

Even if you are using a static typed language, there's no guarantee that what's inside the data structure even makes sense, only that the right type of structure is being passed on. That's helpful, but not as much as people would think.

And you write tests. Lots of tests. It is amazing the amount of Python code that gets written without proper testing. And Java, for that matter. Only Python programmers have less excuses, it's easier to replace what you need for testing.

>Even if you are using a static typed language, there's no guarantee that what's inside the data structure even makes sense.

You should always put validation in constructor so you are 100% sure that the object created is valid.

How can you be sure your validation covered all cases?

Say you have a Circle class, you need a center point so you validate that the center is not null and you check that radius is greater then zero. Do you have an example where is impossible to validate all cases? if so how will your unit tests validate/test all those cases?

In your example, you can't anticipate that center points approximately but not quite equal to pi will cause a ... whatever bug in ... whatever downstream use of the class. One can never really know what states are acceptable.

The circle is valid, what you say is a bug in other section that crashes with valid circles,so types could prevent invalid inputs to functions. What you describe is a bug in an algorithm I don't think that is possible for a compiler to prevent bugs in algorithms, division by zero or precision errors

Type systems can do some pretty miraculous things. For example, you could distinguish between a connected and disconnected socket with types so that you don't accidentally send on a disconnected socket.

I guess I'm arguing both sides now. My point is that one needs to root out the weird cases somehow and that it's often hard to have "100%" validation of inputs. One's initial understanding of what is the valid and invalid state space is often incorrect.

Twisted is a massive codebase, and uses zope.interface. There are definitely options for stricter guarantees, but it's optional, and generally you'd rather be treating input as ducks.

Also, even without actual type checking (mypy), IDE support + type hinting (PEP 484 or reST et al) means you can find most issues before the interpreter/compiler is involved. PyCharm is fantastic at this.

>How do other Python projects avoid this?

https://github.com/jonathanslenders/pymux/blob/master/pymux/...

Here's one way. Those asserts will typically pick up on errors during spikes, development and tests and turn a head scratching problem into a straightforward issue.

A couple of times on a disastrously tech debt ridden project I used it to pick up obscure configuration errors in prod, too (shouldn't see it in prod on a relatively well written project tho).

You might consider this too much of an edge case but what if I want to pass something that quacks like an EventLoop to Process init ?

Unless it's a subclass (which would pass), I don't imagine that would be at all likely.

Never been much of a fan of duck typing. It only really makes sense when you have objects that approximate built in types.

Take a random object with a .run() method, for example, and you don't have a fucking clue what it might be doing so there's no point interpreting that as a quack.

Well, there's schema.

https://pypi.python.org/pypi/schema/0.6.6

Everything's a server, er, microservice, passing JSON back and forth!

Microservices are great, because you can enforce static typing within their interfaces. \s

Reply on news.ycombinator.com