Hacker News Reader

Comment by haberman | Hacker News Reader

I think the real point the author is trying to make is: use modular, layered designs of composable components, instead of monolithic APIs that only have a single entry point. The single-entry-point model imposes costs and decisions onto the application that are hard to work around.

I think this is a good point. I think that it's hard to get from there to "more classes are always better" though. More classes don't always make a design more flexible. You have to consciously design for flexibility and reusability, and even then it takes a lot of deep thought and hard work to get your interfaces right, such that they are truly reusable in practice.

View on HN

Quoting from the end of the article: "And our solution to monolithic code in Python are classes."

So, I guess the question I ask as someone who doesn't write a ton of python code is: Is that true? And if so, why? Is it not possible to compose a layered API that doesn't rely on inheritance / classes in Python?

One can write Python in a procedural manner, and even in a functional way, so one can use the same techniques as are used in those paradigms (OO is probably more natural for Python though).

You can also easily write component-based architectures in Python, for example using zope.interface/zope.component (ie: the modules only, not the full Zope CMS). For example the Trac bug tracker/wiki works like this, and for its purpose I think it's a perfect fit. It allows re-usable and pluggable components with declared, adaptable interfaces that are not restricted by some hierarchical taxonomy the framework designer came up with.

At the end of the day there will always be different ways to build layered architectures using re-usable components, and classes are just one of them. I'm pretty sure the author of the article didn't intend it like that, but 'write more classes' doesn't sound like great advice for novice/inexperienced programmers.

It is perfectly acceptable to design a library with a simplified, restricted interface that covers the majority of use cases, but not all. e.g. requests vs urllib2 For me, that's a win for common sense. I have never even considered streaming JSON. The lack of streaming support should be a clue that the OP needs to look at whether what he's attempting is even sensible or could be better served by a different transport protocol with built in streaming support.

JSON supports streaming. You not having considered it is not an argument.

Firstly:

http://www.ietf.org/rfc/rfc4627.txt

"An implementation may set limits on the size of texts that it accepts. " - How does that "support" streaming?

My argument is that most people don't need it most of the time (I can't prove this). It's also NOT a protocol designed for streaming. As regards API design, it's a matter of taste but my preference and that of many others who use python, as we can see with the popularity of requests, Flask, is that an API should be designed for simplicity for the common use cases rather than for absolute completeness. I'm not saying don't expose a low-level API full of BufferedReaders, JSONParserCursors, but don't force it on me to parse a tiny JSON document of known length.

The protocol can support streaming very well. How easy it is or if it's possible at all is an implementation issue. With the Go standard library's json implementation, for example, you can decode JSON streams very easily: http://golang.org/pkg/encoding/json/#example_Decoder

> So, I guess the question I ask as someone who doesn't write a ton of python code is: Is that true?

I don't think it is. His objections to the json library API can be addressed without introducing a single class.

He seems to want access to the internal implementation of converting bytes into a single Python object. That is probably just an internal function that can just be exposed. If he wants to override it from the top level, the library could permit that by just adding the function as a default parameter.

He wants it to deal with streaming. Python generators can help with this. The library could be refactored into implementing a json.load_stream function instead, which takes a string generator and returns some kind of object stream generator. The original json.loads function could be replaced with a wrapper around it for backwards compatibility and for API users who don't want the additional complexity.

None of this involves writing classes. A generator could be seen as a type of class instance, and one can implement a generator with classes, but generators have a well known and simple API which eliminates the need to define a new API, as the traditional writing of classes would do.

It seems to me that the writer has little experience of not using classes, so wants to solve every problem he has with a class. The API problems he want to address are genuine, but he's blind to any solution that doesn't use classes.

> It seems to me that the writer has little experience of not using classes, so wants to solve every problem he has with a class.

That's a bold statement.

Where is the trend to forcefully avoid classes here coming from? Classes are a perfectly valid tool in Python to solve problems. Someone sent me a mail this morning proposing a dictionary with callback functions in addition to a function as if that would solve anything.

The question is, what classes do that module systems don't? The answer is generally inheritance, and through it, class polymorphism.

With first class function, you hardly need class polymorphism. Just pass the function as argument already, don't bother with writing a whole new class just to override one method.

The other use for inheritance is plain code reuse. This is bad most of the time because it promotes thick interfaces, and function calls are great at code reuse anyway.

The question is, what classes do that module systems don't?

Modules (at least in Python) are singletons.

Doesn't count. When you define a single data structure in a module, (along with functions to manipulate it) you can instantiate it just like you would a class.

Ugh. That's like saying that high-level programming languages "don't count" because you can do that stuff in assembler.

I'm not arguing Turing completeness here. I'm arguing trivial one to one mapping. Syntax sugar. Compare:

  struct foo {
    int   bar;
    float baz;
  };
  int   get_bar(foo);
  float get_baz(foo);

Which would emulate this:

  class foo {
  public:
    int   get_bar();
    float get_baz();
  private:
    int   bar;
    float baz;
  };

If you're interested, I wrote more about that here: http://loup-vaillant.fr/articles/classes-as-syntactic-sugar

> With first class function, you hardly need class polymorphism. Just pass the function as argument already, don't bother with writing a whole new class just to override one method.

Here, here! After returning to functional programming after a 30 year hiatus, I am beginning to realize just how detrimental the OO orthodoxy has been, and just how impoverished the world has been without first class functions. If only Scheme had caught on in the mainstream when it was invented, the programming world would be a much happier and safer place.

> Here, here!

Where, where? ITYM "Hear! Hear!"

http://en.wikipedia.org/wiki/Hear,_hear

"Hear, hear is an expression used as a short, repeated form of hear him, hear him. It represents a listener's agreement with the point being made by a speaker."

Yes, I know, but it's not nice to mess with the lysdexics.

:) I would have let it go, but a lot of young people read HN and I didn't want them to think the expression really was "Here! Here!" Once these things get started ...

> Where is the trend to forcefully avoid classes here coming from?

See the "Stop Writing Classes" video.

Classes hold state. More state adds more complexity, and typical OO design add lots of layers of indirection which also add complexity. Most of the time, you don't need the flexibility that the extra indirection gives you. So you get complexity for little benefit.

If you don't need to hold state, then a collection of functions will usually suffice. In Python, you can use modules for this.

By all means use classes if they are the best way to solve a problem. But for those who know nothing else, how do they even know this?

My problem with this article is that it says "I have a problem; I can solve it with classes; therefore everyone should use classes more". It fails to even consider anything else.

I'd be so bold as to make a categorical statement: If your classes are invariably stateful, or if they can reasonably be described with the phrase "lots of layers", then you're doing it wrong. You're not doing object-oriented programming so much as the mongrel "class-oriented programming" style that rose to prominence among C++ programmers in the 1990s.

Good well-modularized object-oriented code should not be an alternative to a collection of functions. It should be a technique for making your collections of functions more effective. It allows you to write a smaller, more concise collection of functions that can operate in a consistent manner on a whole plethora of datatypes. It accomplishes this because you can write your functions against a common set of mixins that your datatypes inherit rather than having to worry about specializing them for every datatype.

> Classes hold state. More state adds more complexity, and typical OO design add lots of layers of indirection which also add complexity. Most of the time, you don't need the flexibility that the extra indirection gives you. So you get complexity for little benefit.

Functions hold state as well, they just encapsulate it. You can still break up your algorithm into a class with multiple template methods and then encapsulate it in a function if you're afraid of leaking state elsewhere.

What I see instead is that people can't get rid of state and put it as global variables into Python modules. Case in point: pickle. It uses sys.modules and pickle.dispatch_table to store some of its state.

> If you don't need to hold state, then a collection of functions will usually suffice. In Python, you can use modules for this.

Classes have another advantage: virtual method calls. If one function in a module calls into another function in the module I can only do two things: a) copy/paste the library and change one of the functions or b) a monkeypatch which modifies a shared resource someone else might want to use with the old semantics.

A class I can subclass and change the behavior for each method separately.

> What I see instead is that people can't get rid of state and put it as global variables into Python modules.

What I see is people writing classes that hold state in instances for far longer than is needed. Other code written around these classes then need to take this into account. This tends to the same place where global variables are. A tangled mess.

> If one function in a module calls into another function in the module I can only do two things...

3) Adjust the function to include the behaviour that you need. In Python, default parameters often mean that you can do this without breaking backwards compatibility. If a function is deficient in some way, why not fix the function rather than working around it?

> You can still break up your algorithm into a class with multiple template methods

Template methods are an abomination. They are only widely used because many OO languages don't have first-class functions. A template method means that the method is being parameterized by one or more functions. Parameters should be expressed as parameters! You wouldn't design a Circle class so that you need to subclass it to specify the radius of the circle. The radius is a parameter. Likewise for the callback functions used by a template method. Template methods are intrinsically difficult to understand and spaghetti-like, unless well-documented, because it is not immediately clear what the parameters to it are. And when overriding a callback method, it's not immediately clear what template method the overridden method is paramaterizing. Or that it is even parameterizing anything. (Caveat: Sometimes template methods are warranted, but they are greatly overused.)

> Classes have another advantage: virtual method calls.

You don't need virtual method calls in order to parameterize functionality. See previous paragraph. Furthermore, it is not safe to override a method without knowing if it has been designed for overriding, and what the class expects of the overriding method. See Effective Java, "Item 17: Design and document for inheritance or else prohibit it". Also see "Item 16: Favor composition over inheritance", and a similar discussion in the Gang of Four book.

> If one function in a module calls into another function in the module I can only do two things: a) copy/paste the library and change one of the functions or b) a monkeypatch which modifies a shared resource someone else might want to use with the old semantics.

If a method is not designed to be overridden, you shouldn't override it anyway. (See previous paragraph.) When using a functional approach (in a language with first-class functions), you can usually design for flexibility just as easily or more easily than when using an inheritance-based OO approach for flexibility.

> Furthermore, it is not safe to override a method without knowing if it has been designed for overriding, and what the class expects of the overriding method.

This is true, but likewise, its not safe to parameterize functionality by passing callbacks unless you know what the function you are passing the callback to expects of callback functions (in terms of arguments, return values, and side effects.)

Both of these, really, are the same principle: its not safe to call code without knowing what the code expects of you when calling it.

> This is true, but likewise, its not safe to parameterize functionality by passing callbacks unless you know what the function you are passing the callback to expects of callback functions

Of course; this goes without saying. But when programming in an OO style, programmers often override methods thinking that they are only changing the behavior of that one method, when in fact they could be inadvertently changing the behavior of other methods in the class.

When passing in functions as arguments, on the other hand, this makes it much more explicit what is supported and what isn't.

> Of course; this goes without saying. But when programming in an OO style, programmers often override methods thinking that they are only changing the behavior of that one method

This is not a problem with using classes or overriding methods, its a problem either with either failure to document behavior on the part of the library author or failure to read documentatio on the part of the library consumer.

Neither of those is any less a danger in the case here the design of the library involves parameterizing functionality by passing functions around as parameters.

> This is not a problem with using classes or overriding methods, its a problem either with either failure to document behavior on the part of the library author or failure to read documentation on the part of the library consumer.

Missing or poor documentation is a sad reality of programming in the real world. And is the normal state when working on a active project with other developers. This would be mitigated if programmers were very careful about declaring methods to be final if it's not perfectly safe to override them, but most are not that careful, or even fully aware of the issues. And in many programming languages there is no way to declare a method to be final.

> Neither of those is any less a danger in the case here the design of the library involves parameterizing functionality by passing functions around as parameters.

When programming in a functional style, functionality that is not intended to be modified by the client will not provide a parameter for doing so, so this significant source of confusion is eliminated.

> This would be mitigated if programmers were very careful about declaring methods to be final if it's not perfectly safe to override them

Its always perfectly safe to override a method if you maintain its required features. Its rarely, if ever, perfectly safe to do so if you don't. The issue is correctly documenting the required features.

> The issue is correctly documenting the required features.

I'm not sure where the disconnect here is. First of all, "correct documentation" might exist in an ideal world, but it rarely exists in the real world. Given this imperfect world, it seems prudent to use technologies and idioms that mitigate the consequences of this imperfection to the best that we are able. Of course no solution is going to be able to eliminate such consequences completely, but that's no reason not to use solutions that offer some benefit.

Secondly, the OP claimed that OO design is generally superior to functional design because it's easier to support overriding of behavior and it's easier to override behavior. Neither of these claims is true. In both cases, attention to detail is required.

Furthermore, in the OO case programmers using a class often fall into the attractive nuisance pit of thinking that just because a method is there, they can and should override it, and programmers implementing a class often neglect to even consider what might happen if a subclass overrides some methods. It's not just a matter of documentation; it's a matter of not even considering the consequences of providing this flexibility by default.

Additionally, when you do parameterize behavior using OO idioms such as template methods, the fact that parameterization is occurring has been obscured, while doing so using the typical functional programming idioms represents paramaterization as, well paramaterization. Who could argue that representing something as what it is is not a good thing?

> Given this imperfect world, it seems prudent to use technologies and idioms that mitigate the consequences of this imperfection to the best that we are able.

Assuming that methods which actually mitigate those consequences exist, and all other things being equal, this is true. In the use cases for which class-based object-oriented design is well-suited, all other things are not equal between class-based object-oriented design to support providing base functionality with overriding in subclasses and using functions that take functions as optional parameters to provide base functionality with per-call overrides, even before considering whether, when used for that purpose, the functions-as-parameters approach actually mitigates anything.

Particularly, they are not equal in that the class-based approach avoids repetition and makes it clear the unit the behavior is attached to, while the functional approach does not. The functional approach is obviously cleaner and clearer for per-function-call parameterization, while the OO based approach is cleaner and clearer (unsurprisingly) for per-object or per-class parameterization.

> Secondly, the OP claimed that OO design is generally superior to functional design because it's easier to support overriding of behavior and it's easier to override behavior.

I haven't been defending OPs claim, I've been criticizing your response which argued that functions-as-parameters was not merely as good but actually categorically superior to OO design for this purpose.

> Neither of these claims is true. In both cases, attention to detail is required.

"Attention to detail is required in both cases" does not, if taken as true (which I have no problem with), refute the claim that these things are generally easier to support in OO.

> Additionally, when you do parameterize behavior using OO idioms such as template methods, the fact that parameterization is occurring has been obscured

No, its not. Inheritance is categorical rather than per-call parameterization, and so using it presents what is being done as exactly that. Using functional idioms for categorical parameterization either involves recreating class-oriented structures or conceals (and makes less DRY) the categorical nature of the parameterization behind per-call overrides.

> The functional approach is obviously cleaner and clearer for per-function-call parameterization, while the OO based approach is cleaner and clearer (unsurprisingly) for per-object or per-class parameterization.

Having programmed heavily in both functional and OO styles, I personally find the opposite to be true. I find the functional approach to be cleaner and clearer in general.

I particularly find template methods to be egregious because when you override a callback method, it's not immediately clear that what is being overridden is even a callback. Additionally, in programming languages that don't require an "override" declaration to override a method, it's not immediately clear that a method is being overridden, rather than just a new method being defined. And with multiple inheritance, this is even worse, because you might have to look in a zillion different other places to even determine this.

> Using functional idioms for categorical parameterization either involves recreating class-oriented structures or conceals (and makes less DRY) the categorical nature of the parameterization behind per-call overrides.

Recreating class-oriented structures? There's no work to do this, and there's nothing non-DRY about it. E.g., see the book JavaScript the good parts. It shows you how to define objects using either JavaScript's OO-based mechanism or using a Scheme-like functional approach. The functional approach is elegant and popular.

...indeed, the one thing I "hate" about Python is that somehow it pushes your mind away from functional solutions. You end up writing either OO or procedural code, although in a language with first class functions and immutable strings you's expect to slip more to the functional side.

Anyone else had the feeling that although Python allows functional style, it somehow pushes your mind away from it?

Personally I find myself avoiding writing in a functional style in Python because of it's lack of good anonymous function literals and poor handling of closures in nested functions.

Edit: when I say "poor handling of closures in nested functions" I'm referring to having to use nonlocal in order to modify variables in the parent function's scope.

yeah, I think lack of real anonymous functions is a small annoyance that can get quite "big" from the yucky feeling you get when defining functions just to pass them on. But... what do you mean by "poor handling of closures in nested functions"?

Edit: ok, I get what you mean now :) ...though I consider this "poor handling" a feature of Python - mutating outer scope variables is a side-effect and what it encourages is what I like to call "callback hell functions with data Javascript-style nightmare" :) (true, some people know to keep their sanity and write readable code even in this style, and if you enjoy it you're probably one of them few, but most don't...). By "functional style", I and some people refer to "functional as in mathematical functions", so only using side-effects when actually needed (yeah, some "extremist" languages like Haskell try to never do them, but...) and you get this by having immutable values and passing around anonymous functions (that don't modify their outer scope).

...and this is another part of the paradox: this "feature" of Python should encourage what I call functional style, but why oh why it doesn't? ...maybe Rich Hickey is on to something with Clojure and it's actually more about the data-structures than the language syntax or semantics?

Well it does not really help that Python lacks tail call elimination (last I checked), which is fairly important to (practical) functional programming.

Meh, you'd use fold, map or filter 99% of the time in a functional language like scala or haskell. TCO is great, but you can still write a lot of idiomatic functional stuff even without it.

Right! TCO is not essential to practical functional programming. TCO is essential to practical use of the subset of functional programming that insists on recursion instead of iteration, including fold/map/filter.

TCO is an under-the-hood behaviour that the programmer would count on for practical performance. There's no reason that under-the-hood magic can't be in the implementation of fold/map/filter as primitive control ops.

It's just an accident of history that the [mp]atriarch functional programming language is Lisp, a language that was created in the name of proving a point about recursion. Recursion as the solution-to-all-problems is not essential to functional thinking. The essential ideas are first-class functions, higher-order functions, and referential transparency.

I think this mistake is comparable to thinking that classes are essential to object-orientation. I think many people are confused by university professors who love Scheme (for reasonable reasons), and who like to pretend that Lisps are the only alternatives to Fortran (arguably true in the '60s).

You can do pretty much anything with default arguments, rather than overridable class methods. First class functions make this very flexible.

Sometimes this is a little inconvenient. I think the sign you really need a class is when you are sending in something called "object_type", and a dict of "object_type -> function".

Reply on news.ycombinator.com