Python hasattr() Considered Harmful | Hacker News Reader

Python hasattr() Considered Harmful

hynek.me

45 pointsblaze3354 comments

The opening line:

>Don’t use Python’s hasattr() unless you want its quirks or are writing Python 3-only code.

At this point, you should be writing Python 3-only code. Maybe I should write a post, "Python 2 considered harmful".

It's a stupid function either way. As it invokes a potentially expensive descriptor just to throw away the result. Silent getattr and check is always more appropriate.

Wrecking any hope of retrocompatibility considered harmful (who could have known?)

You should so it can give us a reason to discuss Python and how it's probably not going anywhere with its crazy versioning and how 2020 will probably never happen and people will still be using Python 2 and Guido will have to fly from the sky and hit us every time we use Python 2 in order for us to upgrade.

I love those comment threads.

>At this point, you should be writing Python 3-only code.

At this point it should be obvious that Python 2 will never die. It's Good Enough (TM). Most users will never switch to Py3 because cost outweighs the benefits.

There will be a time every civilized OS will default to Python 3. In fact, the next Ubuntu does so. For us, that time is next March.

There is less and less reason to write code in Python 2. Not many libraries require it.

Full time, career Python developer here. Never worked on a project as part of my day job which is built on Python 3. Recently, for the first time ever, there are projects in the company I work for which are built in Python 3. But I'm still yet to work on them.

> For what’s it worth, Python 3 gets it right: > ... > Also, would you expect hasattr() to raise that error?

Not really no. I would expect hasattr() to just answer whether the attribute exists, and getattr() to actually invoke it. If "hasattr" actually invokes the attribute and just checks for a non-empty value returned, is it even needed? It would completely surprise me, coming from a language with static types + reflection, that a query for a types attributes (mere existence) would invoke them.

Thing is, what does it mean for a property to "exist" in a dynamic language like Python? E.g.

  class P(object):
      def __getattr_(self, attrname):
          return "hello"

  >>> p = P()
  >>> print p.foo
  hello

Does "p" have an attribute "foo"?

That is the snag isn't it. It's really difficult to provide any meaningful "reflection" without having proper static types. So in your example you could say "yes, because it gets a value when called" or "no, because it isn't explicitly declared". Since python is dynamic, the "hasattr" method really just means "does_invoking_this_attribute_return_something", which is very much not reflection.

It would maybe be useful with getting only the explicitly defined attributes (say e.g. if you wanted to convert an object to an xsd schema). Your __getattr__ based attribute would not show up in that list obviously.

People usually don't believe these advices until they get bitten. I'm all the same, and got bitten: a hasattr was hiding a deeper error and it is a nightmare to debug, one couldn't even know where to start.

Isn't this simply a bug? Why would hasattr() catch all exceptions instead of just AttributeError? IMHO a better title would be: "Python2 hasattr() has a bug".

Not a bug, just Python 3 choosing different trade-offs. getattr() and hasattr() are useful for replacing annoying boilerplate like:

    try:
        foo = x.y
    except Exception:
        foo = None

If getattr() and hasattr() can raise, then you have to wrap them in try/except blocks. So much for that previously useful shorthand.

There's no good solution here. Either hasattr() can raise (confusing), or it can return false for attributes that exist (also confusing). Maybe if the name was changed (validattr()?), everyone would be OK with the Python 2 behavior.

I prefer the Python2 solution actually. If we disregard for a moment the madness of "hasattr" actually invoking the attribute code -- if it does invoke it and an error occurs, hasn't that proven that the attribute does in fact exist?

Picture a type that has a "t.connect()" function, and a "t.data" attribute. The data attribute will fail if connect() hasn't been called prior to accessing it.

I'd expect hasattr(t, "data") to answer true and never raise an error, regardless of when I do it. The only way to do that (if you have to invoke the code in the attribute -- not sure if that is the case) is to catch and return true.

Validattr would have been a much better name, yes.

Maybe i wasn't clear enough, I think the bug is this line:

    except Exception:

Why would you want to catch all exceptions? ideally you should catch just those that are applicable to your case and ignore others:

    except AttributeError:

I've previously used hasattr() to check for the existence of an attribute qua instance variable to lazy-load and cache that attribute

    def kdtree(self):
      if not hasattr(self, '_kdtree'):
        self._kdtree = scipy.spatial.KDTree(self.xi.T)
      return self._kdtree

like ruby's

    @foo ||= expensive_calculation

is there a better way?

Well, you can do it this way:

    def kdtree(self):
        try:
            return self._kdtree
        except AttributeError:
            pass
        self._kdtree = scipy.spatial.KDTree(self.xi.T)
        return self._kdtree

And, of course, you could do all of that with a decorator.

> ruby's

Perl has //= (http://perldoc.perl.org/perlop.html#Assignment-Operators)

In this particular case you should initialize self._kdtree to None in __init__, and just check if self._kdtree is None: <calculate>

Ah, nice, thanks

Works great unless None is a valid value for the thing, in which case you'll recalculate it every time. If that was the case you should just use hasattr().

Or try and except AttributeError. Exceptions are a good thing.

I went with this decorator in the end, works a treat! https://github.com/jjgreen/mvpoly/blob/master/src/python/mvp...

I am surprised to see a recommendation to use an exception handler for conditionals, especially for Python.

EAFP is in the Python coding glossary:

Easier to ask for forgiveness than permission. This common Python coding style assumes the existence of valid keys or attributes and catches exceptions if the assumption proves false. This clean and fast style is characterized by the presence of many try and except statements.

http://stackoverflow.com/a/11360880/36092

This style using try/except instead of if/else is known as EAFP (Easier to Ask Forgiveness than Permission). It's actually recommended in the Python glossary [0].

[0] https://docs.python.org/3.5/glossary.html#term-eafp

I though this style was encouraged in Python.

> hasattr() also is not faster than getattr() since it goes through the exactly same lookup process and then throws away the result

In practice this only matters if you have a property. If you use hasattr to find a method within an object, it will not invoke the method while checking for existence.

What about just agreeing that property getters should not throw? C# for example has similar code-defined properties, but the agreed and recommended practice is that property getters should never throw.

We could all agree never to have bugs in our code while we are at it as well.

That isn't quite the same thing now is it. There are always cases that may cause a getter to throw (Unpredictable things like OutOfMemory or IO errors), but the point is that they shouldn't be designed to throw. E.g. they shouldn't have "throw new InvalidOperationException("You must connect to the DB before fetching the Data")" exceptions thrown.

Of course not, it's a property, but things like KeyErrors/AttributeErrors are a lot more common.

Why not? Would silently returning the wrong thing be better?

yeah, but we don't double check the calling/return of every builtin and we don't double check our double check, recurse.

There can be bugs in the OS, or network driver. Life happens.

At some point you have to assume something works bug free. A common and reasonable point for that is "I assume language and builtins to actually work as documented".

Everything can be abused. But most things are not, most of the time.

It’s not about abuse, it’s about behavior/consequences most people don’t realize.

Isn't it either way best practise to better be sorry than to be safe? So the second approach is as far as I can see favoured by the community, because it declutters the code and keeps it simple.

I think it's usually better to be safe than sorry. If the community favours the second approach it's probably because they're unaware of the side-effects (hence this article) or because they're writing Python 3 - as they should be.

The principle the above commenter refers to is known as "Easier to Ask Forgiveness than Permission", i.e. you're not meant to check preconditions – you should just try to do whatever thing you want to do, and it should signal an error if it didn't succeed.

The reason for this has little to do with "clean code" and more to do with the fact that when checking preconditions, programmers have a tendency to check proxies rather than the actual thing they want to do. An an example, it is common to see the following:

    if not file_exists(filepath):
        stderr("Invalid config filepath!")

    config = read_file(filepath)
    ...

This code has the glaring problem that even if the file exists, that doesn't mean you have the required permissions to read it. What you really want to do is read the config file, but you check a proxy (does the file exist at all?) as your precondition.

How would you check for the right thing, that you can read the config file? Well, by trying to read the config file, of course!

    try:
        config = read_file(filepath)
    except IOError:  # can make this more fine-grained
        stderr("Invalid config filepath!")

This is longer code (one extra line, gasp!) but it avoids the problem of checking the wrong precondition which is so easy to do by mistake.

You may have heard of this concept in a different situation – e-mail validation. A lot of people go to great lengths to validate e-mail addresses by regex. Even if the address is perfectly conforming and passes the regex with flying colours it might not lead to an inbox. To really verify that the user can receive e-mail at the address they've specified, guess what you have to do? Send an e-mail and ask them to confirm they got it. It's the only way.

Don't forget the part where checking for preconditions opens you up to race conditions e.g.

    if not file_exists(filepath):
        stderr("invalid config filepath!")
    # filepath is removed by an external tool
    config = read_file(filepath) # blows up file does not exist

Is there still a chance of a race condition when you use the "with" operator? It seems like this is one of the major use cases it was designed for

It depends on how you use it! If you use it to acquire the resource, as in

    with open(fn, 'r'):
        ...

then generally the answer is no, since that is the same thing as the equivalent `try…finally`. If someone removes the file after it's being opened it will not be completely wiped until it's closed.

This shows that get __getattr__-code is executed even on lookup. If your __getattr__ or your property-getter have side-effects, then even the lookup will trigger them.

Then even a bare statement like foo.bar can have side-effects in the foo-object.

“Considered Harmful” Essays Considered Harmful

http://meyerweb.com/eric/comment/chech.html

Pointless, pedantic, patronising comments considered harmful…

It is silly IMO - those people using such click-baity titles for everything show a lack of creativity and really ought to try harder. "considered harmful" was already annoying when Dijkstra coined the term (and for very good reason).

Regarding Python3 sample at the end,

What if the method y is mutating the object? Then when I just check for existence of y with hasattr, it will mutate the object. I wouldn't want that.

The reality is that almost any interaction with an object can cause it to mutate itself, since Python allows classes to override just about any operator. For example: "a > 3", "5 in a", "x = a.b", "len(a)", "print a", etc - all of these can cause a to mutate. In the end, you have to trust the code to do the Right Thing, or just not use it.

It's raining "Considered Harmful" stories. Come on. Be a little bit more creative with titles, please!

TBH it was just a quick write up to point ppl to in discussion as said in the intro. I never spent much time to ponder the attractiveness/SEOability etc.

the point is it's not "considered" harmful <-- emphasis on quotes around considered but not on harmful because nobody "considers" it. Some might have a strong use-case for using hasattr(). The same applied to goto (and still does).

There is absolutely nothing hasattr does that you cannot achieve with other means in a more robust way and with minimal impact. It’s not like ditching goto that requires a completely different approach to programming.

also the downvote-fairies on here today should consider reading https://news.ycombinator.com/item?id=9744916

Look, nobody is defending the title. I’ve even apologized for it here and in the article and explained how it happened. But if you insist on keeping to beat the dead horse tells more about you than the title.

Reply on news.ycombinator.com