Hacker News Reader

There are quite a few nay-votes here, so I'll give a yay vote.

In the past I have had to deal with very complex configuration systems; several thousands of lines of XML which were supposed to direct many applications.

And writing:

    value['this']['that']['the_other']

vs

    value.this.that.the_other

Really starts to matter for code legibility. Of course if you change your XML configs your Python will break - but it would likely break in any case.

The other case where I have found this applicable is when I have complicated JSON structures in the form of user input. Typically I augment this style by using a JSONSchema that ensures attributes do exist and have defaults (or None).

One option would be:

    value['this.that.the_other']

Bonus: Make it support JSONPath and you can use it to extract lots of things at arbitrarily nested levels.

I see the value in the latter for code legibility. My gripe with it is, when I'm new to a project that uses a dict implementation like that: how do I know what happens if for example `that` is missing? Does it raise AttributeError, KeyError or just return None?

Personally I'd prefer a helper functions like

    deep_get(dict_, dotted_path[, default]) -> value

You stil have to check the docs/source what exactly happens, but it is just one simple function instead of magic methods. And i can keep using plain dicts everywhere.

EDIT: To clarify a bit. An experienced Python developer should immediately recognize a call like `deep_get(value,'this.that.the_other')` as something project/framework specific and not built-in, while `value.this.that.the_other` is ambiguous.

You can get part of the way there with operator.attrgetter:

NT = collections.namedtuple('NT', ['x', 'y'])

nt1 = NT(1, 2)

nt2 = NT(nt1, 'asdf')

assert nt2.x.y == 2

get_two = operator.attrgetter('x.y')

assert get_two(nt2) == 2

If you wrap the operator.attrgetter in a try/catch, you can force a default too: https://pastebin.com/bFxGr22E

I never thought much about best implementation because I only used it i think once in ten years. But afaik I used something like this, that does the job and is easy to read

  _empty = object()
  def deep_get(dct, dotted_path, default=_empty):
      for key in dotted_path.split('.'):
        try:
          dct = dct[key]
        except KeyError:
          if default is _empty:
            raise
          return default
      return dct

Personally I try to implement it as an AttributeError, it seems the most intuitive. By the way, I do understand your point and it's pretty valid in a lot of cases.

You can use `get` from the `toolz` package:

    toolz.get(value, ['this', 'that', 'the_other']

There's also a Cython implementation called `cytoolz`.

What do you typically use for enforcing a schema? I've been using Good[0], however since it is meant for general data structures, users don't get line/column printouts for invalid data.

[0] https://pypi.python.org/pypi/good/0.0.7-0

I like jsonschema [1][2], primarily because it's a cross-language standard. Though, I have never used `Good` - it seems to be pretty nice if you're Python focused.

[1] https://pypi.python.org/pypi/jsonschema

[2] http://json-schema.org/

Give `destructure` a try https://github.com/selik/destructure

Better would be to code depth-first-search and breadth-first-search functions and use those on plain nested dictionaries. That's what XML did via XPATH.

If you use something like this, future maintainers are going to hate you.

Agreed. This is not the sort of code you want to be dealing with in a few years time.

Also, I seldom find that string based dicts are what I need. One of the best things about Python is being able to use virtually anything as a key.

> One of the best things about Python is being able to use virtually anything as a key.

I find that to be the thing one of the worst thing ever made into a programming language.

You are facing a program/API/library and you can have absolutely no clue of what it will give or expect without extensive reverse engineering.

I find that's more a problem of bad design and bad names, rather than flexible types.

> You are facing a program/API/library and you can have absolutely no clue of what it will give or expect without extensive reverse engineering.

Or reading the docs.

Wait, you choose what to put in the dict. I like being able to map from, say, db objects to internal representations. Why wouldn't you want that?!

The concern is facing something that expects a map, and not knowing how to correctly populate it.

Time has taught me to prefer lists / flat structures as arguments to functions (and in most other storage and transfer contexts). Besides, either you're passing in entities that conform to the api, or you're not. Nothing about arbitrary keys in dicts changes that.

Yep. Or keys with dots in their names, or keys actually named something like "__eq__" (strange but not impossible).

The worst thing about abstractions is when they break. I can always figure out how to fix or extend ugly but straightforward code. I may not even know where the manual is when a fancy wrapper dumps a trace.

>I may not even know where the manual is when a fancy wrapper dumps a trace.

I read somewhere that if it is written in self-documenting code, the problem and its solution will be obvious.

Was that sarcasm?

What counts as "self-documenting"?

Undocumented code is self-documenting. Because it only ever does exactly what is written, it must be intended to do exactly what it does.

Ah, Dr. Pangloss.

I ended up making a custom data structure and query language around this kind of notation. Not in Python(am working in Haxe) but easily generalized. The structure is either array or string map or ordered string map with a dynamic data field and optional string tags. So it maps very well onto JSON or XML style data, but comes with the benefit of a very unambiguous way to do queries, where I can mix together addressing styles(take this numbered index OR take the named key) and pull elements into an array without much difficulty. The lookup syntax is "@<key>#<index>..." and each key or index entry is parsed into an ADT. I can pass along partially constructed queries as the relevant data to be processed. It's not quite as flexible as SQL since the data has to come in tree form, but it creates a major design affordance: the same query can be passed to a custom algorithm or custom data structure, with less ambiguity than a simple string lookup.

To some extent this retreads ground covered by e.g. XPath, but the specific structure puts it closer to my day-to-day needs.

Yep. Last time I tried one of these I had a dict that had keys that conflicted with the dictionary keywords, like "items" or "keys" or "values", so I wound up functions back instead of data.

Author here: I totally agree with the fact it has shortcomings and definitely isn't aimed to replace everyone's dict everywhere.

I actually created the first iteration of this around 3 years ago, aimed primarily for sysadmins and others who use the console more than actual developing. As I find it extremely convenient to tab complete keys and a lot less typing.

From there it has obviously evolved to try and fit more peoples needs, and I use it regularly in my projects.

Just an FYI: the newest version of ipython allows you to tab complete keys

Also dot means attribute means less speed.

If you have to worry about that, you've taken many wrong turns in the past.

That's a huge micro-optimization. For a trivial example of a class having the attributes "a", "b" and "c", and a dict of the same,

    In [4]: %timeit d['b']  # dict
    10000000 loops, best of 3: 42.7 ns per loop

    In [6]: %timeit f.b  # class using __slots__
    The slowest run took 30.10 times longer than the fastest. This could mean that an intermediate result is being cached.
    10000000 loops, best of 3: 44 ns per loop

    In [9]: %timeit b.b  # class using __dict__
    10000000 loops, best of 3: 48.1 ns per loop

You're talking <6ns per access; except in the very exceptional case where you know you need this, in Python of all languages, it's an over optimization. The maintainability of having the stupid-simple class vastly outweighs the speed.