Boltons: over 100 Python utilities | Hacker News Reader

Boltons: over 100 Python utilities

github.com

282 pointsmhashemi53 comments

Very nice collection. I like the cacheutils LRI and LRU setups [1]. Interested in the StringBuffer [2]. The Table object looks useful [3]. Overall some great additions in utils that you can tell were added through usage and shipping needs especially in strutils, listutils, formatutils etc. Lots of common ones in our own python or other platform libs.

In any review there is always a few WTFs like this one [4]. I am surprised you didn't get arrested during that development

[1] https://github.com/mahmoud/boltons/blob/master/boltons/cache...

[2] https://github.com/mahmoud/boltons/blob/master/boltons/strut...

[3] https://github.com/mahmoud/boltons/blob/master/boltons/table...

[4] https://github.com/mahmoud/boltons/blob/master/boltons/easte...

Hahaha, so glad someone spotted that. :)

What to do about boltons - and you will get boltons.

This doesn't seem portable: https://github.com/mahmoud/boltons/blob/master/boltons/tests...

That probably doesn't even work on other people's computers.

Hahaha, true! Get good look, that'll be gone tonight.

Some irony there at clastic [1]

"A functional Python web framework that streamlines explicit development practices while eliminating global state."

Meanwhile somewhere else, it comes in as an overbearing global solutions provider. Hope that does not spell the end for the table_html utility.

1. https://github.com/mahmoud/clastic

A more permanent link: https://github.com/mahmoud/boltons/blob/e5b75821781b37ee8ccf...

See https://help.github.com/articles/getting-permanent-links-to-...

I know about getting permalinks to Github. Also:

* You can shorten the commit id:

https://github.com/mahmoud/boltons/blob/e5b7582/boltons/test...

* You can highlight ranges:

https://github.com/mahmoud/boltons/blob/e5b75821781b37ee8ccf...

* You can ignore whitespace in diffs:

https://github.com/blog/967-github-secrets

Quite similar to my project, Python Toolbox:

https://github.com/cool-RR/python_toolbox

It supports Python 3.

4,800 commits without commit messages? Whoa.

Yeah, I get that a lot. I don't believe in commit messages. I know that 99% of developers are disturbed by this, but I think it's the best way to work so I'm willing to deal with people not liking it.

Why do you think it's best to give no indication of what was changed by a particular commit? While I feel strongly against this, I'm really asking out of genuine curiousity, not to bash whatever seems to be working for you :)

I could write an entire article about this (possibly someday I will), but to explain it briefly: I feel it hurts my creative process to have to take the time to write these commit messages at every commit. It's making me not want to commit. I treat a commit as an extended save operation, so unconsciously avoiding saving is definitely not something you want to do. I know most people would object to my use of a commit as an extended save, but I stand by that opinion as well, especially in these days of deploy-by-commit systems like Heroku. Sometimes I make commits that are smaller than a commit message would be so having to write a message will be a big waste.

Also, commit messages are a form of documentation; but they are one of the least useful ones. Much less useful in my opinion than docstrings, topical guides, references and blog posts. So I think that spending time on them is a much lower priority than spending time on the other kinds of documentation.

Thanks for sharing. When I want to commit but it's not a useful unit of work, I'll occasionally just commit with a WIP message and then amend the commit as necessary until it's done and write a real message.

I'd love to explore this further, as it looks like you've put a ton into it, but the simple difference in external documentation being there for bolton makes it much more attractive to me.

Does look like you docstring your functions--can't you autogenerate a docs site?

Edit: I do see the docs site, but it's a pretty sharp subset of the modules. The topical guides are nice, but I really prefer web-accessible basic docs too, so skipping those in favor of topical only isn't awesome from my POV.

I agree, more documentation would be very helpful. Unfortunately, no time...

FWIW, even the most basic module:function:purpose docs would help.

The primary challenge in a toolbox is knowing what tools are there. Having to either browse the code or pydoc it myself to know the scope is a bit of a hump.

First thing I did with bolton was read the docs start to finish. That way when I hit a problem, I'll hopefully remember bolton could solve it.

I will check yours out when I have time to go through it--it looks very multifunctional.

Note that Boltons also supports Python 3, while retaining individual module independence. Happy bolting and cooooool-RRunnings!

Really like the concept, and would love to use this in some of my projects, but no Python 3 support.... :'(

Python 3 support is pretty trivial for all the ones that come to mind. I'll look into it tonight.

Python 3 support has been added as of Boltons 0.6.

I'd been wondering how to do this... https://github.com/mahmoud/boltons/blob/master/boltons/easte...

Very nice. These solve a number of small but real problems and it looks like a great utility package!

I think the docs of iterutils.split() vs. iterutils.split_iter() might be a little off.

They both claim to return lists but split_iter() says if you want lists, see split(). Without having installed the package yet, my guess is split_iter() returns an iterator. Either way, the docs are contradictory.

https://boltons.readthedocs.org/en/latest/iterutils.html#ite...

Thanks for this!

From my understanding the docs are correct:

> split_iter() yields lists of non-separator values.

-> returns a generator of lists

vs for split()

> Returns a list of lists.

Note that the examples in the docs always wrap split_iter() in list(), hiding the generator step from the visible output.

One returns a list of lists, the other an iterator of lists.

split_iter() docs say, "split_iter() yields lists of non-separator values," whereas split() docs say, "Returns a list of lists." Still, I'll look at clarifying it. ;)

You're absolutely correct. I read "yields" colloquially, not in terms of "yield." Written as designed. :)

There is a LRU cache in the standard library starting in 3.2: https://docs.python.org/3/library/functools.html#functools.l...

Doesn't have quite the same options, but arguably nicer to use because it's a decorator.

boltons has an all-purpose cache decorator, which gives you an arrangement with a lot more in terms of flexibility and transparency: https://boltons.readthedocs.org/en/latest/cacheutils.html#au...

"The :class:`set` type brings brings the practical expressiveness of set theory to Python. It has a very rich API overall, but lacks a couple of fundamental features. For one, sets are not ordered. On top of this, sets are not indexable"

I suppose that may be because sets are not supposed to be ordered. Nor indexable.

In which context? I guess not in math:

http://mathworld.wolfram.com/TotallyOrderedSet.html

You guessed wrong. From your own link:

>>A total order (or "totally ordered set," or "linearly ordered set") is a set plus a relation on the set<<

Please note that "set" (unqualified) means the general concept of a set. You can extend that general concept, but then what you get is a new thing (in this case a "totally ordered set").

And who said that our use in Python doesn't need this extended concept?

Did anybody say it doesn't?

I am sure I didn't.

But claiming that Python's implementation of sets "lacks fundamental features" is unfair, wrong, and making the world a dumber place. Some might want to live in that place. I don't.

It seems that your confusion stems from the premise that because this Python collection is called a "set" it should have the absolute minimum functionality of the mathematical notion of a set.

Python collections are data structures (containers etc). The standard library is not a symbolic math library, and they are not supposed to model math entities perfectly (e.g. they can me mutable), they are used for their utility, which might or might not need them to be ordered.

The only confused person here is plainly yourself. Either that or you are somehow personally offended and desperately looking to excuse that wrongheaded comment, changing tack every time your latest attempt is shot down. Or you're just a troll. I am running out of care to educate you.

Go on and complain about tree types for not handling cycles. Because hey, sometimes you need cycles, don't you? I will be sure not to comment.

Besides being rude and offensive (plus tacky, with your meme picture), you still don't seem to comprehend that:

1) the Python standard library is not SymPy or some math modelling library, and its collections are not supposed to map 1 to 1 with the corresponding mathematical entities -- just have those properties that are useful. In fact, the standard python set type is mutable to begin with.

2) even if they were, there are versions of sets in maths were ordering is indeed desired.

So this makes your snarky comments needlessly smug and misguiged in 2 different ways. But given the rudeness, I don't think this warrants any more discussion. Have a nice life.

I like the spirit, however a lot of these can be done with a simple one liner... Example for chunked iteration:

```

def chunk_iter(some_seq, n):

    return (seq[pos:pos + n] for pos in range(0, len(seq), n))

```

example for multi-dict: (unless I'm missing something)

```

from collections import defaultdict

d = defaultdict(list)

d['a'].append('b')

d['a'].append('c')

dict(d)

{'a': ['b', 'c']}

```

While your version are usable. They differ in the requirements fulfilled.

e.g.

chunk_iter - iterables not just sequences

multi-dict - usability

Plus nice error messages.

The goal is to be standard library quality.

That's right! Boltons are meant to replace those one liners, unless you really really think downstream customers are going to be impressed with code golf birdies. ;)

Here's my own version of an iterating chunker (itertools needed imports omitted).

  def chunks_of(it, n):
    return takewhile(len, imap(list, imap(islice, repeat(it), repeat(n))))

But this library functions adds error checking and are more expressive (I wrote the above and always have to look at it lenghty to understand it).

I've not paid much attention yet to the license scheme, but if it's friendly to be used on commercial projects, I might take the plunge.

BSD, go nuts!

Python really need something like Go's official libraries. I mean those with a prefix "golang.org/x/". Those libraries update more often than stdlibs but still provide a concrete common ground for developers. It is like "evergreen" stdlibs.

This is being talked about now with [PEP 411 - Provisional packages in the Python standard library][1].

[1]: https://www.python.org/dev/peps/pep-0411/

So glad someone did something like this and made it presentable enough for widespread usage.

In contrast, my own "utils" libraries[1] are often too specific, or not well documented. However, there's already some overlap (eg. I had my own version of funutils.mro_items()), and later today I'll take a look through and see what of mine might be a candidate for inclusion.

[1] https://github.com/ekimekim/pylibs/tree/master/libs

Please do! Quality is not uniform throughout the library, and while mro_items covered my use cases, I fully anticipate others might have more comprehensive approaches.

Is that their real name?

Are they related to that singer guy?

So glad to see this - there are things here I thought I was weird for having to make and want myself - liked chunked iterations. I assumed there as something wrong with me ... Glad to see it's not just me

Nice one

These are great!

You know, this makes me thing about how there are so many things that I wish Python would come with: requests, lxml, gevents and a good amount of stuff from the package.

I wish there was Python 2.8.

I wish the regular threading interface was as nice as gevents. Gevents is stunningly simple to use for a lot of simple things where I find trying to use the threading library (I.e. when you want to deal with paramiko) is much harder.

Where are the tests?

They're around. Doctests cover most of the library, many modules have main()s that take care of some more complex unit tests, then the applications that use the utilities cover a lot of regression cases. Tests are kept as close to the code as possible to keep each module independent and self-explanatory, as explained here: https://boltons.readthedocs.org/en/latest/architecture.html

wow, this is awesome.

boltons.tableutils has a lot of TODOs - if you like the concept, the littletable package [1] takes the concept of a python query language pretty far. I put it in production and extended it even further [2]. We'd both welcome help / pull requests.

[1] https://www.google.com/search?q=python+littletable

[2] https://github.com/asah/littletable

Reply on news.ycombinator.com