Checklist for Python libraries APIs | Hacker News Reader

Checklist for Python libraries APIs

python.apichecklist.com

178 pointssebg20 comments

I agree with all but this one:

Avoid hiding parameters that could be useful E.g. the API is calling another lower-level one, but it isn't exposing some useful parameters the lower-level API supports

APIs have to be supported for years. Anything they expose can become a serious hindrance for API migration in the future. Best to keep the API as simple as possible at first, then only expand it when it is genuinely necessary.

Another one I would add: Keep the API stable. I'm developing a cross-platform file manager with a Python API [1]. I made some (necessary) changes to the API last week. This broke the work flows of some of my users. They were not pleased.

[1]: https://fman.io

Yeah, I also think explicitly exposing each parameter called in a lower level API is a recipe for disaster.

One way to avoid that is to pass down kwargs to the low-level call. `pandas` uses that in their plotting function (passed down the matplotlib call).

But can you decide what is necessary for your library's users?

Yes, they'll let you know. Predicting everything people need is impossible.

And in python, if they really need more they can monkeypatch the world.

Author here. Thanks for the feedback. I wrote this because I've seen too many APIs I couldn't use or had to discard later on because they forgot to expose a lower-level parameter. Yes, supporting additional parameters is a trade-off on stability, but I think it boils down to separating things that change frequently from things that shouldn't change, i.e., having a low-level interface and a high-level interface on the same API. Advanced users that work with internal parameters should know they're on unstable territory. Document that to acknowledge them about the risk. They'll have the flexibility they need knowing about the cost. I guess that if you don't do that, you'll lose users for an API that does.

You don't need to expose each internal parameter or write adapters to everything you use. As suggested by the sibling here, sometimes all you need is to pass down kwargs to the low-level call.

Other API guidelines that suggest a similar approach here:

http://tomasp.net/blog/2015/library-layers/

http://www.thereformedprogrammer.net/what-makes-a-good-softw...

Why not fork the library if you need to modify low level behavior?

For the 1% use cases, expect forks. For 19%, give them options. For 80%, give them good defaults.

Missing the most important one: Provide type/function annotations.

I'm also skeptical to all the recommendations about opening up the internals, getting out of the abstractions and providing replaceable hooks for everything. This can create premature abstractions for every conceivable hook and exposing internals will give you a backwards compatibility maintenance hell when you have to be compatible with internal details you didn't think people depended on.

Please please make your module name consistent with the package name. I don't want to install beautifulsoup4 only to import bs4. It's really really confusing.

For a library that does such a good job of simplifying the complex, that design choice always perplexed me.

When you look at their example import code it makes even less sense. People who need shorter object names already do that, no need for it to be at the library level.

I've made this mistake. I just thought that A) people wouldn't actually shorten the module name even though they could B) it wasn't that confusing having two different names for import/package name.

It was dumb. I won't do it again.

I think if you do it early on you are stuck though. If beautiful soup changes now they will break a ton of code.

Thanks for the suggestion. Will add that to the checklist. Feel free to contribute too: https://github.com/vintasoftware/python-api-checklist

> have sane and smart defaults.

Using Keras has made me appreciate this a lot.

Most deep learning layers have a bunch of parameters that need to be configured for best results. A lot of research has gone into figuring out best practices and Keras uses those as defaults. Saves me a lot of time.

This is great! I want to also posit that these principles can be used not only for public libraries, but for the software inside of your applications as well.

Honestly I couldn’t care less about PEP8... frankly my flake tool triggers a build failure more often than an actual code problem. This list, however, should be used like a golden rule for python software development (and all software, regardless of language) – a lot of these principles are universal.

Python offers programmers great power but with great power comes great responsibility. You can program like a bull in a china shop if you're not actively practicing principles such as those espoused in this checklist. On the other hand, be cognizant of your company's culture. It may not be acceptable that a release is delayed because the code isn't principled enough. "Make it work, make it right, make it fast" is an old programming adage that is still practiced by many people. Make sure you know what principles your organization follows and what the consequences are if you try to make things right before you ought to or are expected to.

This can be viewed without JS enabled:

https://github.com/vintasoftware/python-api-checklist/blob/m...

Really great list. Thank you.

should be python-apichecklist, or otherwise it is missing $language.apichecklist.com

It's nice, but whenever I see lists like these I always say the same thing: If your checklists can't be automated on the CI server they are worthless for developers. I'm not a huge fan of Go, but it needs to get credit where credit is due. Writing code quality tools and linters is extremely easy with that language. Which helps keeping codebase quality extremely tight.

Author here. Not worthless because you can add checklists to your PR template, PR review, QA process, etc. But I agree that automatically checking things is the next step. I've talked about how to achieve this with custom linters in a DjangoCon talk: https://youtu.be/IMRHFlDxaqU

Reply on news.ycombinator.com