PEP 594 – Removing dead batteries from Python's standard library
python.org
python.org
Back when, I lurved me some Perl TMTOWTDI as much as the next guy--plenty of uses for such a thing in a Swiss army chain saw--but then Python had a good (or at least different) answer to that by reducing the language load and focusing more on the problem.
So tell me, how many string formatters do we have now? Could we please decruft and dump some of them while we're at it? % or f' or .format or whatever, but please just pick one and get the rest out of my face.
https://www.python.org/dev/peps/pep-0020/
https://en.wikipedia.org/wiki/There%27s_more_than_one_way_to...
For something as essential as strings, it makes sense to recommend the "new" way, and continue to support the "old" way.
Now, of course Python is interpreted and packages are shipped as raw source code and this makes things harder for library authors, because were there a compile/transpile step, they could write using the new and shiny obvious ways, all the while providing compiled blobs (or transpiled libs) targeting multiple versions.
Technological purity or scratching some philosophical itch would be a poor reason, IMO, given how broad the externalized costs run.
So is it provably easier to reason around the new style? Reduce computing time by an order of magnitude (in the modern era where compute cycles and memory are hella cheap)?
Code is to be written so it’s understandable and useful for humans first, right?
How does a third syntax for a solved usecase offer real value to the user base?
Or are we just being sycophants of so-called experts, peddling some novel “look”. Experts who largely built a rep on first mover advantage, but have since seen that excuse for being deified evaporate as the rest of world learned all the same tricks?
The new syntax is more like other modern languages, not less.
PHP echo "There are $apples apples and $bananas bananas.";
Ruby puts "I have #{apples} apples"
TCL puts "I have $apples apples."
Typescript console.log(`I have ${apples} apples`);
Python print(f"I have {apples} apples")
Looks pretty similar to me, and if you like understand-ability by humans so much I think f-strings win hands down. But if you disagree, there's plenty of code out there still using % strings or .format() and they're not going anywhere any time soon.
"I have {{apples}} apples"
PEP 498 mentions that they did look at other languages to see what was supported:
https://www.python.org/dev/peps/pep-0498/#similar-support-in...
And they link to this Wikipedia article, which lists many examples:
https://en.wikipedia.org/wiki/String_interpolation
Scanning through that, my impression is that "there is nothing new under the sun" and aside from an occasional "$" or "#" prefix, or a willfully arbitrary deviation like ColdFusion, the conventions can all be traced the Bourne shell /bin/sh/ which was released in 1979. Prior to that, the prinf() syntax was presumably the most common, but it's obscure.
Does anyone know if the ${variable} convention was original with the Bourne shell, or if it can be traced back further?
I’m rather done with this interpreted, dynamic typed, language and the hell holes they dig us into.
DRY could be applied to more contexts than code logic: stop rewriting language features unless there is more than a subjective win of “looks nicer.” or performance is improved by an order of magnitude.
The conversations the Python community should be having are not “once again we must discuss and consider a solution to a solved problem.”
When that’s the sort of progress the language devs are prioritizing, it’s a sign to me they’re out of ideas or incapable of fixing the bigger flaws.
HN is a community. Users needn't use their real name, but should have some identity for others to relate to. Otherwise we may as well have no usernames and no community, and that would be a different kind of forum. There are legit uses for throwaways, just not routinely.
Lots more explanation: https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
To me, there's a difference between "one way" and "one interface." I agree we should depricate %-formatting in strings, but .format / f"" are different interfaces on the same functionality - which seems fine.
log.debug("foo bar %s", foobar)
I am, however, in complete agreement that logging should adopt at least the same formatting used by .format
I get all of these arguments. I see formatting errors so often in error handling code I would much prefer they'd get caught during normal execution. I can't see trusting a heavy query to lazy evaluation...or see it coming up all that often.
I see both sides, but really think using f-strings in logging is the right compromise for Python...but I just don't see them dropping c-style formatting.
3PP(Third Party Packages) issues are responsible for a lot of application security vulnerabilities today. Every large enterprise organization has no idea what packages a developer is pulling onto their laptop and into their codebase.
As a security engineer, I like having a core team and a standard library in place that has gone through a long mature process with experienced developers, instead of someone who just git pushes code every night. You have no idea who is on the other end of that push too. It's too hard to keep track of changes and causes us to have to pin packages and versions that have been ok'd for use instead of a new release of Python. You have no guarantee the security engineer who reviewed the code didn't miss anything either.
The question that you’re ultimately seeking the answer to is “what code has been reviewed and which reviewers do I trust?” - lots of ways to solve that.
You are completely correct. There's no reason at all why any random PyPI package can't have meaningful reviews published. I would go so far as to say that this is true without any changes to the standard library or current processes at all.
With that in mind, I'm thinking about all the various packages my colleages use. I don't think I've seen published reviews for an appreciable fraction of them in any language. This suggests that perhaps enabling reviews might not be the hard problem to solve here.
This thought process also hilights to me that the major advantage of a stdlib is that you have a higher degree of assurance that its contents have seen meaningful review by multiple sets of eyes. It's not just the potential for review that matters, it's the degree of assurance.
Thank you for putting this so succinctly. There are other supply-chain problems in decentralizing control of the modules though:
Modules might only have a few people or one person responsible for them. Consequently suspect commits by a malfeasant or compromised (hacked) team member might not be noticed. Maybe this is a variant of the old saw "with enough eyes all bugs are shallow" but my gut feeling is that commits to Python core will get more inspection than those to a small 3rd party module. Today, modules get some review for free, just by being part of stdlib. Even if I know and trust @jdoe I have less assurances that they didn't get phished and their repo tweaked.
Also trusted team members/organizations may change over time. The browser extensions world is the poster child for this, where we've seen not only similarly-named malicious extensions posted to stores but also once-legitimate extensions quietly purchased and subsequently subverted by bad actors.
This is one example (adware: could've been much worse): https://www.bleepingcomputer.com/news/security/-particle-chr...
I like the idea of a review process but I have a hard time imagining a crowd-sourced system that wouldn't get gamed. We have a "dissolution of responsibility" problem: millions of companies rely on these components but have no explicit responsibility of care. Perhaps that needs to change, somehow.
And anyone who pays into the fund should be able to vote on what package to review. (And there should be a weighted lottery, so eventually small contributors' wishes have a chance to get fulfilled.)
For good measure anyone can put this on the blockchain, make a flattr/patreon thing out of this. (Somehow use github sponsoring...) Who knows.
The problem is that security assumptions change. If you're looking at some code in 1995 that accesses an API, it seems reasonable that it disables validation for the SSL library it brings in to access an HTTPS URL. Handling that properly would have been a huge pain, faster is better.
But in 2019 you'd be appalled, how is this thing not depending on certifi and requests? As it is it's wide open to a MitM attack and you'd get blind sided.
A much older example: The C standard library includes a function that's very narrowly useful for copying strings into fixed width buffers with no terminator, called strncpy() and in 1970s Unix I'm sure this was invaluable. But obviously today "copying strings into fixed width buffers with no terminator" is essentially never what you wanted, but people assume based on the name that strncpy() will do some other thing they do want, and insert a security defect into their code.
Arguably, what makes C the perfect example is not necessarily its standard library functions per se but the language's official string data type.
Yes, but do you like funding them? This is all freely downloadable open source software.
This is an appealing sentiment, but you will get hacked if you rely on it.
Python 2's standard library didn't do SSL certificate verification by default until the fairly controversial PEP 476 from late 2014, which changed the default in a point release of Python 2.7. Through 2014 (and for some time later, given how slow it is to get new upstream releases through distros onto someone's computer), the standard thing to do to use HTTPS securely was to use requests, a module developed by someone who got commit privileges taken away on another of his projects for making multiple releases a day. You were more secure with his code than with the standard library.
I think you have no more idea who's on the other end of pushes to the standard library than who's on the other end of pushes to PyPI. (Which is to say, in part, that you can equally well have an idea of both of these if you put effort into it. I've met the maintainers of several of my third-party dependencies at PyCon.) Something being in the standard library doesn't mean it has a higher class of developer behind it. There's more overhead, but it doesn't mean there's more maturity; many third-party module developers are more experienced (either in their field, if it's something like crypto, or just in general as responsible developers) than standard library developers. And if anything it means that security updates are slower and rarer because the process is more painful.
Consider the argument of the 2009 paper "Security Impact Ratings Considered Harmful" https://arxiv.org/abs/0904.4058 (disclosure: I'm a coauthor). Whether people find a vulnerability important enough to patch has little correlation in practice with how exploitable it is. And the only thing that gets regular attention is the latest development version of the code. If some module was significantly refactored or reimplemented, no upstream developer is looking for bugs in the old version. So, if you want to be safe and you haven't personally both audited and fuzzed the code you're running, you actually want to be running the latest released version of that code, regardless of whether someone stuck a CVSS on the old version yet. I've met very few companies who can upgrade to the latest Python 3.x minor release promptly when it comes out (and several who are still on Python 2!). I've met many companies who can pip install the latest versions of their dependencies without too much trouble, though.
But, after the initial pain of the 2.7->3.0 transition, I doubt we'll ever see another major version number jump, even if a logical use of version numbers would merit it.
For the point of removing the modules, check the "rationale" part of the PEP.
API 101 - there is very little cost to leaving them in, but a hidden major cost to having them disappear, usually for non-developers.
--
By the way, "batteries included" is one of the BEST features of python.
Have you tried to fix something in your house with a "homeowner's toolkit" which is usually something like a hammer, pliers, 2 screwdrivers, a putty knife and a few more basic tools?
It is REALLY tedious, like writing a C program with a few basic tools like stdio and ctypes.
More languages need "batteries included", maybe like Perl.
I think if the cost of deploying a script is 1, deploying it + a dependency is literally something like 100x. You have to make assumptions about all the environments the script will run in, and they are usually wrong.
None of these libraries need to be nuked from existence as a result of this change. I'd wager they'll move into PyPI modules so that teams relying on them could safely continue to do so.
> Have you tried to fix something in your house with a "homeowner's toolkit" which is usually something like a hammer, pliers, 2 screwdrivers, a putty knife and a few more basic tools? It is REALLY tedious, like writing a C program with a few basic tools like stdio and ctypes.
That's one extreme, but I don't think that's what's being proposed. The proposed model is closer to what Rust does today, where the core is slim, opening up new potential use cases, and the more complex functionality built on top of it is left to the community to maintain.
Take a look over some of the modules they're deprecating, like smtpd. What kind of standard library requires an SMTP daemon built in? That's akin to a homeowners toolkit including a planishing hammer [2] for some reason.
> I think if the cost of deploying a script is 1, deploying it + a dependency is literally something like 100x. You have to make assumptions about all the environments the script will run in, and they are usually wrong.
Depends on how it's done, honestly. Check out this Rust "scripting" system [1]. It has full support for third-party crates.
Just because it's not widely used doesn't mean it cannot be included in the stdlib. For instance, where I work, we have a MTA test framework that makes extensive use of smtpd to receive messages processed by our configured MTA which we then write to disk and subsequently make assertions on the message (e.g., contains certain headers, has certain recipients, etc.).
Having various protocols (client and server) in the stdlib is not a bad thing and, personally, I think it's very useful for testing purposes.
That's not true as newcomers will typically bias towards bundled modules instead of 3rd party ones. If it's well accepted that the bundled modules are bad in some way then you are encouraging further use of them by leaving them in. That's real cost, and not just on the Python maintainers side of thing.
maybe you could make it explicit for new scripts?
from __past__ import cruftBut really that's just a tutorial/discovery issue, which has many solutions. And not even a skill isolated to programming - how do you pick the "right" item on Amazon to buy?
Built-in functionality is less important if you can easily use packages.
Agreed with both points, the lack of batteries makes Lua quite annoying for me personally. Python and Perl are much more comfortable because you don't have to hunt down the most common operations.
Of course, Perl 6 has decided not ship batteries included, replacing batteries with a fusion reactor instead.
Sure. But did you read the list of proposed deprecations?
It's not like that homeowners toolkit becomes any more useful when it's also got a buggy whip tool, a set of special allen-like keys that only open the case of a TRS80 or a Commodore64, and one of those picks for getting stones out of horses feet.
Most of the proposed things to take out seem obviously "right" to me. AIFF sound file support? MacOS9 binhex? CGI? _Maybe_people are still using those things, but it seems to me like they're edge-case enough to let them know now that they'll need to do "pip install aifc" or "pip install cgi" sometime after late 2021 if they update their python installation (which they can choose to put off until at least 2024 or so, just like all of us who're still sometimes using Python 2.7...)
We'd be on way more than Python 3 if the major version was bumped for each one.
I'm sure Python 2.7 broke compatibility, but you don't see people refusing to upgrade from 2.6 ten years after its release.
How about <Very breaking changes>.<Breaking changes>.<Bugfixes>? That's what Python already does.
Semver is garbage.
I've since become disillusioned.
The problem with semver, in my experience, is that it's impossible to predict whether a change will actually break someone else's code. Of course there are certain classes of change that are more likely to cause problems for other people. Changing a function signature, or deleting a function outright, is obviously a breaking change.
But the line between breaking and non-breaking isn't a bright one. Move away from the obvious examples and things start to get murky. Even the humble bugfix can be problematic. What if a client application unwittingly relies on the buggy behavior? Now that fix is breaking for them. Is that a contrived concern? Maybe—though anyone who has written an emulator can attest that this a real problem.
What about a non-breaking feature addition? Let's say the new feature requires some extra branches in a function, but doesn't change the function's interface or behavior for people who don't use the feature. Fine, non-breaking. Now say these branches alter the function's performance, and a client application's batch job that used to run in under an hour now takes four hours. It does run, so it's not "broken," but four hours is an unacceptable runtime to the users. Is that still a non-breaking change?
What these examples demonstrate to me is that semver's breaking/non-breaking change concept is incoherent. It conceives of changes as universally breaking or non-breaking, out of context, but a change can only be breaking or non-breaking in the context of a specific client application. Even the seemingly obvious example of deleting a function is non-breaking for applications that don't use the function!
I think the way we release software reflects that we know this deep down, even if we don't admit it. Imagine how you might handle upgrading a library in an application you've written. The new library is a bugfix release. Do you upgrade the library, push it to production at 100%, and gallop off to lunch without so much as a glance over your shoulder? My guess is no. Personally I'd be running my test suite, reading the library's change log, making a gradual release, and keeping a close watch on instrumentation during and after.
The interactions between software systems are simply too complex, too nuanced, too specific to the particular applications. We put all these safeguards in place and do things cautiously because we've been burnt too many times. And we've learned that in actual fact, the line between breaking and non-breaking doesn't exist.
Python's slow-moving gently gently approach to breaking changes has not been good for the ecosystem. I'll be glad when 2.x is dead. 7 months 8 days 14 hours... https://pythonclock.org/
This has knock-on effects: authors that want to deploy scripts/apps with the minimum fuss will avoid adding deps to whatever /opt-based repo RH ships Python 2.recent (and the hoops you have to jump through to install and activate that). So they remain compatible with 2.6.
All of the other applications and 3rd-party modules shipping with RH 6 are also chained to Py2.6.
Many conservative shops (industry verticals) will refuse to upgrade _anything_ until they absolutely have to. I suspect we live in slightly different IT worlds (lucky you!). This is a problem I see frequently and that's why I'm suggesting Python needs more strict impetus for timely upgrades, not more decade-long opportunities for balkanization and incompatibility.
Do you think the transition would have been less painful if 3.0 had been called 2.8?
Developers don't have a compelling reason to use 3rd party modules instead of the standard library. Therefore they don't. You consider that a problem and want to encourage them to use 3rd party modules more.
In short, you want developers to use 3rd party libraries because there is no compelling reason to do so?
> A lean and mean standard library benefits platforms with limited resources like devices with just a few hundred kilobyte of storage (e.g. BBC Micro:bit). Python on mobile platforms like BeeWare or WebAssembly (e.g. pyodide) also benefit from reduced download size.
That's a silly reason. Just make a separate distribution with a stripped down standard library.
>That's a silly reason. Just make a separate distribution with a stripped down standard library.
True, you could have a std and core library. I think Rust does this for embedded. IMHO if you take away the large standard lib, except for pandas I have no more reasons to use Python. I don't like the language that much, but the std library is pretty good for scripting when you need something on a target system that only has standard Python.
https://docs.python.org/3.8/library/urllib.request.html#modu...
1. urllib [1] 2. BeautifulSoup and a comment mentioning requests [2] 3. requests [3] 4. urllib, httplib [4]
Which is looking better than I expected... but no information on 2 or 3 relating to how you install those libraries. So 1 and 4 will Just Work.
[1]: https://stackoverflow.com/questions/45717889/read-the-text-o... [2]: https://stackoverflow.com/questions/26050064/automating-down... [3]: https://stackoverflow.com/questions/44553348/how-to-download... [4]: https://stackoverflow.com/questions/2646288/retrieve-some-in...
So I'm not quite sure what is the best thing to do. It sounds like requests III is a Python3 forward version. So it might make sense to ship that?
Python has decided to include packages developed outside of the stdlib before; I think json, pathlib, and maybe optparse.
Hm interesting comment. Does anybody know what the new approach to parsing Python in 3.9 is ? I searched python-dev@ but couldn't find any references to it.
I found an interesting tidbit about Rust "switching" from LL to LR here:
https://www.reddit.com/r/ProgrammingLanguages/comments/brhdt...
And I noticed some rules in Python's grammar that are awkward in LL parsing (set and dict literals, and comprehensions).
I wonder if those things motivated the switch? They certainly work though.
https://discuss.python.org/t/switch-pythons-parsing-tech-to-...
And I posted here about it:
https://www.reddit.com/r/ProgrammingLanguages/comments/brz2y...
It looks like the set and dict literals I noticed weren't so much the motivating use cases, but even more fundamentally assignments and keyword args!
I’ve been out of the Python loop for a few years, but my last impression was that packaging and distribution of Python modules was far from a solved problem. Has this changed?
I'm always wondering if it has to do with the language and or just the package manager itself. Rust's cargo, Node's npm and probably quite a few others exist that work amazingly well.
Genuinely curious. I use Python daily and I rarely encounter problems with it. There is some getting used to in the beginning, but I went through a similar phase when I started using npm and cargo as well.
What I can say is that I had to go through a lot of experimentation myself to arrive at the tools I use now (pyenv + Poetry). And if anything, maybe the lack of one way that is adopted by everyone in the community is the problem.
That is, I think, the big problem. Just using pip has never really worked so people have built tools on top of pip, in parallel with pip and replacements for pip. Basically there is no one way to do things.
Besides, I have 15 years of Python behind me, I code and train in Python for a living, and I still use just pip.
pip and venv are packaged with recent versions of Python and do the job fine.
There is no good heuristic for picking one package over the other when they occupy a similar problem space. This is mostly a cultural problem, the community's efforts are lackluster.
virtualenv is not installed by default, making bootstrapping into a separate prefix that is independent from the system installation unnecessarily aggravating.
Semiautomatic packaging tools (e.g. pypi into rpm) produce low kwalitee packages and manual intervention more often needed compared to similar languages.
Worse are languages that simply don't have much manpower behind them in absolute numbers, e.g. CL/quicklisp. Given Python's mindshare, the results are subpar.
It is, but it's called venv these days:
python3 -m venv my_venv
I _hate_ this working on Java/Maven projects. Why do I need to run all the tests in order to install dependencies? The tests should have been run at _packaging_ time. I'm installing binary dependencies, why should I not trust that they have been tested?
These are fundamentally separate concerns. If you're worried about the robustness of your dependencies, review them first.
> There is no good heuristic for picking one package over the other when they occupy a similar problem space. This is mostly a cultural problem, the community's efforts are lackluster.
Stars on Github? Issues opened vs closed? Stack Overflow? The community is active in all sorts of places and it's not that hard to get a feel for the prevailing best tool for a given job. It just takes a little effort. The community's responsibility is to maintain those parts which they have created, not give you recommendations.
> virtualenv is not installed by default
python -m venv
> Semiautomatic packaging tools (e.g. pypi into rpm)
If you want to package your application and its dependencies as an OS package, that's fine. Do it as a single unit and isolate it from the rest of the system. We have venv (or pyenv if you need a different version of Python) to solve this.
I've experienced every headache there is with Python packaging. It's been bad, it is now better but still rather quirky, but it does require some domain knowledge. If you know how to use it, you will rarely experience serious problems.
I was trying to upgrade a package. I first looked for a "pip upgrade/update <package>"--doesn't exist. I then tried "pip install <package>" to see if it would offer to upgrade--it didn't. I think found "pip install --upgrade <package>".
I'm sure I'd learn these things if I did them all the time, but it's some slightly poor defaults (so slightly bad and impactful I can't imagine them being changed), and some discoverability and friction I don't see in package managers I use way less often.
First it forces the node_module in your repository, which mean you have to configure tons of tools to ignore it or you'll have a bad time.
Then there is no way to have comfortable local command, so plenty of time you gotta have sudo -g. Recent npm version now have a tool for that but the experience is meh.
Plus there is no builtin tool to manage several versions of node, so you gotta add nvm on top of that.
And then if you work on the front end, you can't even use whatever you npm installed stuff, you gotta setup the entire webpack shenaigan, making it literally the hardest stuff to setup. It's so complicated we rely on black box such as create-react-app to do the job for us.
export PATH=$PATH:./node_modules/.bin/
or put the command into `scripts` of package.json, and use `npm run-script foo`.
Have never used npm -g, and never will. Global packages are owned by the system package manager.
E.G:
- people telling you to sudo pip install
- people telling you to use virtualenv instead of venv
- people not telling you to do python -m
- people not telling you about setup.cfg
Don't run before you know how to walk.
you need python (and the distutils module) and you can use the get-pipenv.py
and separating venv management from pip is what causes The Mess
Python, lean and mean? Seems like an incredibly niche use case to restrict the python community to.
I think my Commodore/Amiga persecution complex is acting up again.
Why should this be fast-tracked outside the normal process? I can’t imagine any of these removals are urgent.
difflib - Text diffs, even html output
textwrap - obvious no?
rlcompleter - autocomplete symbols and identifiers, used in interactive mode
pprint - for printing complex data structures with indentation
reprlib - repr with traversal depth and string size limits fraction - Fraction('-.125') becomes Fraction(-1, 8)
statistics - averages and deviances tempfile — Generate temporary files and directories
glob — Unix style pathname pattern expansion
gzip, bz2, zipfile, tarfile - obvious
configparser - ini-like format
secrets - use this instead of random for safe cryptography
sched — Event scheduler
turtle — Turtle graphics
shlex — Simple lexical analysis
webbrowser — Convenient Web-browser controller, for the antigravity module!
Also secrets deserves to stay in the library, Python has hashlib and it should have a secure RNG by default.
glob too (or Path.glob).
And I've used shlex for command-line-escape parsing (it might not have been the optimal solution?)
LZMA provides much better compression at much higher costs. Generally speaking it's pretty strictly better than bzip2, not necessarily gzip (DEFLATE, really).
In my experience, zstd can be considered better than gzip/deflate (almost every time I tried it, it provided as-good-or-better compression at much faster throughput).
A few percent better compression than gzip and nearly 50 % faster decompression.
Gzip is pretty slow in the python standard lib.
The python zstandard bindings unfortunately do not allow back seeking.
rlcompleter - used by Python itself for the shell.
pprint - trememdously useful for debugging.
statistics - added recently because people kept rewritting mean() functions that were broken
tempfile.gettempdir(), glob("*.ext"), gzip, bz2, zipfile, and tarfile are kinda mandatory for a language you use massively for scripting
secrets - we had random. People used it for security all the time and created stupid security holes. So we added this.
webbrowser — fantastic module that open a new tab in the default browser from any code. It's one file, so for the value it provides, I'm happy it is here.
It was Windows-only (presumably a wrapper around the native tooling), and was primarily for creating Python's own installer, which apparently doesn't get built as an MSI anymore.
Semi-relatedly, Microsoft has already been exploring with the Python team MSIX deployments for the Python interpreter itself (you can even find it in the Microsoft Store now on Windows 10, and typing `python` on a command line in recent builds of Windows 10 will auto jump you to the Store if you don't have a python.exe in your PATH/installed).
What the uu module proper provides is a pair of awful file-to-file interfaces. Default to spitting the decoded file directly to disk at with the name in its header is, uh, questionable.
I have a couple of questions:
- I realize there may not be a fork with this PEP implemented, but how might this impact Python's local relative build time, and how might that convert over to the build pipelines? Drastically faster build times would be really nice.
- Are Python 2 -> 3 migrations for stdlib packages mostly rewrites or mostly tacking on compatibility layers like `from __future__ import unicode_literals`? If stdlib packages were updated with Python 3 syntax, it might indicate sustained demand for said package going forward. I'm not sure.
Packages in the stdlib don't need compatibility layer since they are always used with the Python version they were written for, it's motly incremental rewrites.
Regarding the build time, I don't think most of the time is spent testing those parts of the stdlib.
stdlib usually written in Python, not in C. So I guess it won't affect the build time much. It will reduce test time but I'm not sure.
I've never had a strong opinion about "batteries included", but boy there are some weird ones in there...
Pretty sure it tries to emulate awk's programming model, but even if I had a use for that I'd rather write these 3 lines of code myself - so the code is actually clear without looking at documentation, and so that it can be modified easily.
Of course if you'd only used fileinput for that it would be worthless.
However what it does is way more useful, namely iterate on the lines of all files provided as parameter (or sys.argv[:1]), fallback to sys.stdin if no files were provided, and swap the special sentinel `-` by sys.stdin.
Different strokes I suppose?
Why reinvent the wheel, when Python ships with an already written and tested solution? The 3 lines of code you would write are probably going to miss some corner case.
If I'm reading some unfamiliar code I would rather see one use of a standard library function call than several lines that I have to read and understand. And if it was really the first time I came across the `fileinput` module, I only have to pay the cost of reading the documentation once, and then forever benefit from having to read and understand less code whenever it is used.
Here's a better search, still 100k+: https://github.com/search?l=Python&q=%22import+fileinput%22&...
from importlib.machinery import SourceFileLoader spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
?
The complaint is that the module is designed poorly, which is fair; to remove CGI support without any "batteries included" replacement seems a bit of a shame, though.
What's actually in the module is a rather clunky, but serviceable, system for processing html form data.
Perl removed CGI from its core back in 2014 - https://perl5.git.perl.org/perl.git/commitdiff/e9fa5a80
If you’re writing a Python script that outputs an audio file, outputting to AIFF seems like a save bet. Then again I’m not sure how many people are actually using the module from the stdlib--but neither does the PEP’s author, it seems. His level of research was asking his friends on Twitter:
https://twitter.com/ChristianHeimes/status/11302577994753351...
For example-- why does "add" add two fragments, but "mul" multiplies one fragment by a scalar value? Was the assumption that "mul" will probably just be used for attenuation?
It would be fun to build a tk-based audio app around audioop for the sole purpose of complaining about the deprecation of this module. :)
Also, the changes sound like a major version bump to me, but I doubt people want that to happen after Python 2 to 3...
I think few other legacy platforms Java(JDK), JS(NodeJS), Ruby etc. also in need of this kind of refactoring exercise and removing the old cruft.
I guess we're getting to the point where people now say things like "it was developed for TSR-80", where it is more correct to say "for the TSR-80." :)
> The uu module provides uuencode format, an old binary encoding format for email from 1980.
while true, uu was also used extensively in usenet, at least throughout 1997. But as we learn later, noone has touched nntp code in 5 years! and there is no interest in porting nntplib to py3.
> originally introduced for Commodore and Amiga. The format is no longer relevant.
not disagreeing. Just saying "ouch."
> The module only supports AU, AIFF, HCOM, VOC, WAV, and other ancient formats.
Again, not disagreeing, but implies .wav is ancient. It's still in heavy use (they're keeping "wave"). And ancient is hyperbolic and slightly insulting, but I'm just going to let it go as humor in service of persuasoin. It's hard to let go sometimes when these formats provided so much joy and delight in such a pure fashion. Particularly AIFF vs WAV.
AIFF was for Amigas and Macs. Lots of great early techno used that format, while clumsy windows machines used .wav and lagged seriously in music software. Life then was more natural and computers were these shiny, productive boxes of creativity. We sat down at them for an hour at a time and expressed wonderful songs. Then got up and lived life. I don't need to mention that we now spend almost all day and all night inside a computer, at its keyboard or tablet, its screen and increasingly its headphones. Instead of augmenting our lives like a microwave, they "shelter" us.
I know the tools are objectively better now. They're more reliable, more productive and have more features. But I disagree that they provide more of what we actually need as human beings. Thinking like a designer, we need nourishing human experiences, challenges and interactions. We need opportunities to use our bodies physically, to run and cook, and especially to work, play and function together. But our health is objectively decreasing. Suicide is on the rise in several demographics. We're spending more time e-socializing so we eat more fast food. Obesity is on the rise. We can't even put the darn things down to drive.
Traditionally, human functioning meant organizing socially in groups that combined strangers' abilities. It required us to have patience and instruct each other. Increasingly, computers/apps/networks give us all the same concentrated superpowers. Now I can find any recipe I want, but I don't have anyone to cook it with. Our interactions are increasingly mediated, robotizing our communication. All this pausing caused by http-to-sql operations after every mouse click or sentence stilts us. It's almost as if our world is more black and white and lower resolution because we're experiencing so much of it through a densely mediated stack.
I'll also point out that in the old days, a library that supported five formats was a flipping miracle. But time marches on.
Having watched and been part of this whole python and internet phenomenon, experiencing it rise from nowhere and evolve from a neato thing into a mechanized, hyper-official "means of production" this is somewhat emotional for me. Guess I'm just getting old and I'll suck it up, but I just want to write these notes for people who wonder about what life was like before, or forget to wonder.
The uu codec is provided by the binascii module. The uu module is a "high-level" interface for conversion between binary and uuencoded files ("-", paths, or file-like objects).
Basically uuencode(1) / uudecode(1) reimplemented in python.
> A programming language that throws away the past is doomed to become a passing fad.
That's complete nonsense.
> If the code is old who cares as long as it still works?
Keeping it working is a maintenance burden. Why keep it if it's worthless?