Time to Upgrade Your Python: TLS v1.2 Will Soon Be Mandatory | Hacker News Reader

Time to Upgrade Your Python: TLS v1.2 Will Soon Be Mandatory

pyfound.blogspot.com

155 pointstrurl4252 comments

Test procedure that doesn't require 3rd-party libs:

* For Python 2:

  $ python -c "import json, urllib2; print json.load(urllib2.urlopen('https://www.howsmyssl.com/a/check'))['tls_version']"

* For Python 3:

  $ python3 -c "import json, urllib.request; print(json.loads(urllib.request.urlopen('https://www.howsmyssl.com/a/check').read().decode('UTF-8'))['tls_version'])"

I get TLS 1.2 for both Python2 and Python3. Latest Arch Linux.

Same on CentOS 7.3.

Same on Windows 8.1 with python 2.

I don't know if it matters to anyone, but stock OS X (I develop on an Ubuntu partition) shows TLS 1.0.

As it says in the article, Apple has not updated their system Python in a long time, so soon you will not be able to Pip install packages using system python.

The real issue is not that Apple has not updated Python, but rather that about 4 years ago Apple depreciated OpenSSL. So the OpenSSL library included on MacOS has not seen a major update since then (only back-ported fixes). The article does an o.k. job of explaining this in note 2, but fails to mention that this has been a known problem for over 4 years, and little has been done in the mean time. Only now that there is a looming deadline does there seem to be any motion on it, and it is obvious that they are going to miss the deadline.

I do think that they are missing a fairly simple solution for the majority of use cases: move things like `urllib2` to use `libcurl` underneath rather than OpenSSL. The CURL project has already solved this problem by being portable to things like Apple's CommonCrypto library and Window's security libraries.

I should also note that the article's suggested workarounds are less then ideal as both use versions of OpenSSL that do not consult the system keychain for root certificates, and so create strange problems that most people will not understand.

I do think that they are missing a fairly simple solution

That doesn't sound very simple at all. For one thing, it means the python runtime loses control over the http implementation it ships. It's also not obvious that libcurl provides an API that can do everything something like http.client can do. I think this is somewhere between 'not simple' and 'non-starter'.

Does libcurl provide the abstractions to make that transition simpler than directly adding support for the platform libraries to Python's TLS implementation?

I can't say for certain regarding this case specifically, but CURLOPT_CONNECT_ONLY (as I recall) does the TLS and proxy shenanigans, and then waits for you to say "HTTP 1.1" and so on. You can send and receive data using a BSD_socket-a-like API.

https://curl.haxx.se/libcurl/c/CURLOPT_CONNECT_ONLY.html

Rewriting all the network code to use libcurl is not gonna happen in such a short timeframe.

I expect the issue will be solved by linking and bundling a recent openssl, which is relatively trivial to do for only one platform.

That's already the solution. It's suboptimal.

I like how you phrase it so that it sounds like it's Python's fault that Apple is not supporting an open, cross-platform standard.

Two relevant quotes regarding Macs:

"the system Python shipped with MacOS does not yet support TLSv1.2 in any MacOS version;" (and notes on installing current Python 2.7 or 3.6 alongside the system version)

"Python's TLS implementation is falling behind on macOS is that Python continues to use OpenSSL, which Apple has stopped updating on macOS." It's unclear what the fix for this is going to end up being.

> It's unclear what the fix for this is going to end up being.

Is there some reason why `brew install python` doesn't work? (I do this on every system, and mine is TLS 1.2)

Otherwise, if you're bundling stuff to give to an end user, bundle up Python, since Apple seems intent on letting most of the core utils (not just Python) bitrot?

I get TLS 1.2 with Python 2.7.13 installed via homebrew.

Yes, the solution is to use a package manager that builds Python with an up-to-date version of OpenSSL. The long term solution is to patch Python to use SecureTransport, but I don't see anyone rushing to do that.

I think I'd rather use a third party library than a third party service though, if that was what I could pick between. I see that you just modified the code mentioned in the article so it's not criticism towards you.

Over the past two years I've been working on a cross-platform crypto library for Python. One of the aspects of it is that it doesn't require a compiler (it uses ctypes or cffi), and works with the cryptography libraries shipped via your OS.

Part of this library is a TLS implementation that uses SecureTransport on OS X. If someone is inclined to get requests (and pip, etc) running on the system Python on OS X, it should be possible with the guidance outlined at https://github.com/wbond/oscrypto/issues/10. Unfortunately, it isn't something I have time to work on right now.

I like better security but let's be honest it's a smokescreen. Python can migrate to a non-PCI CDN if _really_ needed. It is more an excuse (and a good one, I like how it sounds if I will have to tell it to my boss – we use Python 3 anyway, so it's unlikely).

I was also surprised not to find this on the page. As I understand, it will vary not only across Python versions but also on systems where it's installed. A lot of fun is about to come, I think (especially from RHEL 6 where people are stuck with Python 2.6).

I have no particular knowledge about this, but I just tried with a python2.6 (from one of Amazon's EC2 images) and it seemed to work:

    $ /usr/bin/python26
    Python 2.6.9 (unknown, Dec 17 2015, 01:08:55)
    [GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import json, urllib2
    >>> json.load(urllib2.urlopen('https://www.howsmyssl.com/a/check'))['tls_version']
    u'TLS 1.2'

Uh, but it does have a "bad" rating for other reasons (looks like it supports some insecure cypher suites).

Actually, you are right (but should have been tested on CentOS 6&7):

    Linux localhost.localdomain 2.6.32-642.11.1.el6.x86_64 #1 SMP Fri Nov 18 19:25:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
    Python 2.6.6
    TLS 1.2

    Linux localhost.localdomain 3.10.0-514.2.2.el7.x86_64 #1 SMP Tue Dec 6 23:06:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
    Python 2.7.5
    TLS 1.2

Tested on RHEL 7 (actual RHEL, not CentOS), got TLS 1.2 for both:

    $ python2 --version
    Python 2.7.5
    $ python2 -c "import json, urllib2; print json.load(urllib2.urlopen('https://www.howsmyssl.com/a/check'))['tls_version']"
    TLS 1.2

    $ python3 --version
    Python 3.4.5
    $ python3 -c "import json, urllib.request; print(json.loads(urllib.request.urlopen('https://www.howsmyssl.com/a/check').read().decode('UTF-8'))['tls_version'])"
    TLS 1.2

fastly is offering their services to the python community for free.

And large parts of the Python community are offering their software for free. Without that software, there would be no need for a CDN or the PSF with all its "directors".

The software on the other hand would do quite well without a lot of organizations that do peripheral work and take the credit.

I don't understand why fastly would have to turn off TLS 1.1 and older for everyone. The other CDNs out there are PCI compliant and still offer TLS 1.0 and even SSLv3 about a year ago.

The most interesting bit about this is the mention that PCI compliance now requires this. As much as PCI seems like security theater, that change in particular may help move people forward.

On macOS, Homebrew's Python links against it's own openssl which is currently @1.1, so that is the easiest way to avoid this problem (unless one is a homebrew hater).

I am running a Python 2.7.13 installed with official Python installer on macOS Sierra (10.12.3). Besides the built-in openssl (0.9.8zh) which came along with the macOS (/usr/bin/openssl), I have a newer version (1.0.2j) installed with macports (/opt/local/bin/openssl).

However, even though I have configured my path variables that it will use the macports openssl-files, the Python installer seems to link by default to the /usr/... openssl-files.

I've tried quite a bit re-installing Python (with the installer and building it from source; the latter failed with some obscure error I wasn't able to resolve), so I was wondering whether there are any better options. Upgrading via homebrew seems simple enough, but due to using macports, I'd rather not use homebrew. Furthermore, I am a bit reluctant to use the macports-python, but might do that if you were to suggest me that this is the only viable solution.

I'd be happy to hear your suggestions on this!

The python.org official installers are all linked against the system OpenSSL. You need to build from source to avoid that. I recommend getting Python from macports in this case, as they should allow you to link against their provided OpenSSL.

The 3.6 macOS installer is bundled with its own, recent version of OpenSSL and supports TLS 1.2.

Thanks for your replies! It's as I feared then. The switch to macports luckily isn't all that painful.

The macports Python works fine, if you are able to use Python 3 you can also just use the official installer instead.

There is so much legacy code written in Python 2 that I can't imagine someone isn't going to have a large enough need that they will backport TLS support. Am I missing something on why that wouldn't work?

The value proposition might be there if the alternative was a python 3 port, but if you're installing packages of pypi these days it's unlikely you're running python <2.6 and I'm not sure a Python 2.6 -> 2.7.9 upgrade is of comparable difficulty to adding TLS 1.2 support to python < 2.7.9.

I mean, I know there's some old versions of Red Hat and co knocking about with 2.4 but you're probably better off to use your system package manager for those these days.

In RHEL those old Python interpreters are available precisely via the system package manager. And they won't support you if you install Python from a 3rd party RPM as I understand.

And people pay money for that?

Yes. There are people who value support for a fixed version for 10 or more years much more than they'd value "rolling" support for a "latest" version that they have to keep updating their apps for.

But none of the packages they use will be up-to-date. e.g. Django dropped support for 2.6 years ago and will drop support for 2.7 soon.

People who use RHEL this way do not keep any third-party packages up-to-date. If that means a decade on Django 1.2, well, they spend a decade on Django 1.2 -- Red Hat will backport security fixes into a Red-Hat-packaged Django 1.2.

Absolutely. Even on the gratis side, Debian stable gets very, very cranky if you mix in anything new. Devs hate it; ops love it.

It's not a 2.7 vs 3.x issue. There are minor versions of both that have TLS 1.2 support. The issue is making sure that the minor version installed on a system is linked against a library that supports TLS 1.2.

The problem isn't Python, it's OpenSSL.

OpenSSL got TLSv1.2 support in version 1.0.1. Any older version of OpenSSL doesn't support TLSv1.2. That affects a number of platforms:

- the system Python on macOS, which link against the system OpenSSL (an anaemic 0.9.8zh)

- most of the older (pre-3.6) python.org releases on macOS, which have the same problem

- any Python on a Linux operating system with an older OpenSSL

So the backport needed is to backport a newer OpenSSL to the platform.

the article points to http://docs.python-guide.org/en/latest/starting/install/osx/ which tells you to "You do not need to install or configure anything else to use Python." However, it suggest to install python via homebrew. Just for reference, if you install it with `brew install python` then you may also need to do a `brew link --overwrite python` then it works.

Be nice if pypi stopped using MD5 for integrity checking as well.

what is the minimum version then? 2.7.x <- what x?

Likely 9, the Python 3 SSL improvements were backported to that version

No, the OpenSSL version is the only relevant factor here.

It's OpenSSL version that matters.

I'm getting TLS 1.2 using anaconda on the latest update of Mac Sierra (10.12.3). The openssl is one installed by the conda command of anaconda, openssl version: 1.0.2j-0

I also have homebrew on the system, but not used in this case. Is there any reason not to use anaconda on Mac?

FYI conda update openssl will give you the latest 1.0.2k. Continuum does a good job at getting OpenSSL updates out within hours of their upstream release.

There's no explicit reason not to use anaconda if you're okay with being vendor locked to anaconda's interpreter/environment. Anaconda is primarily developed with the data scientist in mind over the software developer. And the ecosystem Continuum Analytics has built with Anaconda reflects that design/vision.

Reply on news.ycombinator.com