Hacker News Reader

I have customers that use RHEL 5 still (supported until 2017 or 2020 in the extreme), which ships with python 2.4 as the system python[1].

My solution was to use pyinstaller[2] to basically ship my own python, first 2.7, and now 3.5.

[0] https://access.redhat.com/support/policy/updates/errata/#Lif...

[1] http://distrowatch.com/table.php?distribution=redhat

[2] http://www.pyinstaller.org

Pretty much this. Also US Govt and especially the millitary. When they buy "Linux" they buy RHEL. (Sometimes SUSE when they are working with Lockheed). Mostly because of support and RHEL5 and RHEL6 are certified with an alphabet soup of certifications (EAL4, FIPS-140-2, etc) and so that is what is used. Heck even RHEL7 doesn't have all those rubber stamp even though it is 2 years old almost now.

I don't do a lot of python anymore but we never depended on installed version. Everything was in an a virtual environment where we could control the libraries.

virtualenv doesn't manage different versions of the Python interpreter though. By default it uses the system Python. If you want another Python, then you need to install it yourself (somehow) and then point virtualenv at it. virtualenv is just a way to create separate bundles of libraries.

It's for both libraries and different versions of Python. This is common and trivial. I install specific versions of Python using the Deadsnakes PPA since I can easily pull in any version going back to 2.3 if I need. Or you can compile and install from source.

Then create a folder for your project, 'virtualenv -p /usr/bin/python3.5 venv' to create the venv in that directory. Activate/deactivate the environment as you wish when you want go between the system Python environment and the one in your folder.

I replied to @mrits because @mrits was presenting virtualenv as replacement for using pyinstaller to manage the installation of non-OS-managed Python versions (which @cowsandmilk was talking about).

I'm unsure how what I said:

> If you want another Python, then you need to install it yourself (somehow) and then point virtualenv at it.

Differs from what you are saying:

> 'virtualenv -p /usr/bin/python3.5 venv'

You can include the python interpreter inside the virtualenv. It's not entirely painfree (python and any pip packages are normally dynamically linked), but it works reasonably enough, especially when deploying it on the same OS version.

By default virtualenv links to the Python interpreter that you point it at (when you create your virtualenv). I'm unsure what method you would use to include the Python interpreter install in the virtualenv directory (other than creating a normal virtualenv, and then copying the files in after the fact).

virtualenv --always-copy

I sometimes wonder if it's possible to use a gobo-linux style system, whereby the package manager just downloads the package to a versioned folder, possibly expanding it out, to a a package cache - then the venv need only symlink to it. The same could be done for python installs. Then do the same for the system python (e.g /bin/python points to /someplace/pyjunk/system/python, which in turn is a symlink to a particular interpreter, e.g. /someplace/pyjunk/interpreters/py2/2.7/cython_2.7_blub-i386/python).

I never understood the fear of leaving the system's package manager's packages. Maybe I've been on Slackware too long, but compiling/installing packages from source is the norm. Also, package managers tend to ship with --enable-everything-everywhere-no-matter-what configure options. Compiling from source you can say yes, support libpng, no don't care about libjpeg etc and get a much leaner setup.

I know there's the argument that by straying you don't get security updates, but I can't really imagine Python 2.4 getting much security updates nowadays.

On the other hand, occasionally security updates will be surfaced as a patch file and you're waiting on the package managers. At least if you build it yourself you have the option of applying patches.

This is just an argument against bad package managers. Good ones will offer binary packages by default, but allow you to tweak (or just pass ./configure options to) an installation formula, at which point they'll switch over to pulling in the source package, applying your changes to it, building it, and installing the result, while still offering you full package-level management of the result (including having the package satisfy deps correctly.)

I never understood the fear of leaving the system's package manager's packages.

It's trivial when you have a handful of machines, all under your control. It's somewhat less trivial when you are dealing with 1000s of machines under the control of dozens of different people/departments.

Hi, I'm the Founder at Distelli and I wrote the original version of the Distelli agent in Python that Brian rebuilt using Lua. The main reason I picked Python 2.4 is because we have large enterprise customers that are / were running Python 2.4 on servers and I had to support Python 2.4 to serve their use case.

They were probably running RHEL5, but even in 2014 they would have been in the Production 3 phase which is really should only be used for legacy purposes, and that's definitely not the sort of system you push new software to. Red Hat even pushes those customers towards virtualization because a) hardware rarely lasts 7 years and b) they won't bother backporting drivers for new hardware to the old kernel at that point.

I still use a few RHEL5 machines but nobody would ever dream of pushing new software to them.

In the real world...if you have a large fleet of RHEL5 boxes/VM's, this a fact of life. We're slowly phasing out our CentOS 5 boxes, but you still need to be able to run new software on them.

>not the sort of system you push new software to.

Telling customers that they are doing things wrong isn't really a great way to win customers.

This is something I don't understand. If someone is using Java, they'll install the version of Java they need. If they use Ruby, they'll install the version of Ruby they need. If they need Lua, they'll do the same thing.

Why everyone is relying on python that's installed there for RedHat's needs (e.g. yum).

Unlike other languages, Python is probably the best one that can have multiple versions coexist together.

As a consultant I have very different experiences. Even in Java development you find shops that refuse to upgrade their JVM for various reasons.

RHEL 5 is really common.

A lot (all that I'm familiar with) of the enterprise software packages that I've worked with ship with their own, very specific (read: regression tested against) version of java. Enterprise software usually requires a ton of servers, so it's not like anything else on those servers will be using the jvm.

> Why everyone is relying on python that's installed there for RedHat's needs

Environments where the build of your server is strictly controlled and conservative...banks, mission critical apps, that kinda thing where stability from upstream is hugely important.

AFAICT if you want support from Red Hat, you better not mess with the system configuration. It's probably best not to mess with too much on your production boxes anyway (maybe that's why Docker is so popular)

How about running a (newer) local version of Python, without of course touching the system Python?

To my mind, it's usually a good idea not to use system Python, both packages and the interpreter, if you're not also controlling them. This makes building your deployables more involved, but makes the actual deployment more painless.

> I had to support Python 2.4... large enterprise customers that are / were running Python 2.4 on servers...

How did the switch to Lua not break this in the same way?

Sorry for the naive question, I'm just not following how it can be possible to change languages, but impossible to change versions. At first glance, the latter seems like a special case of the former.

Shipping it as a single download with zero dependencies meant that we would not have to worry in the future about what else was installed on the machine and the versions of system packaged the customers were running.

Couldn't you just bundle Python instead?

Lua and Luajit are small embeddable languages. They link into a single executable, including if you want the program they'll run.

That's fair enough, but it sounds like you've constrained yourselves and your code for the sake of a few enterprise customers, and now you've had to clear your technical debt with a complete rewrite. Even if batbomb were right I would argue that any solution is better than supporting 2.4. The very minimum you would want is 2.5 at a stretch.

In that case, Lua makes a lot of sense, frankly. I can write 100 lines of code in perl, but trying to get that deployed on mac, windows and linux can be a nightmare, especially if its using a library external to perl.

(EDIT: LUA->Lua)

LUA? Still?

Lua used to be LUA, you'll find LUA written in old source code comments :) Don't be so mad about the Lua Uppercase Accident :)

Deleted comment

I think he referring to LUA vs Lua

I'm a Python devotee, but I appreciate the anecdote. I'm curious whether the switch 1) cost any features/algorithms and 2) whether the anticipated performance improvement really panned out since the switch, or if the gain wound up being simply architecture/deployment-related. Stepping back, did the whole re-write really balance out with all the work that had to be done, or could you have stuck with Python and simply solved the problems at hand?

1) All features/algorithms were ported, and we only added new functionality that never existed in the old Python code base. A few of the features that got added include: dispatching builds, getting live logs, re-encrypt keys, deploy encrypted artifacts, fix many bugs, support running multiple agents on a single computer, etc. I should also mention that we previously had two separate downloads: a CLI used for uploading releases, and the agent used for dispatching deployments. The functionality of both of these code bases where merged into our new Lua based "client" so customers only have to install a single binary that is less than 10MB (the binary size depends on your platform).

2) Deployments with the new agent were noticeably faster, (like 1 second deployments that used to take 20+ seconds). However, this speed up was largely due to the Python codes design which had a "continuation" loop which was polling based, but the Lua code used coroutines and simply continued the threads when the steps were complete.

Overall, I think the choice to use Lua (or more specifically luvi) was a great decision. Note that it did take a few iterations in order to come up with the optimal way of using Lua + libuv. Originally I was taking the Node.JS approach of using callbacks, but that approach has two problems:

[a] It is difficult to properly implement "back pressure" so that all queues between producer and consumer are bounded.

[b] It is difficult to handle errors properly.

Lua coroutines allowed me to write a simple "green thread" library that encapsulated these challenges.

Couldn't you have run two versions of Python at the same time on their systems? Would it have been possible to ship a Python version with your software?

easily

"I had never seriously used Python before joining Distelli, so there was a bit of a learning curve"

That was your problem, you just made a huge mistake and depending on how big your python code base was, you have now alienated most of your developers by moving to a language they are not already comfortable with.

Do your customers already have lua installed on their servers or are you bundling it now? You could have just a easily bundled a newer python interpreter. Lua is a nice scripting language, if you want to embed it into another app or to create a plugin system. It's not a nice general purpose scripting language, mostly because of the vast pypi repo.

How is this alienating anyone? This is a command line tool. The end users are suddenly using a different language as a result. Their install process was streamlined and system dependencies were removed by including it in the install package.

The same thing could have been achieved within Python, but from an end user perspective I'm guessing this is an improvement for most of their customers.

It's alienating Distelli developers. The reason cited for the move is totally unacceptable in terms of the new skillset that their developers will need to acquire to continue development of the tool.

The cost of effort their developers will have to put in to be as effective as a LUA developer as they were as a python developer is really not worth this move. There are more to moving to a different platform/language than just the end users.

They only have a few developers, so it's really not a big deal. I think they only have 2 or 3, including the CEO who, obviously, has a lot of other things to do than write code.

Being as the size of the team is so dependent on a small number of high performers, I don't think it's unusual that they picked a language one of the main people in the company is extremely fluent in.

I've worked with Brian before and he is a beast. I don't know if 10x engineers exist, but if they do I would certainly consider him one.

Describing him as a 10x developer makes me even more skeptical of the decision to switch. Developers who deliver 10x as many features inherently do so at the cost of maintainability.

> Developers who deliver 10x as many features inherently do so at the cost of maintainability

I don't think what you say is completely true. People who are more experienced tend to be able to produce features much more quickly than those who are junior, just because they've already run into issues before so can anticipate them or solve them more quickly. If anything I would argue people like that actually write code more quickly that is more maintainable.

I don't know if he can ship 10x as many features; I didn't think term 10x engineer was used literally. I just know I've worked with quite a number of people and he is one of the best developers I know.

> he is one of the best developers I know

But then, we'd need to know how good your assessment skill/method is. Maybe you misjudge all the developers you work with? What feedback do you get?

@rifung - Thanks for the kind words.

@Chris2048 - Distelli is hiring. If you want to make your own assessment of my skill, then join us :).

The end users are the ones that keep the lights on though. It sounds a lot like that Python was more and more of an anchor around the team, and the costs of migration were smaller than the costs of the other options. Yes, some developers were probably upset that they couldn't use their preferred language anymore. However, it sounds like because they're now using a language designed to be embedded, they can focus a lot more on solving problems than fighting with a language that has a lot more platform-specific behavior.

"It's alienating Distelli developers."

Are you in a position to speak for Distelli developers?

"The cost of effort their developers will have to put in to be as effective as a LUA developer as they were as a python developer is really not worth this move."

I suppose this could be true in a poisonous environment where scheduling does not accommodate the need to learn a new skillset.

Unless you are familiar with the employer this is an incredibly hostile presumption. I.E. I would not work for such an employer nor would I want to be their customer

What does accommodating for a new skillset make a difference to the cost?

There will still be a minimum effort (e.g. in hours) that cannot be reduced by the nature of the employer. A dev will still have to put in the hours to learn a new language.

It seems the cost being described here is cost to the employer, not to the developer.

I use Lua as a general use language and I'm incredibly satisfied. It's an amazing language.

I also need to mention that Lua is the easiest language I've ever encountered, so any competent developer can learn it quickly, it's not alienating anyone.

Given that, I agree that LuaRocks isn't so vast as other languages' repositories and this can be a drawback of using Lua. But why again are you judging a language by it's package manager? If it's a good language wouldn't it be better to suggest more people to write packages instead of suggesting people not to use it?

> "the current code is not optimal, let's re-write it in X"

I read it more as "relying on a runtime that must be installed on the user's OS instead of included in our executable is a nightmare". The dramatically reduced conceptual footprint of the language is a nice bonus though.

I'd guess RHEL 5 is to blame for supporting 2.4 until 2014.

Honestly, every time I hear a story like the one in the OP, I'm tempted to find a way to make once-per-decade upgrades be so ruinously expensive that companies will stop opting for agreements structured around them.

Sure, maybe there would be a few "smart" companies that do a proper cost-benefit analysis and decide to budget for a faster upgrade cycle or that bigger chunk of change for decade upgrades... but most companies would file it as a problem "for somebody else" in a decade as most companies don't even seem to think past the next quarterly earnings report.

Or how about finding out exactly why companies do once-per-decade upgrades instead of something more timely. Its not about being lazy.

My experience of this is there are a few categories:

* Some companies have absolutely no in-house tech staff to speak of, and so they want to invest in technology exactly once and never have to deal with it again.

* Some companies have in-house tech people, but are entirely marketing/sales driven with little or no input from the tech staff. So they focus exclusively on writing new features and never clean up technical debt.

* Some companies choose a platform to bet the farm on, develop a huge amount of in-house code for it, and then realize that since they wrote it for specifically the exact version of the platform they first installed, any upgrade involves a complete refactor/rewrite, which they then dismiss as too expensive.

The first case can be solved somewhat by subscription-model managed services where the upgrades just happen at the discretion of whoever's managing the platform.

The other two are failures of management and planning, and absolutely should be laid to rest at the feet of the management staff involved. Preferably in an expensive way, pour encourager les autres.

Or the new version doesn't offer a compelling reason to upgrade, and would require extensive regression testing, potential downtime, and potential lost money for little to no benefit to the end user. Upgrading core software costs money, upgrading core software just because a new version comes out costs more often. Sure one should test to see if there is a compelling reason, but if there's not, why should one mess with something that already works well?

Refusing to "mess with something that already works" is a recipe for disaster.

What companies often learn too late is that there are basically two options:

1. Make maintenance -- both in terms of upgrading underlying software, and addressing accumulated technical debt -- a regular, required part of their process, or

2. Occasionally face a potential existential threat from the unbounded cost of staying fixed in perpetuity to a stack that "already worked" once upon a time but no longer does due to lack of upstream support and available developers to keep it limping along.

The genuine problem is that "what does this do for next quarter's results" is the only concern, which means long-term thinking is actively discouraged.

The problem is that is something the company will figure out at least 5 years after the decision has been made.

To extend this, CentOS 7 has no simple python3 package available (I'm aware of scl). This will be a massive hassle on the way forward.

sudo yum -y install epel-release && sudo yum -y install python34

python35 will be soon be coming to EPEL also.

Installing from source is really not that terrible - we do so for one of our clients (running CentOS 6.5, IIRC) and it just took us a couple of hours to have a full build script.

What's wrong with SCL?

I thought Python 2.4 was a RHEL4 thing...

RHEL4 = Python 2.4 RHEL5 = Python 2.5 RHEL6 = Python 2.6 RHEL7 = Python 2.7

The article did mention a desire to use netifaces, but I didn't quite understand why the author thought they couldn't.

uuid.get_node(): ...and which iface's address will I get if I'm on a multi-interface machine (not to mention that it was added in 2.5 version of Python)?

netifaces: ...would require me to build a C extension which would mean going down the rabbit hole of building and bundling python and this library.

As opposed to shipping LUA with C extensions? Also, there are things like shedskin: https://github.com/shedskin/shedskin - you don't even have to ship python. Or micropython: https://github.com/micropython/micropython with the code embedded. Then you can easily either shell out or include any C extensions you want.

I just tried it (it was uuid.getnode(), without underscore) and I got the interface with default gateway set.

> Yes, in 2014 you should be supporting Python 3, or at least have it on your roadmap.

Why? What's wrong with 2.7?

Nothings wrong with it (except the obvious things that are, but that's another discussion). You can support 2 and 3 in the same codebase.

What's not wrong with it?

Was it by design your first 3 points (of 5) are the same?