Python Development on Linux and Why You Should Too | Hacker News Reader

Python Development on Linux and Why You Should Too

thelinuxexperiment.com

80 pointsjeffreyfox33 comments

One thing I'd like to add/contradict. For system-wide Python packages, avoid pip, use your system package manager, else you will eventually end up with conflicts when something you install requires the library you have pip installed, and your package manager wants to install it. Install virtualenv and virtualenvwrapper with your system package manager! Then use virtualenv for any actual projects you are working on. If your repository version of virtualenv/virtualenvwrapper is outdated, use better repos/PPAs/AUR.

Basically, you should never ever find yourself typing sudo pip. If you refuse to use a virtualenv for some reason, use pip --user.

The other benefit of this approach is that your system package manager will keep your system packages up to date and the required versions for any software on your system.

Honestly, I think the best solution in the long term would be for pip, by default, to install to a user environment (installing in ~/.local using the --user flag) if it is run unprivileged and not in a virtualenv.

My workflow (which has never led to conflicts):

- Never run pip outside of a virtualenv. I prefer to use virtualenv directly so that I have everything contained in my working directory, but virtualenvwrapper is fine too.

- Never install system-wide Python packages directly. Okay, I'm exaggerating, but I don't think I've ever needed a system-wide Python package as anything more than a dependency - and in the rare case when I have, the AUR (in my case) makes it easy to wrap a setup.py file inside a 'system' package.

The only problem with this workflow is that I still haven't figured out how to change the IPython module path, so when I run IPython inside a working directory with a virtualenv activated, I access the system packages, not the locally-installed ones. I could pip install IPython, but that'd add a completely unnecessary dependency to every project. Does anybody know an easy way to switch the path for IPython via command line switch, etc.?

I have a "pip install ipython" in ~/.virtualenvs/postmkvirtualenv to make that a bit easier.

My major problem I have is packages using cython or depending on external C header files in some other way. For example, matplotlib and the IPython qtconsole.

For one, I don't even have gcc on servers that I want to deploy to using a virtualenv. but the major pain is finding out what *-dev packages I need to have installed in order to install using pip (if it can be done at all).

I don't think I ever had the IPython qtconsole working from a virtualenv without using system site packages.

If you install ipython within your virtualenv it seems to work.

Well yes, but as I said, that's exactly what I don't want to do, because then it'll be included in my requirements.txt file when I run pip freeze (unless I manually delete it each time).

A quote from the article: "Install this bad boy with the simple command sudo apt-get install python ..."

Two things:

1. Fedora and related distribution users will want to say "yum -y install python" instead.

2. But it's hard to imagine a distribution that doesn't have Python installed by default, because so many Linux system utilities are written in Python.

Another quote: "Alternatively, you can just start hacking away and use StackOverflow to fill in any of the gaps in your knowledge."

Yes, good suggestion, but there's always the searchable online Python documentation:

http://www.python.org/doc/

And if you're offline for any amount of time, the documentation can be downloaded:

http://docs.python.org/download.html (2.x)

http://docs.python.org/py3k/download.html (3.x)

Python has some amazing libraries -- even symbolic algebra (what Mathematica does):

    from sympy import *
    x,a,b,c = var('x a b c')

    pprint(solve(a * x**2 + b * x + c,x), use_unicode=False)

The result:

             _____________   /       _____________\
            /           2    |      /           2 |
     -b + \/  -4*a*c + b    -\b + \/  -4*a*c + b  /
    [---------------------, -----------------------]
              2*a                     2*a

The Unicode output is, of course, much nicer:

http://i.imgur.com/fR7qO.png

You can actually paste the Unicode output here:

  ⎡        ⎽⎽⎽⎽⎽⎽⎽⎽⎽⎽⎽⎽⎽   ⎛       ⎽⎽⎽⎽⎽⎽⎽⎽⎽⎽⎽⎽⎽⎞⎤
  ⎢       ╱           2    ⎜      ╱           2 ⎟⎥
  ⎢-b + ╲╱  -4⋅a⋅c + b    -⎝b + ╲╱  -4⋅a⋅c + b  ⎠⎥
  ⎢─────────────────────, ───────────────────────⎥
  ⎣         2⋅a                     2⋅a          ⎦

Well... http://i.imgur.com/Rxh63.png

That's interesting, it shows up perfectly for me in the comment (latest Firefox, Ubuntu 12.04).

Not on the iPad :(

Your browser's monospace font isn't monospace. You should choose a different one.

If the monospace font doesn't support those characters, maybe the browser fell back to a non-monospace font.

No, if you look at the ASCII version posted above, it's distorted too (the big parenthesis are wrong, for example).

Yes, I see the problem -- I assumed that other browsers were using a monospace font for the HN monospace-markup sections. This isn't always the case. Even my rendering in Google Chrome on Linux isn't perfect.

You can, but most people will see little boxes all over the place.

True, but in my opinion they should install fonts with decent unicode support anyway.

And how would one go about doing that on an iPhone?

And Sympy's output in the IPython notebook is even nicer: http://i.imgur.com/ldfLd.png

To enable that, run:

  %load_ext sympyprinting

holy shit, did not know about this. This was really the one thing from Mathematica that I was still missing in iPython. Thank you!

Though your comment was not in reply to me, thanks for the above info. I knew about IPython but not about the IPython Notebook or the QtConsole. I just googled for IPython notebook and read some of that info, but can you give an example or two of what that style of working is useful for? Is it good for exploratory stuff?

I run my Python dev environment on a virtualized ubuntu VM that I control with Vagrant. After spending hours trying to get Homebrew to install Postgresql, someone at a Django user group suggested this methodology. I haven't looked back. It also comes with the added bonus that I can move my virutalized Ubuntu python dev stack around to other machines/platforms, too. And, as a bonus, you eventually are likely going to deploy to Linux anyway,...

I'm a .NET developer by trade with some very brief experience writing Java applications in Linux. My only real experiences with Linux is from my degree as since then I've been dedicated to .NET.

Something I've wanted to do for a very long time is pick up Linux and to be able to use it as well as your typical developer that uses Linux from day-to-day. For this reason, I've decided to pick up Python and use it on my new Ubuntu desktop (dual-boot on top of Windows 7) along with the online version of Learn Python the hard way.

So far, it's been a real struggle to get things working the way I'd like. Writing Python on Windows 7 has been a surprisingly painless experience so far, but getting something like Django set up took me hours. Perhaps I just suck with Linux, who knows?

Outside of the pain of setting things up things just seem to work well and I'm glad that I decided to take the plunge in Linux over Windows. Python might not ever pay the bills, but I can apply myself in a different way and perhaps make myself employable in the future outside of .NET or Java. I love "The Unix Way" of doing things and hopefully it'll eventually click.

I went through the same thing. My experience showed that the difficulty rose from the two very different work flows. Windows is mosly a point and click deal. Even the .NET IDE of choice (VS) is less about coding and more about hitting the space bar. I'm not critizing it, because it is very good and helps save time. Though it did make me lazier, which I thought would be impossible. On the other hand, Linux/python/any language really, is a bit more raw. You have to do a lot of things yourself. There is little hand holding (though it has been improved over the last years). It pushed me to really dig into the OS and learn how it works, instead of just sitting there clicking. It has also helped as an engineer, because I know write code that is more modular, instead of platform specific. And by modular I dont mean OOP abstractions, but it the end product being able to be used by different OS versions.

I suggest you just jump right into it. Being well versed in Linux doesn't mean you will not hit walls. You will. I guarantee it. It just means you will understand why did you hit the wall and how to jump over it. I suggest you create a VM with Ubuntu running on it and just install stuff without a fear of breaking it. One of the most fun (and frustrating) experiences I had was with learning how to use chmod. I once changed almost every file on the OS to file settings 777 (writeable, readable by anyone).

Just keep hacking at it. It has taken me at least 6 years to reach this point.

Point-and-click vs. the command line isn't just about hand holding. Setting things up with the command line might seem less convenient, but it's very easy to make a script when you want to automate that setup routine.

I'm a little surprised that the grandparent post had trouble setting up Django, though. Once you've installed the python-django package, you should be able to follow the Django tutorial straight away.

Setting things up with the command line might seem less convenient, but it's very easy to make a script when you want to automate that setup routine.

This comes off to me as a major disconnect. It almost looks like you're saying making a script to automate a setup is as easy as a point-and-click interface. I would say setting up the script is less convenient, in that you need to learn basic shell scripting first. Point and click requires you to learn basically nothing. Using a CLI rather than a mouse requires worlds more brain power and is worlds less intuitive when you're coming from a Windows world.

Ok, now repeat your setup procedure on 50 machines.

Writing scripts is a medium-long term investment. You're right that for one off tasks it's quicker to point and click, but once you need to repeat that task, scripts are faster.

I agree that it's much less intuitive, but it's worth explaining why we put up with that. It's not just because Linux isn't as polished as Windows - shell scripts are a feature.

A case in point: I was setting up automated tests on a Windows machine today, and I had to install pywin32 automatically in a virtualenv. Both the download page and the installer are built for a human to click through, and are problematic to do from a script.

When you want to document a process, it is easier to copy paste a couple of commands than to take a dozen of screendumps with check boxes.

> but getting something like Django set up took me hours. Perhaps I just suck with Linux, who knows?

That's a Django thing; I remember its setup/configuration to be a bit wonky compared to most Python packages.

Just learn how to use virtualenv + pip, and that's most of the setup you should need for Python development on Ubuntu. The added plus about using virtualenv from the start (I don't think it was around when I was learning Python) is that you get a very clear understanding of what the project structure and dependencies are... it took me a while to figure that out when I first started.

Also:

> Perhaps I just suck with Linux, who knows?

Nah, that's just the process of migrating for the first time from a completely different OS. You'll get the hang of it the more you use it.

Very strange. It completely contradicts with my experience. Almost everything that easy to do in Linux is someway distorted or missing completely in Windows.

Basic things: apache, nginx, supervisord - underpowered or completely missing. Ipython is my favorite shell in Windows, but in Linux it works better (just because console is better in Linux).

Have you tried to run Django under IIS? I assure you - it's pain.

Great outline. I've been learning Python on and off for the past few months. After getting comfortable with easy_install, pip and virtualenv things improved dramatically. However, I found it very difficult to get a dev environment running for MySQL development. It took quite a bit of Googling to figure out which libraries needed to be installed on the Mac. Pip worked very well for everything else.

I went thought this pain with Postgresql, trying to get it working on the Mac. I went with the Vagrant/VM route and now run a linux-based dev environment inside of OS X. It also has the benefit of not polluting my host OS. And has prepared me for actual Linux deployment. I can mount the VM web-directory from my host OS so I can continue using some of my favorite tools.

I really don't understand this obsession with git. I've been heavily using both git and mercurial, and git keeps getting in my way no matter how much more I learn about using it. Whenever I have an option I choose mercurial, as it simply makes much more sense, to me at least.

Reply on news.ycombinator.com