Take the 2-minute tour ×

I work with a lot of scientific software on Linux distributions (normally Debian variants and Red Hat variants). In order to maximize performance and get access to the latest features, it's often preferable to compile software from source.

However, doing so is messy, and can lead to conflict with the package manager. I looked at all of the suggestions in the Unix & Linux question "How to compile and install programs from source" and the Unix & Linux question Where should I put software I compile from source? for some guidance. These questions are a good place to start, but it's not clear to me how I can manage dependencies easily. For example, if I build the development branch of NumPy from source using something like checkinstall, how should I detect dependencies so that I can add them correctly into the package built by checkinstall? My goal here is to make installing and uninstalling software relatively painless without cluttering my configuration.

I'd also like to make my configuration repeatable. By that, I mean, I'd like to set up a workflow for installing software on my machine so that I could repeat it exactly on a fresh OS install. Is there a best practice for that? When it comes to standard packages in a package manager, the simplest method for repeating a software install is to make a script containing the appropriate apt-get install <package> (or equivalents). However, when installing software from source, I could see this method getting very complicated. Is there a better way?

share|improve this question
Create rpm/debian packages or look into virtualized environments for your programming environment, e.g. bundler for ruby or virtualenv for python. You then can use e.g. pip to install numpy into your virtualized environment. –  Ulrich Dangel Jun 10 '12 at 23:04

1 Answer 1

The Right Way is to build packages (RPMs/Debs) of that software and only ever install packages. It's the only way which properly integrates with the package manager and handles dependencies correctly. I suggest that you learn how RPMs/Debs are built, and then find existing packaging of that software to base your packages on.

Python modules can be packaged and installed system wide, and generally work with virtualenvs (unless you're using --no-site-packages). Whether this is better than installing directly into the virtualenv with pip is less clear cut though. Packages are more heavy weight, but virtualenv installs tend to be less repeatable because you can be surprised by new versions.

To define the list of packages which make up your system, I would suggest creating a metapackage, which is a package containing no files but has dependencies. This metapackage would depend on all the packages necessary (both packages you've built yourself as well as upstream packages). Installing a new system is then simply a case of installing this metapackage.

share|improve this answer
I like this idea. I think it will work well for most of the software I install. My only concerns are with Python packages, for which the preferred method of installation is using pip and virtualenv. Do you know if there is any way to integrate that type of installation with a package manager as well? Also, if I want to reproduce the installation, should I just put together a gigantic script? –  Geoff Oxberry Jun 11 '12 at 2:50
@GeoffOxberry Updated answer. –  mgorven Jun 11 '12 at 4:06

Your Answer


