We are well known for our work with Ruby and Rails here at thoughtbot, but generally we always try to use the most appropriate language or framework for the problem at hand. With that, I’ve recently been exploring machine learning techniques so I’ve been working a lot more in Python.
One of the big differences between working on Ruby projects and Python projects
is the way that dependencies are typically managed. There currently isn’t
anything similar to Bundler or Gemfiles in the Python
universe so usually a Python developer will create a virtual environment
using Virtualenv, and then annotate a
requirements.txt text file with
a list of dependent packages, which they can then install using Pip.
This approach works fine but sometimes it can be a juggling act, as you have to
manually install or remove packages with particular versions, and remember to
regularly update the
requirements.txt file, in order to keep the project
environment consistent. Especially when there are Python packages you want
installed in your virtual environment, but not necessarily associated with the
project itself. Moreover, some projects sometimes maintain two versions of the
requirements.txt file – one for the development environment and one for the
production environment – which can lead to further complications.
Fortunately Kenneth Reitz’s latest tool, Pipenv, serves to simplify the management of dependencies in Python-based projects. It brings together Pip, Pipfile and Virtualenv to provide a straightforward and powerful command line tool.
Begin by using
pip to install Pipenv and its dependencies,
pip install pipenv
Then change directory to the folder containing your Python project and initiate Pipenv,
cd my_project pipenv install
This will create two new files,
Pipfile.lock, in your project
directory, and a new virtual environment for your project if it doesn’t exist
already. If you add the
--three flags to that last command above,
it will initialise your project to use Python 2 or 3, respectively. Otherwise
the default version of Python will be used.
Pipfiles contain information about the dependencies of your project,
and supercede the
requirements.txt file that is typically used in Python
projects. If you’ve initiated Pipenv in a project with an existing
requirements.txt file, you should install all the packages listed in that file
using Pipenv, before removing it from the project.
To install a Python package for your project use the
install keyword. For
pipenv install beautifulsoup4
will install the current version of the Beautiful Soup package. A package
can be removed in a similar way with the
pipenv uninstall beautifulsoup4
The package name, together with its version and a list of its own dependencies,
can be frozen by updating the
Pipfile.lock. This is done using the
There are usually some Python packages that are only required in your
development environment and not in your production environment, such
as unit testing packages. Pipenv will let you keep the two
environments separate using the
--dev flag. For example,
pipenv install --dev nose2
will install nose2, but will also associate it as a package that is only required in your development environment. This is useful because now, if you were to install your project in your production environment with,
the nose2 package won’t be installed by default. However, if another developer
were to clone your project into their own development environment, they could
pipenv install --dev
and install all the dependencies, including the development packages.
In order to activate the virtual environment associated with your Python project
you can simply use the
You can also invoke shell commands in your virtual environment, without
explicitly activating it first, by using the
run keyword. For example,
pipenv run which python
will run the
which python command in your virtual environment, and display the
path where the
python executable, that is associated with your virtual
environment, is located. This feature is a neat way of running your own Python
code in the virtual environment,
pipenv run python my_project.py
If you’re like me and shudder at having to type so much every time you want to run Python, you can always set up an alias in your shell, such as,
alias prp="pipenv run python"
I hope this post has shown you how to manage your Python projects with Pipenv. It has been around for less than a month now, so I, for one, will be interested to see how it develops over time. I certainly don’t want, or expect, it to become exactly like Bundler for Ruby, but I’ll definitely champion it for simplifying the management of dependencies in Python projects. I hope you do too!