Manage library dependencies in Python

Sometimes I come across my past Python programs and ponder on what libraries or packages were necessary for them to function correctly. Generally speaking, this can be done through trial and error as one runs the code and extract the dependencies by hand from the errors thrown at the startup. Of course, this method is very time-consuming and not particularly fun.

pipreqs and pipdeptree

This is where pipreqs comes to the rescue: This library can generate the requirements.txt file based on import statements in any project. It is also very easy to use:

$ pipreqs /home/project/devlabs
Successfully saved requirements file in /home/project/devlabs/requirements.txt

This will of course only work in case, if pipreqs is added to the path (or installed globally via pip). But you can call it also directly:

$ python3 -m pipreqs.pipreqs /home/project/devlabs

Another useful tool for dependency management in Python is pipdeptree (https://github.com/tox-dev/pipdeptree): It can be used to show the dependency tree of your installed packages, and can also output the tree in a requirements.txt. pipdeptree can help also with identifying conflicting dependencies installed in the environment.

requirements.txt vs. Pipfile.lock

Both requirements.txt and Pipfile.lock are used to manage dependencies in Python, but they have some key differences:

requirements.txt

requirements.txt is a plain text file that lists all the dependencies for a project. It can be easily created and edited manually, and it's widely supported by various tools and systems. It's a common practice to include a requirements.txt file in a Python project, so that other people can easily install the same dependencies that you used.

Usually you use the pip freeze function to create the requirements.txt file:

python -m pip freeze > requirements.txt
# of course you can use pip directly if its available in the path
pip freeze > requirements.txt

The installation of the dependencies from the requirements.txt file can be done via:

python -m pip install -r requirements.txt

To use the dependencies only in the currently open project (and not globally), use a virtual environment:

• In order to create a python environment .venv use:
  python3 -m venv .venv
• To use the environment, you have to activate it first:
  . .venv/bin/activate
  (for Linux/macOS, on Windows use the activate.bat file)

Of course, these steps must be completed before installing the dependencies from the requirements.txt.

Pipfile.lock

Pipfile.lock on the other hand is a file that is automatically generated by pipenv when you use the pipenv lock command. This command creates a Pipfile.lock, which is similar to a requirements.txt file, but it also includes version information and other details about the dependencies. This file is used to ensure that the same versions of packages are installed across different environments and machines.

If you are using pipenv in your project, it's recommended to use the Pipfile.lock since it provides more detailed information about the dependencies and can be used to ensure consistency across different environments. If you are not using pipenv and you would like to have more control over your dependencies and the version that you are using, the requirements.txt would be a better choice.