Python virtuel environment: template project

Introduction

Python is a very popular programming language for the developement of scripts and small applications. One of its main strenghts is the incredible amount of available libraries.

However, sharing a Python application is unfortunately not always an easy task, as this latter may be based on some libraries, which are not necessarily installed on the target machine. Worse, it's likely for the required library to be already already installed, but with a different version than required (and the library might have compatibility breaks between versions).

To overcome this problem, one can use virtual environments, a very powerful and convenient feature of Python.

Virtual environements ? What are they used for ?

Virtual environments allow to create isolated environments for the execution of various Python projects. Each of those projets can have its own dependencies, independently from other projects. For instance, a project A may require a certain library in version 2.x, while another project B may require the same library, but in version 3.x:

Virtual environment example

This way, both projects can coexist, without interfering. By installing the required dependencies (libraries) in separate virtual environments, we don't rely any more on the installation of these libraries in the base installation of Python.

How does this work in practice ?

Each project is provided with a file "requirements.txt", which lists the required dependencies for their execution. After downloading the project on the machine, we proceed as follow to install the dependencies:

  1. Create the virtual environment
  2. Activate the virtual environment
  3. Using pip (Python's packages manager), install the dependencies, according to file "requirements.txt".

Then, in order to execute the Python scripts in its dedicated virtual environment, we proceed as follow:

  1. Activate the virtual environment
  2. Execute the script

It's that simple !

Template of Python project with virtual environment

You'll find here a template of Python project : https://github.com/pull-up-dev/python-template.

This template project includes:

  • Scripts ("prepare.ps1" for Windows and "prepare.sh" for Linux) allowing to prepare a virtual environment with the required dependencies (in this template the library "pyyaml"), from a file "requirements.txt".
  • Scripts ("launch.ps1" for Windows and "launch.sh" for Linux) allowing to activate the virtual environment and launch an example Python application.
  • Scripts ("mrproper.ps1" for Windows and "mrproper.sh" for Linux) allowing to clean (delete) the virtual environment.

These scripts, as well as the provided application are intended to serve as a starting point for Python projects, that can then easily be shared.

The commands which are present in the scripts are described in more details below. There are some differences between Windows and Linux for the commands. Notably, the command for Python under Windows is "python", while it is "python3" under Linux. This might vary depending on your installation of Python and/or the Linux distribution you are using.

Scripts for preparation of the virtual environment

Scripts "prepare.ps1" (Windows) and "prepare.sh" (Linux) first install the package "virtualenv", which allows the use of virtual environments:

python -m pip install virtualenv

Once this is done, the script creates a virtual environment in subfolder "virtEnv":

python -m virtualenv --no-setuptools --no-pip --no-wheel --no-download --system-site-packages -p python virtEnv

The virtual environment is then activated. Under Windows, we simply execute the command "activate" which is located in the virtual environment:

virtEnv\Scripts\activate

Under Linux, the activation command is slightly different, because "activate" shall be called through the command "source" as follow:

source virtEnv/bin/activate

Finally, the dependencies are installed, according to file "requirements.txt":

python -m pip install -r requirements.txt

The syntax for the file "requirements.txt" is described on the following page: https://pip.pypa.io/en/stable/reference/requirements-file-format/. It is for instance possible to define a fixed version for a library, or a minimal version.

After the execution of the Python scripts, one can deactivate the virtual environment with the command:

deactivate

Scripts for the execution of the Python application

Fir of all, in scripts "launch.ps1" (Windows) or "launch.sh" (Linux), the virtual environment is activated. Under Windows:

.\virtEnv\Scripts\activate

Or under Linux:

source virtEnv/bin/activate

Then, the Python application is launched, with the received arguments passed to it.

The application uses the library "argparse" to handle the received arguments. In this minimal example, the only argument which can be passed is the path to a YAML configuration file. If this argument is not passed, the file "config.yml" (provided with the project) is used. The application loads that configuration file (using library "pyyaml", which is listed as dependency in file "requirements.txt") and prints the value of parameter "dummy_param" of this configuration file.

Additional arguments can easily be added using library "argparse". Its documentation is available here: https://docs.python.org/3/library/argparse.html.

Script for cleanup of the virtual environment

Scripts "mrproper.ps1" (Windows) and "mrproper.sh" (Linux) remove the virtual environment, by simply removing the subfolder it was created in.

Conclusion

The provided project is a minimal example project for the use of virtual environments, including scripts allowing to prepare and clean virtual environments, and execute a Python application in this virtual environment.

The provided Python application implements the use of the library "argparse", allowing to receive arguments, as well as the use of a YAML configuration file.

The project can easily be tailored to your needs, and serve as a starting point for various projets.