Python virtual environment is widely used for Python project development. However, it’s not quite useful for me because I hardly do development on bare-metal machines. Instead, I prefer to use Docker containers for development and Python virtual environment is almost useless in Docker containers. Even if any dependency is screwed up, I could just simply fix and rebuild the Docker image.
Recently, I realized that the way I used to set up Python in Ubuntu Docker containers before 24.04 no longer works for Ubuntu 24.04, because Ubuntu 24.04 enforces the use of Python virtual environment for Python package management. Without Python virtual environment, it’s not possible to install any Python packages system-wide without using some nasty flags such as --break-system-packages. So I decided to change the way I set up Python in Docker containers to use Python virtual environment while keeping the development interface and workflow the same.
In this blog post, I will demonstrate how to set up Python in Docker containers for Ubuntu 22.04 without Python virtual environment and for Ubuntu 24.04 with Python virtual environment, respectively.
Python Setup In Docker
Ubuntu 22.04
This is the old way I used to set up Python in Docker containers for Ubuntu 22.04. I could install Python packages system-wide via pip freely.
If I simply change the base image used in the previous Dockerfile to Ubuntu 24.04 and build the Docker image, I will get an error because Ubuntu 24.04 enforces the use of Python virtual environment for Python package management.
$ docker build -f python_ubuntu_24.04_problematic.Dockerfile --tag python:ubuntu_24.04 . [+] Building 1.4s (6/7) docker:default => [internal] load build definition from python_ubuntu_24.04_problematic.Dockerfile 0.2s => => transferring dockerfile: 885B 0.0s => [internal] load metadata for docker.io/library/ubuntu:24.04 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [1/4] FROM docker.io/library/ubuntu:24.04 0.0s => CACHED [2/4] RUN apt-get update && apt-get install -y --no-install-recommends build-essential software-properties-common autoconf automake 0.0s => ERROR [3/4] RUN cd /usr/local/bin && ln -s /usr/bin/python3 python && ln -s /usr/bin/pip3 pip && pip install --upgrade pip setuptools wheel 0.9s ------ > [3/4] RUN cd /usr/local/bin && ln -s /usr/bin/python3 python && ln -s /usr/bin/pip3 pip && pip install --upgrade pip setuptools wheel: 0.662 error: externally-managed-environment 0.662 0.662 × This environment is externally managed 0.662 ╰─> To install Python packages system-wide, try apt install 0.662 python3-xyz, where xyz is the package you are trying to 0.662 install. 0.662 0.662 If you wish to install a non-Debian-packaged Python package, 0.662 create a virtual environment using python3 -m venv path/to/venv. 0.662 Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make 0.662 sure you have python3-full installed. 0.662 0.662 If you wish to install a non-Debian packaged Python application, 0.662 it may be easiest to use pipx install xyz, which will manage a 0.662 virtual environment for you. Make sure you have pipx installed. 0.662 0.662 See /usr/share/doc/python3.12/README.venv for more information. 0.662 0.662 note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. 0.662 hint: See PEP 668 for the detailed specification. ------ python_ubuntu_24.04_problematic.Dockerfile:32 -------------------- 31 | 32 | >>> RUN cd /usr/local/bin && \ 33 | >>> ln -s /usr/bin/python3 python && \ 34 | >>> ln -s /usr/bin/pip3 pip && \ 35 | >>> pip install --upgrade pip setuptools wheel 36 | -------------------- ERROR: failed to solve: process "/bin/sh -c cd /usr/local/bin && ln -s /usr/bin/python3 python && ln -s /usr/bin/pip3 pip && pip install --upgrade pip setuptools wheel" did not complete successfully: exit code: 1
To fix the problem, I need to use Python virtual environment for Python package management in Ubuntu 24.04. The following is the new way I set up Python in Docker containers for Ubuntu 24.04.
Note that I installed Python using the python3-full package in Ubuntu 24.04 instead of the python3, python3-dev, python3-pip, and python3-setuptools packages in Ubuntu 22.04. Also the Python virtual environment path is prepended to the PATH environment variable so that the Python virtual environment is activated by default. Then I can install Python packages using pip freely again, as if there is no Python virtual environment. Note that this Python setup method will not work for Ubuntu 22.04.