Integrating Python Poetry with Docker

by

There are several things to keep in mind when using poetry together with docker.

Installation

Official way to install poetry is via:

curl -sSL https://install.python-poetry.org | python3 -

This way allows poetry and its dependencies to be isolated from your dependencies. But, in my point of view, it is not a very good thing for two reasons:

  1. poetry version might get an update and it will break your build. In this case you can specify POETRY_VERSION environment variable. Installer will respect it
  2. I do not like the idea to pipe things from the internet into my containers without any protection from possible file modifications

So, I use pip install 'poetry==$POETRY_VERSION'. As you can see, I still recommend to pin your version.

Also, pin this version in your pyproject.toml as well:

[build-system]
# Should be the same as `$POETRY_VERSION`:
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

It will protect you from version mismatch between your local and docker environments.

Caching dependencies

We want to cache our requirements and only reinstall them when pyproject.toml or poetry.lock files change. Otherwise builds will be slow. To achieve working cache layer we should put:

COPY poetry.lock pyproject.toml /code/

After the poetry is installed, but before any other files are added.

Virtualenv

The next thing to keep in mind is virtualenv creation. We do not need it in docker. It is already isolated. So, we use poetry config virtualenvs.create false setting to turn it off.

Development vs Production

If you use the same Dockerfile for both development and production as I do, you will need to install different sets of dependencies based on some environment variable:

poetry install $(test "$YOUR_ENV" == production && echo "--no-dev")

This way $YOUR_ENV will control which dependencies set will be installed: all (default) or production only with --no-dev flag.

You may also want to add some more options for better experience:

  1. --no-interaction not to ask any interactive questions
  2. --no-ansi flag to make your output more log friendly

Result

You will end up with something similar to:

FROM python:3.6.6-alpine3.7

ARG YOUR_ENV

ENV YOUR_ENV=${YOUR_ENV} \
  PYTHONFAULTHANDLER=1 \
  PYTHONUNBUFFERED=1 \
  PYTHONHASHSEED=random \
  PIP_NO_CACHE_DIR=off \
  PIP_DISABLE_PIP_VERSION_CHECK=on \
  PIP_DEFAULT_TIMEOUT=100 \
  POETRY_VERSION=1.0.0

# System deps:
RUN pip install "poetry==$POETRY_VERSION"

# Copy only requirements to cache them in docker layer
WORKDIR /code
COPY poetry.lock pyproject.toml /code/

# Project initialization:
RUN poetry config virtualenvs.create false \
  && poetry install $(test "$YOUR_ENV" == production && echo "--no-dev") --no-interaction --no-ansi

# Creating folders, and files for a project:
COPY . /code

You can find a fully working real-life example here: wemake-django-template

Update on 2019-12-17

  • Update poetry to 1.0

Update on 2022-11-24

  • Update curl command to use modern poetry installation script

More Related Contents:

  1. Deploying a minimal flask app in docker – server connection issues
  2. Python app does not print anything when running detached in docker
  3. Docker how to run pip requirements.txt only if there was a change?
  4. Activate python virtualenv in Dockerfile
  5. toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading
  6. standard_init_linux.go:211: exec user process caused “exec format error”
  7. Install Oracle Instant client into Docker container for Python cx_Oracle
  8. How do I reduce a python (docker) image size using a multi-stage build?
  9. Is distributing python source code in Docker secure?
  10. Error: pg_config executable not found when installing psycopg2 on Alpine in Docker
  11. standard_init_linux.go:178: exec user process caused “exec format error”
  12. Django connection to postgres by docker-compose
  13. Can’t install pip packages inside a docker container with Ubuntu
  14. Flask CLI throws ‘OSError: [Errno 8] Exec format error’ when run through docker-compose
  15. How to avoid reinstalling packages when building Docker image for Python projects?
  16. Reading in environment variables from an environment file
  17. installing cPickle with python 3.5
  18. WARNING: Running pip as the ‘root’ user
  19. ImportError: cannot import name ‘soft_unicode’ from ‘markupsafe’
  20. Encrypted and secure docker containers
  21. Docker- django throws error while connecting to postgres: psycopg2.OperationalError: could not connect to server: Connection refused
  22. Using a pip cache directory in docker builds
  23. Installing numpy on Docker Alpine
  24. Feature comparison between npm, pip, pipenv and Poetry package managers [closed]
  25. Python multiprocessing application is getting stuck in docker container
  26. What’s the difference between Docker and Python virtualenv?
  27. Access Jupyter notebook running on Docker container
  28. Activate conda environment in docker
  29. “x not in y” or “not x in y”
  30. Argparse optional positional arguments?

Leave a Comment