trexio/docker

Docker build

Containerized builds related to TREXIO library.

Producing manylinux wheels for Python API

Distributing binary wheels for Python packages that rely on C extensions is a non-trivial task. For more details, see the associated PEP 425 and PEP 600.

This issue is particularly tricky on Linux platforms. However, PyPA provides Docker containers that can be used as an isolated environment to produce binary wheels that are compatible with most Linux distributions. Such wheels should contain manylinux in their platform tag (e.g. ...-manylinux_2_24_x86_64.whl on 64-bit Intel with glibc >= 2.24). More technical details can be found in the corresponding GitHub repository.

This section summarizes the steps needed to build manylinux-compatible wheels using the aforementioned containers.

Building the Docker containers

The primary TREXIO back end at the moment is based on the HDF5 library. Because of this, one has to build new Docker images from the PyPA ones. These images will contain the HDF5 library and will be used to produce wheels for the Python API.

To build the Docker images on x86_64 platform, execute the following:

cd hdf5-on-docker/ && ./build_images.sh

This should produce several images tagged with hdf5_1_12_ prefix and _x86_64 suffix. These images inherit from the PyPA manylinux containers. All available images can be listed using docker image list command.

Building the manylinux wheels

First, make sure that you have the source code distribution (e.g. trexio-0.2.0.tar.gz) of the Python API in the current directory. Then run one of the previously produced containers in the interactive mode using the following command:

docker run --rm -it -e PLAT=manylinux2014_x86_64 -v `pwd`:/tmp hdf5_1_12_on_2014_x86_64 /bin/bash

where 2014_x86_64 can be replaced with any other available platform suffix (e.g. 2010_x86_64 or 2_24_x86_64) The docker run command line arguments used here:

• -i (run the docker container in interactive mode)
• --rm (remove the container upon exit)
• -v `pwd`:/tmp (mount current directory into the tmp directory on the container
• -e (set the environment variables according to the provided list)
• hdf5_1_12_on_2014_x86_64 (name of the Docker image to run)
• /bin/bash (which binary the container should execute)

To run the ARM-based container (e.g. with _aarch64 suffix) on Intel/AMD processor, just add the --platform linux/arm64 to the docker run arguments.

Once the Docker container is launched and the corresponding prompt is switched to the container, run the following (example for release 0.2.0):

cd tmp/ && ./build_manylinux_wheels.sh trexio-0.2.0.tar.gz

This script creates virtual envirionments for several versions of CPython (e.g. 3.7, 3.8, 3.9), installs all requirements, produces trexio wheels with conventional linux_x86_64 tag and repairs them to be tagged with manylinux using the auditwheel tool from PyPA

The produced wheels with the manylinux platfrom tag can be found in trexio-0.2.0/wheelhouse directory.

Exporting the wheels

You may want to produce wheels for several versions of glibc. Prior to running a new docker container, make sure to rename/move the wheelhouse directory. Otherwise, the build_manylinux_wheels.sh script will crush.