mirror of
https://github.com/TREX-CoE/trexio.git
synced 2024-12-23 04:43:57 +01:00
72 lines
3.3 KiB
Markdown
72 lines
3.3 KiB
Markdown
|
# Docker build
|
||
|
|
|
||
|
|
Containerized builds related to TREXIO library.
|
||
|
|
|
||
|
|
## Producing manylinux wheels for Python API
|
||
|
|
|
||
|
|
Distributing binary wheels for Python packages that rely on C extensions is a non-trivial task.
|
||
|
|
For more details, see the associated
|
||
|
|
[PEP 425](https://www.python.org/dev/peps/pep-0425/) and
|
||
|
|
[PEP 600](https://www.python.org/dev/peps/pep-0600/).
|
||
|
|
|
||
|
|
This issue is particularly tricky on Linux platforms. However, PyPA provides `Docker` containers
|
||
|
|
that can be used as an isolated environment to produce binary wheels that are compatible with most Linux distributions.
|
||
|
|
Such wheels should contain `manylinux` in their platform tag
|
||
|
|
(e.g. `...-manylinux_2_24_x86_64.whl` on 64-bit Intel with `glibc >= 2.24`).
|
||
|
|
More technical details can be found in the corresponding [GitHub repository](https://github.com/pypa/manylinux).
|
||
|
|
|
||
|
|
This section summarizes the steps needed to build `manylinux`-compatible wheels using the aforementioned containers.
|
||
|
|
|
||
|
|
|
||
|
|
### Building the Docker containers
|
||
|
|
|
||
|
|
The primary TREXIO back end at the moment is based on the [HDF5](https://portal.hdfgroup.org/display/HDF5/HDF5) library.
|
||
|
|
Because of this, one has to build new Docker images from the PyPA ones.
|
||
|
|
These images will contain the HDF5 library and will be used to produce wheels for the Python API.
|
||
|
|
|
||
|
|
To build the Docker images on x86_64 platform, execute the following:
|
||
|
|
|
||
|
|
`cd hdf5-on-docker/ && ./build_images.sh`
|
||
|
|
|
||
|
|
This should produce several images tagged with `hdf5_1_12_` prefix and `_x86_64` suffix.
|
||
|
|
These images inherit from the PyPA manylinux containers.
|
||
|
|
All available images can be listed using `docker image list` command.
|
||
|
|
|
||
|
|
|
||
|
|
### Building the manylinux wheels
|
||
|
|
|
||
|
|
First, make sure that you have the source code distribution
|
||
|
|
(e.g. `trexio-0.2.0.tar.gz`) of the Python API in the current directory.
|
||
|
|
Then run one of the previously produced containers in the interactive mode using the following command:
|
||
|
|
|
||
|
|
``docker run --rm -it -e PLAT=manylinux2014_x86_64 -v `pwd`:/tmp hdf5_1_12_on_2014_x86_64 /bin/bash``
|
||
|
|
|
||
|
|
where `2014_x86_64` can be replaced with any other available platform suffix (e.g. `2010_x86_64` or `2_24_x86_64`)
|
||
|
|
The `docker run` command line arguments used here:
|
||
|
|
|
||
|
|
- `-i` (run the docker container in interactive mode)
|
||
|
|
- `--rm` (remove the container upon exit)
|
||
|
|
- ``-v `pwd`:/tmp`` (mount current directory into the `tmp` directory on the container
|
||
|
|
- `-e` (set the environment variables according to the provided list)
|
||
|
|
- `hdf5_1_12_on_2014_x86_64` (name of the Docker image to run)
|
||
|
|
- `/bin/bash` (which binary the container should execute)
|
||
|
|
|
||
|
|
Once the Docker container is launched and the corresponding prompt is switched to the container, run the following (example for release 0.2.0):
|
||
|
|
|
||
|
|
`cd tmp/ && ./build_manylinux_wheels.sh trexio-0.2.0.tar.gz`
|
||
|
|
|
||
|
|
This script creates virtual envirionments for several versions of `CPython` (e.g. 3.7, 3.8, 3.9),
|
||
|
|
installs all requirements,
|
||
|
|
produces `trexio` wheels with conventional `linux_x86_64` tag and
|
||
|
|
repairs them to be tagged with `manylinux` using the `auditwheel` tool from PyPA
|
||
|
|
|
||
|
|
The produced wheels with the `manylinux` platfrom tag can be found in `trexio-0.2.0/wheelhouse` directory.
|
||
|
|
|
||
|
|
|
||
|
|
### Exporting the wheels
|
||
|
|
|
||
|
|
You may want to produce wheels for several versions of `glibc`.
|
||
|
|
Prior to running a new docker container, make sure to rename/move the `wheelhouse` directory.
|
||
|
|
Otherwise, the `build_manylinux_wheels.sh` script will crush.
|
||
|
|
|