TREX/trexio
1
0
Fork 0
mirror of https://github.com/TREX-CoE/trexio.git synced 2024-08-24 22:21:43 +02:00
Code Issues Projects Releases Wiki Activity

completed README

Browse Source
This commit is contained in:
q-posev 2021-09-16 16:40:29 +02:00
parent 66cf305a40
commit 79f233e00d
2 changed files with 71 additions and 45 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

45
docker/README
View File

@ -1,45 +0,0 @@
First, make sure to place the source code distribution (suffixed with .tar.gz) of the TREXIO Python API in this directory.
TODO: the scripts have to be adapted for an arbitrary version of TREXIO and Python !
Outside Docker image:
# Build containers with hdf5 inside:
# for manylinux2014_x86_64
sudo docker build -t hdf5_1_12_on_2014_x86_64 . -f Dockerfile_2014_x86_64
# for manylinux_2_24_x86_64
sudo docker build -t hdf5_1_12_on_2_24_x86_64 . -f Dockerfile_2_24_x86_64
# (create an image using HDF5 containers, see https://github.com/h5py/hdf5-manylinux)
# -t hdf5_1_12_on_${PLATFORM} builds an image with a custom name (hdf5_1_12_1)
# Run one of the produced containers in the interactive mode
# for manylinux2014_x86_64
sudo docker run --rm -it -e PLAT=manylinux2014_x86_64 -v `pwd`:/tmp hdf5_1_12_on_2014_x86_64 /bin/bash
# for manylinux_2_24_x86_64
sudo docker run --rm -it -e PLAT=manylinux_2_24_x86_64 -v `pwd`:/tmp hdf5_1_12_on_2_24_x86_64 /bin/bash
#(PLAT specifies the base platform tag for manilinux)
#-i (run docker container interactively)
#-t f8e4232fa208 (run an image using ID if the image is not named)
#/bin/bash (a binary to run inside a container environment)
In the Docker image:
cd tmp && ./build_manylinux_wheels_py_36_37_38.sh
# (creates virtual environments, installs things with pip, produces conventional wheels and repairs them with auditwheel)
# auditwheel repair trexio-0.1.0/dist/trexio-0.1.0-cp37-cp37m-linux_x86_64.whl
# (repairs wheel and produces manylinux one)
After Docker container execution:
# the produced manylinux wheels are in trexio-0.1.0/wheelhouse directory
# ALTERNATIVELY: one can copy the produced manylinux wheel from the container back to the host system
sudo docker cp $CONTAINER_ID:/tmp/trexio-0.1.0/wheelhouse/MANYLINUX_NAME.whl directory-on-the-user-machine/

71
docker/README.md Normal file
View File

@ -0,0 +1,71 @@
# Docker build
Containerized builds related to TREXIO library.
## Producing manylinux wheels for Python API
Distributing binary wheels for Python packages that rely on C extensions is a non-trivial task.
For more details, see the associated
[PEP 425](https://www.python.org/dev/peps/pep-0425/) and
[PEP 600](https://www.python.org/dev/peps/pep-0600/).
This issue is particularly tricky on Linux platforms. However, PyPA provides `Docker` containers
that can be used as an isolated environment to produce binary wheels that are compatible with most Linux distributions.
Such wheels should contain `manylinux` in their platform tag
(e.g. `...-manylinux_2_24_x86_64.whl` on 64-bit Intel with `glibc >= 2.24`).
More technical details can be found in the corresponding [GitHub repository](https://github.com/pypa/manylinux).
This section summarizes the steps needed to build `manylinux`-compatible wheels using the aforementioned containers.
### Building the Docker containers
The primary TREXIO back end at the moment is based on the [HDF5](https://portal.hdfgroup.org/display/HDF5/HDF5) library.
Because of this, one has to build new Docker images from the PyPA ones.
These images will contain the HDF5 library and will be used to produce wheels for the Python API.
To build the Docker images on x86_64 platform, execute the following:
`cd hdf5-on-docker/ && ./build_images.sh`
This should produce several images tagged with `hdf5_1_12_` prefix and `_x86_64` suffix.
These images inherit from the PyPA manylinux containers.
All available images can be listed using `docker image list` command.
### Building the manylinux wheels
First, make sure that you have the source code distribution
(e.g. `trexio-0.2.0.tar.gz`) of the Python API in the current directory.
Then run one of the previously produced containers in the interactive mode using the following command:
``docker run --rm -it -e PLAT=manylinux2014_x86_64 -v `pwd`:/tmp hdf5_1_12_on_2014_x86_64 /bin/bash``
where `2014_x86_64` can be replaced with any other available platform suffix (e.g. `2010_x86_64` or `2_24_x86_64`)
The `docker run` command line arguments used here:
- `-i` (run the docker container in interactive mode)
- `--rm` (remove the container upon exit)
- ``-v `pwd`:/tmp`` (mount current directory into the `tmp` directory on the container
- `-e` (set the environment variables according to the provided list)
- `hdf5_1_12_on_2014_x86_64` (name of the Docker image to run)
- `/bin/bash` (which binary the container should execute)
Once the Docker container is launched and the corresponding prompt is switched to the container, run the following (example for release 0.2.0):
`cd tmp/ && ./build_manylinux_wheels.sh trexio-0.2.0.tar.gz`
This script creates virtual envirionments for several versions of `CPython` (e.g. 3.7, 3.8, 3.9),
installs all requirements,
produces `trexio` wheels with conventional `linux_x86_64` tag and
repairs them to be tagged with `manylinux` using the `auditwheel` tool from PyPA
The produced wheels with the `manylinux` platfrom tag can be found in `trexio-0.2.0/wheelhouse` directory.
### Exporting the wheels
You may want to produce wheels for several versions of `glibc`.
Prior to running a new docker container, make sure to rename/move the `wheelhouse` directory.
Otherwise, the `build_manylinux_wheels.sh` script will crush.