mirror of
https://github.com/TREX-CoE/trexio.git
synced 2024-08-24 22:21:43 +02:00
completed README
This commit is contained in:
parent
66cf305a40
commit
79f233e00d
@ -1,45 +0,0 @@
|
||||
First, make sure to place the source code distribution (suffixed with .tar.gz) of the TREXIO Python API in this directory.
|
||||
|
||||
TODO: the scripts have to be adapted for an arbitrary version of TREXIO and Python !
|
||||
|
||||
Outside Docker image:
|
||||
|
||||
# Build containers with hdf5 inside:
|
||||
|
||||
# for manylinux2014_x86_64
|
||||
sudo docker build -t hdf5_1_12_on_2014_x86_64 . -f Dockerfile_2014_x86_64
|
||||
|
||||
# for manylinux_2_24_x86_64
|
||||
sudo docker build -t hdf5_1_12_on_2_24_x86_64 . -f Dockerfile_2_24_x86_64
|
||||
|
||||
# (create an image using HDF5 containers, see https://github.com/h5py/hdf5-manylinux)
|
||||
# -t hdf5_1_12_on_${PLATFORM} builds an image with a custom name (hdf5_1_12_1)
|
||||
|
||||
# Run one of the produced containers in the interactive mode
|
||||
|
||||
# for manylinux2014_x86_64
|
||||
sudo docker run --rm -it -e PLAT=manylinux2014_x86_64 -v `pwd`:/tmp hdf5_1_12_on_2014_x86_64 /bin/bash
|
||||
|
||||
# for manylinux_2_24_x86_64
|
||||
sudo docker run --rm -it -e PLAT=manylinux_2_24_x86_64 -v `pwd`:/tmp hdf5_1_12_on_2_24_x86_64 /bin/bash
|
||||
|
||||
#(PLAT specifies the base platform tag for manilinux)
|
||||
#-i (run docker container interactively)
|
||||
#-t f8e4232fa208 (run an image using ID if the image is not named)
|
||||
#/bin/bash (a binary to run inside a container environment)
|
||||
|
||||
In the Docker image:
|
||||
|
||||
cd tmp && ./build_manylinux_wheels_py_36_37_38.sh
|
||||
# (creates virtual environments, installs things with pip, produces conventional wheels and repairs them with auditwheel)
|
||||
|
||||
# auditwheel repair trexio-0.1.0/dist/trexio-0.1.0-cp37-cp37m-linux_x86_64.whl
|
||||
# (repairs wheel and produces manylinux one)
|
||||
|
||||
After Docker container execution:
|
||||
|
||||
# the produced manylinux wheels are in trexio-0.1.0/wheelhouse directory
|
||||
|
||||
# ALTERNATIVELY: one can copy the produced manylinux wheel from the container back to the host system
|
||||
sudo docker cp $CONTAINER_ID:/tmp/trexio-0.1.0/wheelhouse/MANYLINUX_NAME.whl directory-on-the-user-machine/
|
||||
|
71
docker/README.md
Normal file
71
docker/README.md
Normal file
@ -0,0 +1,71 @@
|
||||
# Docker build
|
||||
|
||||
Containerized builds related to TREXIO library.
|
||||
|
||||
## Producing manylinux wheels for Python API
|
||||
|
||||
Distributing binary wheels for Python packages that rely on C extensions is a non-trivial task.
|
||||
For more details, see the associated
|
||||
[PEP 425](https://www.python.org/dev/peps/pep-0425/) and
|
||||
[PEP 600](https://www.python.org/dev/peps/pep-0600/).
|
||||
|
||||
This issue is particularly tricky on Linux platforms. However, PyPA provides `Docker` containers
|
||||
that can be used as an isolated environment to produce binary wheels that are compatible with most Linux distributions.
|
||||
Such wheels should contain `manylinux` in their platform tag
|
||||
(e.g. `...-manylinux_2_24_x86_64.whl` on 64-bit Intel with `glibc >= 2.24`).
|
||||
More technical details can be found in the corresponding [GitHub repository](https://github.com/pypa/manylinux).
|
||||
|
||||
This section summarizes the steps needed to build `manylinux`-compatible wheels using the aforementioned containers.
|
||||
|
||||
|
||||
### Building the Docker containers
|
||||
|
||||
The primary TREXIO back end at the moment is based on the [HDF5](https://portal.hdfgroup.org/display/HDF5/HDF5) library.
|
||||
Because of this, one has to build new Docker images from the PyPA ones.
|
||||
These images will contain the HDF5 library and will be used to produce wheels for the Python API.
|
||||
|
||||
To build the Docker images on x86_64 platform, execute the following:
|
||||
|
||||
`cd hdf5-on-docker/ && ./build_images.sh`
|
||||
|
||||
This should produce several images tagged with `hdf5_1_12_` prefix and `_x86_64` suffix.
|
||||
These images inherit from the PyPA manylinux containers.
|
||||
All available images can be listed using `docker image list` command.
|
||||
|
||||
|
||||
### Building the manylinux wheels
|
||||
|
||||
First, make sure that you have the source code distribution
|
||||
(e.g. `trexio-0.2.0.tar.gz`) of the Python API in the current directory.
|
||||
Then run one of the previously produced containers in the interactive mode using the following command:
|
||||
|
||||
``docker run --rm -it -e PLAT=manylinux2014_x86_64 -v `pwd`:/tmp hdf5_1_12_on_2014_x86_64 /bin/bash``
|
||||
|
||||
where `2014_x86_64` can be replaced with any other available platform suffix (e.g. `2010_x86_64` or `2_24_x86_64`)
|
||||
The `docker run` command line arguments used here:
|
||||
|
||||
- `-i` (run the docker container in interactive mode)
|
||||
- `--rm` (remove the container upon exit)
|
||||
- ``-v `pwd`:/tmp`` (mount current directory into the `tmp` directory on the container
|
||||
- `-e` (set the environment variables according to the provided list)
|
||||
- `hdf5_1_12_on_2014_x86_64` (name of the Docker image to run)
|
||||
- `/bin/bash` (which binary the container should execute)
|
||||
|
||||
Once the Docker container is launched and the corresponding prompt is switched to the container, run the following (example for release 0.2.0):
|
||||
|
||||
`cd tmp/ && ./build_manylinux_wheels.sh trexio-0.2.0.tar.gz`
|
||||
|
||||
This script creates virtual envirionments for several versions of `CPython` (e.g. 3.7, 3.8, 3.9),
|
||||
installs all requirements,
|
||||
produces `trexio` wheels with conventional `linux_x86_64` tag and
|
||||
repairs them to be tagged with `manylinux` using the `auditwheel` tool from PyPA
|
||||
|
||||
The produced wheels with the `manylinux` platfrom tag can be found in `trexio-0.2.0/wheelhouse` directory.
|
||||
|
||||
|
||||
### Exporting the wheels
|
||||
|
||||
You may want to produce wheels for several versions of `glibc`.
|
||||
Prior to running a new docker container, make sure to rename/move the `wheelhouse` directory.
|
||||
Otherwise, the `build_manylinux_wheels.sh` script will crush.
|
||||
|
Loading…
Reference in New Issue
Block a user