mirror of
https://github.com/TREX-CoE/trexio.git
synced 2024-07-22 18:57:39 +02:00
Merge pull request #57 from TREX-CoE/swig-python
Add tools and README for manylinux wheels
This commit is contained in:
Evgeny Posenitskiy
GitHub
commit
418aa304cd
@ -1,45 +0,0 @@
|
||||
First, make sure to place the source code distribution (suffixed with .tar.gz) of the TREXIO Python API in this directory.
|
||||
|
||||
TODO: the scripts have to be adapted for an arbitrary version of TREXIO and Python !
|
||||
|
||||
Outside Docker image:
|
||||
|
||||
# Build containers with hdf5 inside:
|
||||
|
||||
# for manylinux2014_x86_64
|
||||
sudo docker build -t hdf5_1_12_on_2014_x86_64 . -f Dockerfile_2014_x86_64
|
||||
|
||||
# for manylinux_2_24_x86_64
|
||||
sudo docker build -t hdf5_1_12_on_2_24_x86_64 . -f Dockerfile_2_24_x86_64
|
||||
|
||||
# (create an image using HDF5 containers, see https://github.com/h5py/hdf5-manylinux)
|
||||
# -t hdf5_1_12_on_${PLATFORM} builds an image with a custom name (hdf5_1_12_1)
|
||||
|
||||
# Run one of the produced containers in the interactive mode
|
||||
|
||||
# for manylinux2014_x86_64
|
||||
sudo docker run --rm -it -e PLAT=manylinux2014_x86_64 -v `pwd`:/tmp hdf5_1_12_on_2014_x86_64 /bin/bash
|
||||
|
||||
# for manylinux_2_24_x86_64
|
||||
sudo docker run --rm -it -e PLAT=manylinux_2_24_x86_64 -v `pwd`:/tmp hdf5_1_12_on_2_24_x86_64 /bin/bash
|
||||
|
||||
#(PLAT specifies the base platform tag for manilinux)
|
||||
#-i (run docker container interactively)
|
||||
#-t f8e4232fa208 (run an image using ID if the image is not named)
|
||||
#/bin/bash (a binary to run inside a container environment)
|
||||
|
||||
In the Docker image:
|
||||
|
||||
cd tmp && ./build_manylinux_wheels_py_36_37_38.sh
|
||||
# (creates virtual environments, installs things with pip, produces conventional wheels and repairs them with auditwheel)
|
||||
|
||||
# auditwheel repair trexio-0.1.0/dist/trexio-0.1.0-cp37-cp37m-linux_x86_64.whl
|
||||
# (repairs wheel and produces manylinux one)
|
||||
|
||||
After Docker container execution:
|
||||
|
||||
# the produced manylinux wheels are in trexio-0.1.0/wheelhouse directory
|
||||
|
||||
# ALTERNATIVELY: one can copy the produced manylinux wheel from the container back to the host system
|
||||
sudo docker cp $CONTAINER_ID:/tmp/trexio-0.1.0/wheelhouse/MANYLINUX_NAME.whl directory-on-the-user-machine/
|
||||
|
||||
71
docker/README.md
Normal file
71
docker/README.md
Normal file
@ -0,0 +1,71 @@
|
||||
# Docker build
|
||||
|
||||
Containerized builds related to TREXIO library.
|
||||
|
||||
## Producing manylinux wheels for Python API
|
||||
|
||||
Distributing binary wheels for Python packages that rely on C extensions is a non-trivial task.
|
||||
For more details, see the associated
|
||||
[PEP 425](https://www.python.org/dev/peps/pep-0425/) and
|
||||
[PEP 600](https://www.python.org/dev/peps/pep-0600/).
|
||||
|
||||
This issue is particularly tricky on Linux platforms. However, PyPA provides `Docker` containers
|
||||
that can be used as an isolated environment to produce binary wheels that are compatible with most Linux distributions.
|
||||
Such wheels should contain `manylinux` in their platform tag
|
||||
(e.g. `...-manylinux_2_24_x86_64.whl` on 64-bit Intel with `glibc >= 2.24`).
|
||||
More technical details can be found in the corresponding [GitHub repository](https://github.com/pypa/manylinux).
|
||||
|
||||
This section summarizes the steps needed to build `manylinux`-compatible wheels using the aforementioned containers.
|
||||
|
||||
|
||||
### Building the Docker containers
|
||||
|
||||
The primary TREXIO back end at the moment is based on the [HDF5](https://portal.hdfgroup.org/display/HDF5/HDF5) library.
|
||||
Because of this, one has to build new Docker images from the PyPA ones.
|
||||
These images will contain the HDF5 library and will be used to produce wheels for the Python API.
|
||||
|
||||
To build the Docker images on x86_64 platform, execute the following:
|
||||
|
||||
`cd hdf5-on-docker/ && ./build_images.sh`
|
||||
|
||||
This should produce several images tagged with `hdf5_1_12_` prefix and `_x86_64` suffix.
|
||||
These images inherit from the PyPA manylinux containers.
|
||||
All available images can be listed using `docker image list` command.
|
||||
|
||||
|
||||
### Building the manylinux wheels
|
||||
|
||||
First, make sure that you have the source code distribution
|
||||
(e.g. `trexio-0.2.0.tar.gz`) of the Python API in the current directory.
|
||||
Then run one of the previously produced containers in the interactive mode using the following command:
|
||||
|
||||
``docker run --rm -it -e PLAT=manylinux2014_x86_64 -v `pwd`:/tmp hdf5_1_12_on_2014_x86_64 /bin/bash``
|
||||
|
||||
where `2014_x86_64` can be replaced with any other available platform suffix (e.g. `2010_x86_64` or `2_24_x86_64`)
|
||||
The `docker run` command line arguments used here:
|
||||
|
||||
- `-i` (run the docker container in interactive mode)
|
||||
- `--rm` (remove the container upon exit)
|
||||
- ``-v `pwd`:/tmp`` (mount current directory into the `tmp` directory on the container
|
||||
- `-e` (set the environment variables according to the provided list)
|
||||
- `hdf5_1_12_on_2014_x86_64` (name of the Docker image to run)
|
||||
- `/bin/bash` (which binary the container should execute)
|
||||
|
||||
Once the Docker container is launched and the corresponding prompt is switched to the container, run the following (example for release 0.2.0):
|
||||
|
||||
`cd tmp/ && ./build_manylinux_wheels.sh trexio-0.2.0.tar.gz`
|
||||
|
||||
This script creates virtual envirionments for several versions of `CPython` (e.g. 3.7, 3.8, 3.9),
|
||||
installs all requirements,
|
||||
produces `trexio` wheels with conventional `linux_x86_64` tag and
|
||||
repairs them to be tagged with `manylinux` using the `auditwheel` tool from PyPA
|
||||
|
||||
The produced wheels with the `manylinux` platfrom tag can be found in `trexio-0.2.0/wheelhouse` directory.
|
||||
|
||||
|
||||
### Exporting the wheels
|
||||
|
||||
You may want to produce wheels for several versions of `glibc`.
|
||||
Prior to running a new docker container, make sure to rename/move the `wheelhouse` directory.
|
||||
Otherwise, the `build_manylinux_wheels.sh` script will crush.
|
||||
|
||||
95
docker/build_manylinux_wheels.sh
Executable file
95
docker/build_manylinux_wheels.sh
Executable file
@ -0,0 +1,95 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -x
|
||||
set -e
|
||||
|
||||
export H5_LDFLAGS=-L/usr/local/lib
|
||||
export H5_CFLAGS=-I/usr/local/include
|
||||
|
||||
|
||||
# build wheel directly from developer-provided .tar.gz of TREXIO (generated with `python setup.py sdist`)
|
||||
# note: trexio-VERSION.tar.gz has to be in the root directory of the host machine and provided as an argument to this script
|
||||
|
||||
# process input: first argument is the name of the .tar.gz with the source code of the Python API
|
||||
if [[ -z "$1" ]]; then
|
||||
echo "Please specify the name of the TREXIO source code distribution (with .tar.gz suffix)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# derive TREXIO version from the file name
|
||||
TREXIO_SOURCE=${1}
|
||||
# remove prefix that ends with "-"
|
||||
tmp=${TREXIO_SOURCE#*-}
|
||||
# remove suffix that ends with ".tar.gz"
|
||||
TR_VERSION=${tmp%.tar.gz*}
|
||||
# print the computed version
|
||||
echo "TREXIO VERSION:" ${TR_VERSION}
|
||||
|
||||
# unzip and enter the folder with TREXIO Python API
|
||||
gzip -cd /tmp/trexio-${TR_VERSION}.tar.gz | tar xvf -
|
||||
cd trexio-${TR_VERSION}
|
||||
|
||||
# the function below build manylinux wheels based on the provided version of python (e.g. build_wheel_for_py 36)
|
||||
function build_wheel_for_py()
|
||||
{
|
||||
|
||||
if [[ -z "$1" ]]; then
|
||||
echo "Empty string provided instead of the Python version"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# derive PYVERSION from the input argument
|
||||
PYVERSION=${1}
|
||||
|
||||
# python versions <= 3.7 required additional "m" in the platform tag, e.g. cp37-cp37m
|
||||
if [[ ${PYVERSION} -eq 36 ]] || [[ ${PYVERSION} -eq 37 ]]; then
|
||||
PYM="m"
|
||||
else
|
||||
PYM=""
|
||||
fi
|
||||
|
||||
CPYTHON="cp${PYVERSION}-cp${PYVERSION}${PYM}"
|
||||
|
||||
# create and activate a virtual environment based on CPython version ${PYVERSION}
|
||||
/opt/python/${CPYTHON}/bin/python3 -m venv --clear trexio-manylinux-py${PYVERSION}
|
||||
source trexio-manylinux-py${PYVERSION}/bin/activate
|
||||
python3 --version
|
||||
|
||||
# upgrade pip, otherwise it complains that manylinux wheel is "...not supported wheel on this platform"
|
||||
pip install --upgrade pip
|
||||
# install dependencies needed to build manylinux wheel
|
||||
pip install --upgrade setuptools wheel auditwheel numpy
|
||||
|
||||
# set an environment variable needed to locate numpy header files
|
||||
source tools/set_NUMPY_INCLUDEDIR.sh
|
||||
|
||||
# produce conventional (non-manylinux) wheel
|
||||
python3 setup.py bdist_wheel
|
||||
|
||||
# use auditwheel from PyPA to repair all wheels and make them manylinux-compatible
|
||||
auditwheel repair dist/trexio-${TR_VERSION}-${CPYTHON}-*.whl
|
||||
|
||||
# install the produced manylinux wheel in the virtual environment
|
||||
python3 -m pip install wheelhouse/trexio-${TR_VERSION}-${CPYTHON}-manylinux*.whl
|
||||
|
||||
# run test script
|
||||
cd test && python3 test_api.py && cd ..
|
||||
|
||||
# cleaning
|
||||
rm -rf -- dist/ build/ trexio.egg-info/
|
||||
|
||||
# deactivate the current environment
|
||||
deactivate
|
||||
|
||||
# remove the virtual environment
|
||||
rm -rf -- trexio-manylinux-py${PYVERSION}
|
||||
|
||||
}
|
||||
|
||||
|
||||
# build wheels for all versions of CPython in this container
|
||||
for CPYVERSION in 36 37 38 39
|
||||
do
|
||||
build_wheel_for_py ${CPYVERSION}
|
||||
done
|
||||
|
||||
@ -1,188 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -x
|
||||
set -e
|
||||
|
||||
export H5_LDFLAGS=-L/usr/local/lib
|
||||
export H5_CFLAGS=-I/usr/local/include
|
||||
|
||||
# install emacs for Debian
|
||||
|
||||
#apt-get update
|
||||
#apt install software-properties-common -y
|
||||
#apt-get install wget -y
|
||||
|
||||
#wget -q http://emacs.ganneff.de/apt.key -O- | apt-key add
|
||||
|
||||
#add-apt-repository "deb http://emacs.ganneff.de/ stretch main"
|
||||
#apt-get update
|
||||
#apt-get install emacs-snapshot -y
|
||||
#update-alternatives --config emacsclient
|
||||
|
||||
# ===============================
|
||||
|
||||
# install TREXIO in the container from the GitHub repo clone
|
||||
|
||||
#apt-get install git -y
|
||||
|
||||
#git clone https://github.com/TREX-CoE/trexio.git
|
||||
#cd trexio
|
||||
#git checkout swig-python
|
||||
|
||||
#./autogen.sh
|
||||
#TREXIO_DEVEL=1 ./configure --enable-silent-rules
|
||||
#make
|
||||
#make check
|
||||
|
||||
# ===============================
|
||||
|
||||
# alternatively: build wheel directly from developer-provided .tar.gz of TREXIO (generated with `python setup.py sdist`)
|
||||
# note: trexio-VERSION.tar.gz has to be in the root directory of the host machine
|
||||
|
||||
# process input: first argument is the name of the .tar.gz with the source code of the Python API
|
||||
if [[ -z "$1" ]]; then
|
||||
echo "Please specify the name of the TREXIO source code distribution (with .tar.gz suffix)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
TREXIO_SOURCE=${1}
|
||||
|
||||
# remove prefix that ends with "-"
|
||||
tmp=${TREXIO_SOURCE#*-}
|
||||
# remove suffix that ends with ".tar.gz"
|
||||
TR_VERSION=${tmp%.tar.gz*}
|
||||
|
||||
echo "TREXIO VERSION:" ${TR_VERSION}
|
||||
|
||||
# unzip and enter the folder with TREXIO Python API
|
||||
gzip -cd /tmp/trexio-${TR_VERSION}.tar.gz | tar xvf -
|
||||
cd trexio-${TR_VERSION}
|
||||
|
||||
# create and activate a virtual environment based on CPython version 3.6
|
||||
/opt/python/cp36-cp36m/bin/python3 -m venv --clear trexio-manylinux-py36
|
||||
source trexio-manylinux-py36/bin/activate
|
||||
python3 --version
|
||||
|
||||
# upgrade pip, otherwise it complains that manylinux wheel is "...not supported wheel on this platform"
|
||||
pip install --upgrade pip
|
||||
# install dependencies needed to build manylinux wheel
|
||||
pip install --upgrade setuptools wheel auditwheel numpy
|
||||
|
||||
# set an environment variable needed to locate numpy header files
|
||||
source tools/set_NUMPY_INCLUDEDIR.sh
|
||||
|
||||
# produce conventional (non-manylinux) wheel
|
||||
python3 setup.py bdist_wheel
|
||||
|
||||
# use auditwheel from PyPA to repair all wheels and make them manylinux-compatible
|
||||
auditwheel repair dist/trexio-${TR_VERSION}-cp36-cp36m-*.whl
|
||||
|
||||
# install the produced manylinux wheel in the virtual environment
|
||||
python3 -m pip install wheelhouse/trexio-${TR_VERSION}-cp36-cp36m-manylinux*.whl
|
||||
|
||||
# run test script
|
||||
cd test && python3 test_api.py && cd ..
|
||||
|
||||
# cleaning
|
||||
rm -rf -- dist/ build/ trexio.egg-info/
|
||||
|
||||
# deactivate the current environment
|
||||
deactivate
|
||||
|
||||
# create and activate a virtual environment based on CPython version 3.7
|
||||
/opt/python/cp37-cp37m/bin/python3 -m venv --clear trexio-manylinux-py37
|
||||
source trexio-manylinux-py37/bin/activate
|
||||
python3 --version
|
||||
|
||||
# upgrade pip, otherwise it complains that manylinux wheel is "...not supported wheel on this platform"
|
||||
pip install --upgrade pip
|
||||
# install dependencies needed to build manylinux wheel
|
||||
pip install --upgrade setuptools wheel auditwheel numpy
|
||||
|
||||
# set an environment variable needed to locate numpy header files
|
||||
source tools/set_NUMPY_INCLUDEDIR.sh
|
||||
|
||||
# produce conventional (non-manylinux) wheel
|
||||
python3 setup.py bdist_wheel
|
||||
|
||||
# use auditwheel from PyPA to repair all wheels and make them manylinux-compatible
|
||||
auditwheel repair dist/trexio-${TR_VERSION}-cp37-cp37m-*.whl
|
||||
|
||||
# install the produced manylinux wheel in the virtual environment
|
||||
python3 -m pip install wheelhouse/trexio-${TR_VERSION}-cp37-cp37m-manylinux*.whl
|
||||
|
||||
# run test script
|
||||
cd test && python3 test_api.py && cd ..
|
||||
|
||||
# cleaning
|
||||
rm -rf -- dist/ build/ trexio.egg-info/
|
||||
|
||||
# deactivate the current environment
|
||||
deactivate
|
||||
|
||||
# create and activate a virtual environment based on CPython version 3.8
|
||||
# NOTE: starting from CPython 3.8 there is no need to add m in the abi-tag, e.g. use cp38-cp38 instead of cp38-cp38m
|
||||
/opt/python/cp38-cp38/bin/python3 -m venv --clear trexio-manylinux-py38
|
||||
source trexio-manylinux-py38/bin/activate
|
||||
python3 --version
|
||||
|
||||
# upgrade pip, otherwise it complains that manylinux wheel is "...not supported wheel on this platform"
|
||||
pip install --upgrade pip
|
||||
# install dependencies needed to build manylinux wheel
|
||||
pip3 install --upgrade setuptools wheel auditwheel numpy
|
||||
|
||||
# set an environment variable needed to locate numpy header files
|
||||
source tools/set_NUMPY_INCLUDEDIR.sh
|
||||
|
||||
# produce conventional (non-manylinux) wheel
|
||||
python3 setup.py bdist_wheel
|
||||
|
||||
# use auditwheel from PyPA to repair all wheels and make them manylinux-compatible
|
||||
auditwheel repair dist/trexio-${TR_VERSION}-cp38-cp38-*.whl
|
||||
|
||||
# install the produced manylinux wheel in the virtual environment
|
||||
python3 -m pip install wheelhouse/trexio-${TR_VERSION}-cp38-cp38-manylinux*.whl
|
||||
|
||||
# run test script
|
||||
cd test && python3 test_api.py && cd ..
|
||||
|
||||
# cleaning
|
||||
rm -rf -- dist/ build/ trexio.egg-info/
|
||||
|
||||
# deactivate the current environment
|
||||
deactivate
|
||||
|
||||
# create and activate a virtual environment based on CPython version 3.8
|
||||
/opt/python/cp39-cp39/bin/python3 -m venv --clear trexio-manylinux-py39
|
||||
source trexio-manylinux-py39/bin/activate
|
||||
python3 --version
|
||||
|
||||
# upgrade pip, otherwise it complains that manylinux wheel is "...not supported wheel on this platform"
|
||||
pip install --upgrade pip
|
||||
# install dependencies needed to build manylinux wheel
|
||||
pip3 install --upgrade setuptools wheel auditwheel numpy
|
||||
|
||||
# produce conventional (non-manylinux) wheel
|
||||
python3 setup.py bdist_wheel
|
||||
|
||||
# use auditwheel from PyPA to repair all wheels and make them manylinux-compatible
|
||||
auditwheel repair dist/trexio-${TR_VERSION}-cp39-cp39-*.whl
|
||||
|
||||
# install the produced manylinux wheel in the virtual environment
|
||||
python3 -m pip install wheelhouse/trexio-${TR_VERSION}-cp39-cp39-manylinux*.whl
|
||||
|
||||
# run test script
|
||||
cd test && python3 test_api.py && cd ..
|
||||
|
||||
# cleaning
|
||||
rm -rf -- dist/ build/ trexio.egg-info/
|
||||
|
||||
# deactivate the current environment
|
||||
deactivate
|
||||
|
||||
# remove all virtual environments used to produce the wheels
|
||||
rm -rf -- trexio-manylinux-py39 \
|
||||
trexio-manylinux-py38 \
|
||||
trexio-manylinux-py37 \
|
||||
trexio-manylinux-py36
|
||||
|
||||
15
docker/hdf5-on-docker/Dockerfile_2010_x86_64
Normal file
15
docker/hdf5-on-docker/Dockerfile_2010_x86_64
Normal file
@ -0,0 +1,15 @@
|
||||
# ===========================================================================
|
||||
# Script from the hdf5-manylinux repo:
|
||||
# https://github.com/h5py/hdf5-manylinux/blob/master/Dockerfile_x86_64
|
||||
# ===========================================================================
|
||||
|
||||
FROM quay.io/pypa/manylinux2010_x86_64
|
||||
|
||||
ENV HDF5_DIR /usr/local
|
||||
ENV HDF5_VERSION 1.12.1
|
||||
|
||||
COPY install_libaec.sh /tmp/install_libaec.sh
|
||||
RUN bash /tmp/install_libaec.sh
|
||||
COPY install_hdf5_centos.sh /tmp/install_hdf5.sh
|
||||
RUN bash /tmp/install_hdf5.sh
|
||||
|
||||
15
docker/hdf5-on-docker/Dockerfile_2014_aarch64
Normal file
15
docker/hdf5-on-docker/Dockerfile_2014_aarch64
Normal file
@ -0,0 +1,15 @@
|
||||
# ===========================================================================
|
||||
# Script from the hdf5-manylinux repo:
|
||||
# https://github.com/h5py/hdf5-manylinux/blob/master/Dockerfile_aarch64
|
||||
# ===========================================================================
|
||||
|
||||
FROM quay.io/pypa/manylinux2014_aarch64
|
||||
|
||||
ENV HDF5_DIR /usr/local
|
||||
ENV HDF5_VERSION 1.12.1
|
||||
|
||||
COPY install_libaec.sh /tmp/install_libaec.sh
|
||||
RUN bash /tmp/install_libaec.sh
|
||||
COPY install_hdf5_centos.sh /tmp/install_hdf5.sh
|
||||
RUN bash /tmp/install_hdf5.sh
|
||||
|
||||
10
docker/hdf5-on-docker/Dockerfile_2014_x86_64
Normal file
10
docker/hdf5-on-docker/Dockerfile_2014_x86_64
Normal file
@ -0,0 +1,10 @@
|
||||
FROM quay.io/pypa/manylinux2014_x86_64
|
||||
|
||||
ENV HDF5_DIR /usr/local
|
||||
ENV HDF5_VERSION 1.12.1
|
||||
|
||||
COPY install_libaec.sh /tmp/install_libaec.sh
|
||||
RUN bash /tmp/install_libaec.sh
|
||||
COPY install_hdf5_centos.sh /tmp/install_hdf5.sh
|
||||
RUN bash /tmp/install_hdf5.sh
|
||||
|
||||
10
docker/hdf5-on-docker/Dockerfile_2_24_aarch64
Normal file
10
docker/hdf5-on-docker/Dockerfile_2_24_aarch64
Normal file
@ -0,0 +1,10 @@
|
||||
FROM quay.io/pypa/manylinux_2_24_aarch64
|
||||
|
||||
ENV HDF5_DIR /usr/local
|
||||
ENV HDF5_VERSION 1.12.1
|
||||
|
||||
COPY install_libaec.sh /tmp/install_libaec.sh
|
||||
RUN bash /tmp/install_libaec.sh
|
||||
COPY install_hdf5_debian.sh /tmp/install_hdf5.sh
|
||||
RUN bash /tmp/install_hdf5.sh
|
||||
|
||||
10
docker/hdf5-on-docker/Dockerfile_2_24_x86_64
Normal file
10
docker/hdf5-on-docker/Dockerfile_2_24_x86_64
Normal file
@ -0,0 +1,10 @@
|
||||
FROM quay.io/pypa/manylinux_2_24_x86_64
|
||||
|
||||
ENV HDF5_DIR /usr/local
|
||||
ENV HDF5_VERSION 1.12.1
|
||||
|
||||
COPY install_libaec.sh /tmp/install_libaec.sh
|
||||
RUN bash /tmp/install_libaec.sh
|
||||
COPY install_hdf5_debian.sh /tmp/install_hdf5.sh
|
||||
RUN bash /tmp/install_hdf5.sh
|
||||
|
||||
16
docker/hdf5-on-docker/build_images.sh
Executable file
16
docker/hdf5-on-docker/build_images.sh
Executable file
@ -0,0 +1,16 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -x
|
||||
set -e
|
||||
|
||||
# currently cannot build ARM64 images on the AMD64 platform due to the error:
|
||||
# standard_init_linux.go:219: exec user process caused: exec format error
|
||||
#for tag in 2010_x86_64 2014_x86_64 2_24_x86_64 2014_aarch64 2_24_aarch64
|
||||
|
||||
# build only x86_64 images
|
||||
for tag in 2010_x86_64 2014_x86_64 2_24_x86_64
|
||||
do
|
||||
echo "Building image on ${tag} platform"
|
||||
docker build -t hdf5_1_12_on_${tag} . -f Dockerfile_${tag}
|
||||
done
|
||||
|
||||
34
docker/hdf5-on-docker/install_hdf5_centos.sh
Normal file
34
docker/hdf5-on-docker/install_hdf5_centos.sh
Normal file
@ -0,0 +1,34 @@
|
||||
# ===========================================================================
|
||||
# Script from the hdf5-manylinux repo:
|
||||
# https://github.com/h5py/hdf5-manylinux/blob/master/install_hdf5.sh
|
||||
# ===========================================================================
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
echo "Installing zlib with yum"
|
||||
yum -y install zlib-devel
|
||||
|
||||
pushd /tmp
|
||||
|
||||
# This seems to be needed to find libsz.so.2
|
||||
ldconfig
|
||||
|
||||
echo "Downloading & unpacking HDF5 ${HDF5_VERSION}"
|
||||
# Remove trailing .*, to get e.g. '1.12' ↓
|
||||
curl -fsSLO "https://www.hdfgroup.org/ftp/HDF5/releases/hdf5-${HDF5_VERSION%.*}/hdf5-$HDF5_VERSION/src/hdf5-$HDF5_VERSION.tar.gz"
|
||||
tar -xzvf hdf5-$HDF5_VERSION.tar.gz
|
||||
pushd hdf5-$HDF5_VERSION
|
||||
chmod u+x autogen.sh
|
||||
|
||||
echo "Configuring, building & installing HDF5 ${HDF5_VERSION} to ${HDF5_DIR}"
|
||||
./configure --prefix $HDF5_DIR --enable-build-mode=production --with-szlib
|
||||
make -j $(nproc)
|
||||
make install
|
||||
popd
|
||||
|
||||
# Clean up to limit the size of the Docker image
|
||||
echo "Cleaning up unnecessary files"
|
||||
rm -r hdf5-$HDF5_VERSION
|
||||
rm hdf5-$HDF5_VERSION.tar.gz
|
||||
|
||||
yum -y erase zlib-devel
|
||||
32
docker/hdf5-on-docker/install_hdf5_debian.sh
Normal file
32
docker/hdf5-on-docker/install_hdf5_debian.sh
Normal file
@ -0,0 +1,32 @@
|
||||
# ===========================================================================
|
||||
# Debian-adapted script from the hdf5-manylinux repo:
|
||||
# https://github.com/h5py/hdf5-manylinux/blob/master/install_hdf5.sh
|
||||
# ===========================================================================
|
||||
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
pushd /tmp
|
||||
|
||||
# This seems to be needed to find libsz.so.2
|
||||
ldconfig
|
||||
|
||||
echo "Downloading & unpacking HDF5 ${HDF5_VERSION}"
|
||||
# Remove trailing .*, to get e.g. '1.12' ↓
|
||||
curl -fsSLO "https://www.hdfgroup.org/ftp/HDF5/releases/hdf5-${HDF5_VERSION%.*}/hdf5-$HDF5_VERSION/src/hdf5-$HDF5_VERSION.tar.gz"
|
||||
tar -xzvf hdf5-$HDF5_VERSION.tar.gz
|
||||
pushd hdf5-$HDF5_VERSION
|
||||
chmod u+x autogen.sh
|
||||
|
||||
echo "Configuring, building & installing HDF5 ${HDF5_VERSION} to ${HDF5_DIR}"
|
||||
./configure --prefix $HDF5_DIR --enable-build-mode=production --with-szlib
|
||||
make -j $(nproc)
|
||||
make install
|
||||
popd
|
||||
|
||||
# Clean up to limit the size of the Docker image
|
||||
echo "Cleaning up unnecessary files"
|
||||
rm -r hdf5-$HDF5_VERSION
|
||||
rm hdf5-$HDF5_VERSION.tar.gz
|
||||
|
||||
apt-get -y remove zlib1g-dev
|
||||
28
docker/hdf5-on-docker/install_libaec.sh
Normal file
28
docker/hdf5-on-docker/install_libaec.sh
Normal file
@ -0,0 +1,28 @@
|
||||
# ===========================================================================
|
||||
# Script from the hdf5-manylinux repo:
|
||||
# https://github.com/h5py/hdf5-manylinux/blob/master/install_libaec.sh
|
||||
# ===========================================================================
|
||||
|
||||
|
||||
# libaec implements szip compression, so the optional szip filter can be built
|
||||
# in HDF5.
|
||||
set -euo pipefail
|
||||
|
||||
pushd /tmp
|
||||
|
||||
aec_version="1.0.4"
|
||||
|
||||
echo "Downloading libaec"
|
||||
# The URL includes a hash, so it needs to change if the version does
|
||||
curl -fsSLO https://gitlab.dkrz.de/k202009/libaec/uploads/ea0b7d197a950b0c110da8dfdecbb71f/libaec-${aec_version}.tar.gz
|
||||
tar zxf libaec-$aec_version.tar.gz
|
||||
|
||||
echo "Building & installing libaec"
|
||||
pushd libaec-$aec_version
|
||||
./configure
|
||||
make
|
||||
make install
|
||||
|
||||
# Clean up the files from the build
|
||||
popd
|
||||
rm -r libaec-$aec_version libaec-$aec_version.tar.gz
|
||||
176
docs/tutorial_benzene.html
Normal file
176
docs/tutorial_benzene.html
Normal file
@ -0,0 +1,176 @@
|
||||
<h1 id="trexio-tutorial">TREXIO Tutorial</h1>
|
||||
<p>This interactive Tutorial covers some basic use cases of the TREXIO library based on the Python API. At this point, it is assumed that the TREXIO Python package has been sucessfully installed on the user machine or in the virtual environment. If this is not the case, feel free to follow the <a href="https://github.com/TREX-CoE/trexio/blob/master/python/README.md">installation guide</a>.</p>
|
||||
<h2 id="importing-trexio">Importing TREXIO</h2>
|
||||
<p>First of all, let’s import the TREXIO package.</p>
|
||||
<div class="sourceCode" id="cb1"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb1-1" title="1"><span class="cf">try</span>:</a>
|
||||
<a class="sourceLine" id="cb1-2" title="2"> <span class="im">import</span> trexio</a>
|
||||
<a class="sourceLine" id="cb1-3" title="3"><span class="cf">except</span> <span class="pp">ImportError</span>:</a>
|
||||
<a class="sourceLine" id="cb1-4" title="4"> <span class="cf">raise</span> <span class="pp">Exception</span>(<span class="st">"Unable to import trexio. Please check that trexio is properly instaled."</span>)</a></code></pre></div>
|
||||
<p>If no error occurs, then it means that the TREXIO package has been sucessfully imported. Within the current import, TREXIO attributes can be accessed using the corresponding <code>trexio.attribute</code> notation. If you prefer to bound a shorter name to the imported module (as commonly done by the NumPy users with <code>import numpy as np</code>), this is also possible. To do so, replace <code>import trexio</code> with <code>import trexio as tr</code> for example. To learn more about importing modules, see the corresponding page of the <a href="https://docs.python.org/3/tutorial/modules.html#more-on-modules">Python documentation</a>.</p>
|
||||
<h2 id="creating-a-new-trexio-file">Creating a new TREXIO file</h2>
|
||||
<p>TREXIO currently supports two back ends for file I/O:</p>
|
||||
<ol type="1">
|
||||
<li><p><code>TREXIO_HDF5</code>, which relies on extensive use of the <a href="https://portal.hdfgroup.org/display/HDF5/HDF5">HDF5 library</a> and the associated binary file format. This back end is optimized for high performance but it requires HDF5 to be installed on the user machine.</p></li>
|
||||
<li><p><code>TREXIO_TEXT</code>, which relies on basic I/O operations that are available in the standard C library. This back end is not optimized for performance but it is supposed to work “out-of-the-box” since there are no external dependencies.</p></li>
|
||||
</ol>
|
||||
<p>Armed with these new definitions, let’s proceed with the tutorial. The first task is to create a TREXIO file called <code>benzene_demo.h5</code>. But first we have to remove the file if it exists in the current directory</p>
|
||||
<div class="sourceCode" id="cb2"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb2-1" title="1">filename <span class="op">=</span> <span class="st">'benzene_demo.h5'</span></a>
|
||||
<a class="sourceLine" id="cb2-2" title="2"></a>
|
||||
<a class="sourceLine" id="cb2-3" title="3"><span class="im">import</span> os</a>
|
||||
<a class="sourceLine" id="cb2-4" title="4"><span class="cf">try</span>:</a>
|
||||
<a class="sourceLine" id="cb2-5" title="5"> os.remove(filename)</a>
|
||||
<a class="sourceLine" id="cb2-6" title="6"><span class="cf">except</span>:</a>
|
||||
<a class="sourceLine" id="cb2-7" title="7"> <span class="bu">print</span>(<span class="ss">f"File </span><span class="sc">{</span>filename<span class="sc">}</span><span class="ss"> does not exist."</span>)</a></code></pre></div>
|
||||
<pre><code>File benzene_demo.h5 does not exist.</code></pre>
|
||||
<p>We are now ready to create a new TREXIO file:</p>
|
||||
<div class="sourceCode" id="cb4"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb4-1" title="1">demo_file <span class="op">=</span> trexio.File(filename, mode<span class="op">=</span><span class="st">'w'</span>, back_end<span class="op">=</span>trexio.TREXIO_HDF5)</a></code></pre></div>
|
||||
<p>This creates an instance of the <code>trexio.File</code> class, which we refer to as <code>demo_file</code> in this tutorial. You can check that the corresponding file called <code>benzene_demo.h5</code> exists in the root directory. It is now open for writing as indicated by the user-supplied argument <code>mode='w'</code>. The file has been initiated using <code>TREXIO_HDF5</code> back end and will be accessed accordingly from now on. The information about back end is stored internally by TREXIO, which means that there is no need to specify it every time the I/O operation is performed. If the file named <code>benzene_demo.h5</code> already exists, then it is re-opened for writing (and not truncated to prevent data loss).</p>
|
||||
<h2 id="writing-data-in-the-trexio-file">Writing data in the TREXIO file</h2>
|
||||
<p>Prior to any work with TREXIO library, we highly recommend users to read about <a href="https://trex-coe.github.io/trexio/trex.html">TREXIO internal configuration</a>, which explains the structure of the wavefunction file. The reason is that TREXIO API has a naming convention, which is based on the groups and variables names that are pre-defined by the developers. In this Tutorial, we will only cover contents of the <code>nucleus</code> group. Note that custom groups and variables can be added to the TREXIO API.</p>
|
||||
<p>In this Tutorial, we consider benzene molecule (C6H6) as an example. Since benzene has 12 atoms, let’s specify it in the previously created <code>demo_file</code>. In order to do so, one has to call <code>trexio.write_nucleus_num</code> function, which accepts an instance of the <code>trexio.File</code> class as a first argument and an <code>int</code> value corresponding to the number of nuclei as a second argument.</p>
|
||||
<div class="sourceCode" id="cb5"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb5-1" title="1">nucleus_num <span class="op">=</span> <span class="dv">12</span></a></code></pre></div>
|
||||
<div class="sourceCode" id="cb6"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb6-1" title="1">trexio.write_nucleus_num(demo_file, nucleus_num)</a></code></pre></div>
|
||||
<p>In fact, all API functions that contain <code>write_</code> prefix can be used in a similar way. Variables that contain <code>_num</code> suffix are important part of the TREXIO file because some of them define dimensions of arrays. For example, <code>nucleus_num</code> variable corresponds to the number of atoms, which will be internally used to write/read the <code>nucleus_coord</code> array of nuclear coordinates. In order for TREXIO files to be self-consistent, overwriting num-suffixed variables is currently disabled.</p>
|
||||
<p>The number of atoms is not sufficient to define a molecule. Let’s first create a list of nuclear charges, which correspond to benzene.</p>
|
||||
<div class="sourceCode" id="cb7"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb7-1" title="1">charges <span class="op">=</span> [<span class="fl">6.</span>, <span class="fl">6.</span>, <span class="fl">6.</span>, <span class="fl">6.</span>, <span class="fl">6.</span>, <span class="fl">6.</span>, <span class="fl">1.</span>, <span class="fl">1.</span>, <span class="fl">1.</span>, <span class="fl">1.</span>, <span class="fl">1.</span>, <span class="fl">1.</span>]</a></code></pre></div>
|
||||
<p>According to the TREX configuration file, there is a <code>charge</code> attribute of the <code>nucleus</code> group, which has <code>float</code> type and <code>[nucleus_num]</code> dimension. The <code>charges</code> list defined above fits nicely in the description and can be written as follows</p>
|
||||
<div class="sourceCode" id="cb8"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb8-1" title="1">trexio.write_nucleus_charge(demo_file, charges)</a></code></pre></div>
|
||||
<p><strong>Note: TREXIO function names only contain parts in singular form.</strong> This means that, both <code>write_nucleus_charges</code> and <code>write_nuclear_charges</code> are invalid API calls. These functions simply do not exist in the <code>trexio</code> Python package and the corresponding error message should appear.</p>
|
||||
<p>Alternatively, one can provide a list of nuclear labels (chemical elements from the periodic table) that correspond to the aforementioned charges. There is a <code>label</code> attribute of the <code>nucleus</code> group, which has <code>str</code> type and <code>[nucleus_num]</code> dimension. Let’s create a list of 12 strings, which correspond to 6 carbon and 6 hydrogen atoms:</p>
|
||||
<div class="sourceCode" id="cb9"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb9-1" title="1">labels <span class="op">=</span> [</a>
|
||||
<a class="sourceLine" id="cb9-2" title="2"> <span class="st">'C'</span>,</a>
|
||||
<a class="sourceLine" id="cb9-3" title="3"> <span class="st">'C'</span>,</a>
|
||||
<a class="sourceLine" id="cb9-4" title="4"> <span class="st">'C'</span>,</a>
|
||||
<a class="sourceLine" id="cb9-5" title="5"> <span class="st">'C'</span>,</a>
|
||||
<a class="sourceLine" id="cb9-6" title="6"> <span class="st">'C'</span>,</a>
|
||||
<a class="sourceLine" id="cb9-7" title="7"> <span class="st">'C'</span>,</a>
|
||||
<a class="sourceLine" id="cb9-8" title="8"> <span class="st">'H'</span>,</a>
|
||||
<a class="sourceLine" id="cb9-9" title="9"> <span class="st">'H'</span>,</a>
|
||||
<a class="sourceLine" id="cb9-10" title="10"> <span class="st">'H'</span>,</a>
|
||||
<a class="sourceLine" id="cb9-11" title="11"> <span class="st">'H'</span>,</a>
|
||||
<a class="sourceLine" id="cb9-12" title="12"> <span class="st">'H'</span>,</a>
|
||||
<a class="sourceLine" id="cb9-13" title="13"> <span class="st">'H'</span>]</a></code></pre></div>
|
||||
<p>This can now be written using the corresponding <code>trexio.write_nucleus_label</code> function:</p>
|
||||
<div class="sourceCode" id="cb10"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb10-1" title="1">trexio.write_nucleus_label(demo_file, labels)</a></code></pre></div>
|
||||
<p>Two examples above demonstrate how to write arrays of numbers or strings in the file. TREXIO also supports I/O operations on single numerical or string attributes. In fact, in this Tutorial you have already written one numerical attribute: <code>nucleus_num</code>. Let’s now write a string <code>'D6h'</code>, which indicates a point group of benzene molecule. According to the TREX configuration file, <code>point_group</code> is a <code>str</code> attribute of the <code>nucleus</code> group, thus it can be written in the <code>demo_file</code> as follows</p>
|
||||
<div class="sourceCode" id="cb11"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb11-1" title="1">point_group <span class="op">=</span> <span class="st">'D6h'</span></a></code></pre></div>
|
||||
<div class="sourceCode" id="cb12"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb12-1" title="1">trexio.write_nucleus_point_group(demo_file, point_group)</a></code></pre></div>
|
||||
<h3 id="writing-numpy-arrays-float-or-int-types">Writing NumPy arrays (float or int types)</h3>
|
||||
<p>The aforementioned examples cover the majority of the currently implemented functionality related to writing data in the file. It is worth mentioning that I/O of numerical arrays in TREXIO Python API relies on extensive use of the <a href="https://numpy.org/">NumPy package</a>. This will be discussed in more details in the <a href="#Reading-data-from-the-TREXIO-file">section about reading data</a>. However, TREXIO <code>write_</code> functions that work with numerical arrays also accept <code>numpy.ndarray</code> objects. For example, consider a <code>coords</code> list of nuclear coordinates that correspond to benzene molecule</p>
|
||||
<div class="sourceCode" id="cb13"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb13-1" title="1">coords <span class="op">=</span> [</a>
|
||||
<a class="sourceLine" id="cb13-2" title="2"> [<span class="fl">0.00000000</span> , <span class="fl">1.39250319</span> , <span class="fl">0.00000000</span> ],</a>
|
||||
<a class="sourceLine" id="cb13-3" title="3"> [<span class="op">-</span><span class="fl">1.20594314</span> , <span class="fl">0.69625160</span> , <span class="fl">0.00000000</span> ],</a>
|
||||
<a class="sourceLine" id="cb13-4" title="4"> [<span class="op">-</span><span class="fl">1.20594314</span> , <span class="fl">-0.69625160</span> , <span class="fl">0.00000000</span> ],</a>
|
||||
<a class="sourceLine" id="cb13-5" title="5"> [<span class="fl">0.00000000</span> , <span class="fl">-1.39250319</span> , <span class="fl">0.00000000</span> ],</a>
|
||||
<a class="sourceLine" id="cb13-6" title="6"> [<span class="fl">1.20594314</span> , <span class="fl">-0.69625160</span> , <span class="fl">0.00000000</span> ],</a>
|
||||
<a class="sourceLine" id="cb13-7" title="7"> [<span class="fl">1.20594314</span> , <span class="fl">0.69625160</span> , <span class="fl">0.00000000</span> ],</a>
|
||||
<a class="sourceLine" id="cb13-8" title="8"> [<span class="op">-</span><span class="fl">2.14171677</span> , <span class="fl">1.23652075</span> , <span class="fl">0.00000000</span> ],</a>
|
||||
<a class="sourceLine" id="cb13-9" title="9"> [<span class="op">-</span><span class="fl">2.14171677</span> , <span class="fl">-1.23652075</span> , <span class="fl">0.00000000</span> ],</a>
|
||||
<a class="sourceLine" id="cb13-10" title="10"> [<span class="fl">0.00000000</span> , <span class="fl">-2.47304151</span> , <span class="fl">0.00000000</span> ],</a>
|
||||
<a class="sourceLine" id="cb13-11" title="11"> [<span class="fl">2.14171677</span> , <span class="fl">-1.23652075</span> , <span class="fl">0.00000000</span> ],</a>
|
||||
<a class="sourceLine" id="cb13-12" title="12"> [<span class="fl">2.14171677</span> , <span class="fl">1.23652075</span> , <span class="fl">0.00000000</span> ],</a>
|
||||
<a class="sourceLine" id="cb13-13" title="13"> [<span class="fl">0.00000000</span> , <span class="fl">2.47304151</span> , <span class="fl">0.00000000</span> ],</a>
|
||||
<a class="sourceLine" id="cb13-14" title="14"> ]</a></code></pre></div>
|
||||
<p>Let’s take advantage of using NumPy arrays with fixed precision for floating point numbers. But first, try to import the <code>numpy</code> package</p>
|
||||
<div class="sourceCode" id="cb14"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb14-1" title="1"><span class="cf">try</span>:</a>
|
||||
<a class="sourceLine" id="cb14-2" title="2"> <span class="im">import</span> numpy <span class="im">as</span> np</a>
|
||||
<a class="sourceLine" id="cb14-3" title="3"><span class="cf">except</span> <span class="pp">ImportError</span>:</a>
|
||||
<a class="sourceLine" id="cb14-4" title="4"> <span class="cf">raise</span> <span class="pp">Exception</span>(<span class="st">"Unable to import numpy. Please check that numpy is properly instaled."</span>)</a></code></pre></div>
|
||||
<p>You can now convert the previously defined <code>coords</code> list into a numpy array with fixed <code>float64</code> type as follows</p>
|
||||
<div class="sourceCode" id="cb15"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb15-1" title="1">coords_np <span class="op">=</span> np.array(coords, dtype<span class="op">=</span>np.float64)</a></code></pre></div>
|
||||
<p>TREXIO functions that write numerical arrays accept both lists and numpy arrays as a second argument. That is, both <code>trexio.write_nucleus_coord(demo_file, coords)</code> and <code>trexio.write_nucleus_coord(demo_file, coords_np)</code> are valid API calls. Let’s use the latter and see if it works</p>
|
||||
<div class="sourceCode" id="cb16"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb16-1" title="1">trexio.write_nucleus_coord(demo_file, coords_np)</a></code></pre></div>
|
||||
<p>Congratulations, you have just completed the <code>nucleus</code> section of the TREXIO file for benzene molecule! Note that TREXIO API is rather permissive and do not impose any strict ordering on the I/O operations. The only requirement is that dimensioning (<code>_num</code> suffixed) variables have to be written in the file <strong>before</strong> writing arrays that depend on these variables. For example, attempting to write <code>nucleus_charge</code> or <code>nucleus_coord</code> fails if <code>nucleus_num</code> has not been written.</p>
|
||||
<h3 id="trexio-error-handling">TREXIO error handling</h3>
|
||||
<p>TREXIO Python API provides the <code>trexio.Error</code> class which simplifies exception handling in the Python scripts. This class wraps up TREXIO return codes and propagates them all the way from the C back end to the Python front end. Let’s try to write a negative number of basis set shells <code>basis_num</code> in the TREXIO file.</p>
|
||||
<div class="sourceCode" id="cb17"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb17-1" title="1"><span class="cf">try</span>:</a>
|
||||
<a class="sourceLine" id="cb17-2" title="2"> trexio.write_basis_num(demo_file, <span class="dv">-256</span>)</a>
|
||||
<a class="sourceLine" id="cb17-3" title="3"><span class="cf">except</span> trexio.Error <span class="im">as</span> e:</a>
|
||||
<a class="sourceLine" id="cb17-4" title="4"> <span class="bu">print</span>(<span class="ss">f"TREXIO error message: </span><span class="sc">{e.</span>message<span class="sc">}</span><span class="ss">"</span>)</a></code></pre></div>
|
||||
<pre><code>TREXIO error message: Invalid argument 2</code></pre>
|
||||
<p>The error message says <strong>Invalid argument 2</strong>, which indicates that the user-provided value <code>-256</code> is not valid.</p>
|
||||
<p>As mentioned before, <code>_num</code>-suffixed variables cannot be overwritten in the file. But what happens if you accidentally attempt to do so? Let’s have a look at the <code>write_nucleus_num</code> function as an example:</p>
|
||||
<div class="sourceCode" id="cb19"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb19-1" title="1"><span class="cf">try</span>:</a>
|
||||
<a class="sourceLine" id="cb19-2" title="2"> trexio.write_nucleus_num(demo_file, <span class="dv">24</span>)</a>
|
||||
<a class="sourceLine" id="cb19-3" title="3"><span class="cf">except</span> trexio.Error <span class="im">as</span> e:</a>
|
||||
<a class="sourceLine" id="cb19-4" title="4"> <span class="bu">print</span>(<span class="ss">f"TREXIO error message: </span><span class="sc">{e.</span>message<span class="sc">}</span><span class="ss">"</span>)</a></code></pre></div>
|
||||
<pre><code>TREXIO error message: Attribute already exists</code></pre>
|
||||
<p>The API rightfully complains that the target attribute already exists and cannot be overwritten.</p>
|
||||
<p>Alternatively, the aforementioned case can be handled using <code>trexio.has_nucleus_num</code> function as follows</p>
|
||||
<div class="sourceCode" id="cb21"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb21-1" title="1"><span class="cf">if</span> <span class="kw">not</span> trexio.has_nucleus_num:</a>
|
||||
<a class="sourceLine" id="cb21-2" title="2"> trexio.write_nucleus_num(demo_file, <span class="dv">24</span>)</a></code></pre></div>
|
||||
<p>TREXIO functions with <code>has_</code> prefix return <code>True</code> if the corresponding variable exists and <code>False</code> otherwise.</p>
|
||||
<p>What about writing arrays? Let’s try to write an list of 48 nuclear indices instead of 12</p>
|
||||
<div class="sourceCode" id="cb22"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb22-1" title="1">indices <span class="op">=</span> [i <span class="cf">for</span> i <span class="kw">in</span> <span class="bu">range</span>(nucleus_num<span class="op">*</span><span class="dv">4</span>)]</a></code></pre></div>
|
||||
<div class="sourceCode" id="cb23"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb23-1" title="1"><span class="cf">try</span>:</a>
|
||||
<a class="sourceLine" id="cb23-2" title="2"> trexio.write_basis_nucleus_index(demo_file, indices)</a>
|
||||
<a class="sourceLine" id="cb23-3" title="3"><span class="cf">except</span> trexio.Error <span class="im">as</span> e:</a>
|
||||
<a class="sourceLine" id="cb23-4" title="4"> <span class="bu">print</span>(<span class="ss">f"TREXIO error message: </span><span class="sc">{e.</span>message<span class="sc">}</span><span class="ss">"</span>)</a></code></pre></div>
|
||||
<pre><code>TREXIO error message: Access to memory beyond allocated</code></pre>
|
||||
<p>According to the TREX configuration file, <code>nucleus_index</code> attribute of a <code>basis</code> group is supposed to have <code>[nucleus_num]</code> elements. In the example above, we have tried to write 4 times more elements, which might lead to memory and/or file corruption. Luckily, TREXIO internally checks the array dimensions and returns an error in case of inconsistency.</p>
|
||||
<h2 id="closing-the-trexio-file">Closing the TREXIO file</h2>
|
||||
<p>It is good practice to close the TREXIO file at the end of the session. In fact, <code>trexio.File</code> class has a destructor, which normally takes care of that. However, if you intend to re-open the TREXIO file, it has to be closed explicitly before. This can be done using the <code>close</code> method, i.e.</p>
|
||||
<div class="sourceCode" id="cb25"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb25-1" title="1">demo_file.close()</a></code></pre></div>
|
||||
<p>Good! You are now ready to inspect the contents of the <code>benzene_demo.h5</code> file using the reading functionality of TREXIO.</p>
|
||||
<h2 id="reading-data-from-the-trexio-file">Reading data from the TREXIO file</h2>
|
||||
<p>First, let’s try to open an existing TREXIO file in read-only mode. This can be done by creating a new instance of the <code>trexio.File</code> class but this time with <code>mode='r'</code> argument. Back end has to be specified as well.</p>
|
||||
<div class="sourceCode" id="cb26"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb26-1" title="1">demo_file_r <span class="op">=</span> trexio.File(filename, mode<span class="op">=</span><span class="st">'r'</span>, back_end<span class="op">=</span>trexio.TREXIO_HDF5)</a></code></pre></div>
|
||||
<p>When reading data from the TREXIO file, the only required argument is a previously created instance of <code>trexio.File</code> class. In our case, it is <code>demo_file_r</code>. TREXIO functions with <code>read_</code> prefix return the desired variable as an output. For example, <code>nucleus_num</code> value can be read from the file as follows</p>
|
||||
<div class="sourceCode" id="cb27"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb27-1" title="1">nucleus_num_r <span class="op">=</span> trexio.read_nucleus_num(demo_file_r)</a></code></pre></div>
|
||||
<div class="sourceCode" id="cb28"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb28-1" title="1"><span class="bu">print</span>(<span class="ss">f"nucleus_num from </span><span class="sc">{</span>filename<span class="sc">}</span><span class="ss"> file ---> </span><span class="sc">{</span>nucleus_num_r<span class="sc">}</span><span class="ss">"</span>)</a></code></pre></div>
|
||||
<pre><code>nucleus_num from benzene_demo.h5 file ---> 12</code></pre>
|
||||
<p>The function call assigns <code>nucleus_num_r</code> to 12, which is consistent with the number of atoms in benzene that we wrote in the previous section.</p>
|
||||
<p>All calls to functions that read data can be done in a very similar way. The key point here is a function name, which in turn defines the output format. Hopefully by now you got used to the TREXIO naming convention and the contents of the <code>nucleus</code> group. Which function would you call to read a <code>point_group</code> attribute of the <code>nucleus</code> group? What type does it return? See the answer below:</p>
|
||||
<div class="sourceCode" id="cb30"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb30-1" title="1">point_group_r <span class="op">=</span> trexio.read_nucleus_point_group(demo_file_r)</a></code></pre></div>
|
||||
<div class="sourceCode" id="cb31"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb31-1" title="1"><span class="bu">print</span>(<span class="ss">f"nucleus_point_group from </span><span class="sc">{</span>filename<span class="sc">}</span><span class="ss"> TREXIO file ---> </span><span class="sc">{</span>point_group_r<span class="sc">}</span><span class="ch">\n</span><span class="ss">"</span>)</a>
|
||||
<a class="sourceLine" id="cb31-2" title="2"><span class="bu">print</span>(<span class="ss">f"Is return type of read_nucleus_point_group a string? ---> </span><span class="sc">{</span><span class="bu">isinstance</span>(point_group_r, <span class="bu">str</span>)<span class="sc">}</span><span class="ss">"</span>)</a></code></pre></div>
|
||||
<pre><code>nucleus_point_group from benzene_demo.h5 TREXIO file ---> D6h
|
||||
|
||||
Is return type of read_nucleus_point_group a string? ---> True</code></pre>
|
||||
<p>The <code>trexio.read_nucleus_point_group</code> function call returns a string <code>D6h</code>, which is exactly what we provided in the previous section. Now, let’s read nuclear charges and labels.</p>
|
||||
<div class="sourceCode" id="cb33"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb33-1" title="1">labels_r <span class="op">=</span> trexio.read_nucleus_label(demo_file_r)</a></code></pre></div>
|
||||
<div class="sourceCode" id="cb34"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb34-1" title="1"><span class="bu">print</span>(<span class="ss">f"nucleus_label from </span><span class="sc">{</span>filename<span class="sc">}</span><span class="ss"> file </span><span class="ch">\n</span><span class="ss">---> </span><span class="sc">{</span>labels_r<span class="sc">}</span><span class="ss">"</span>)</a></code></pre></div>
|
||||
<pre><code>nucleus_label from benzene_demo.h5 file
|
||||
---> ['C', 'C', 'C', 'C', 'C', 'C', 'H', 'H', 'H', 'H', 'H', 'H']</code></pre>
|
||||
<div class="sourceCode" id="cb36"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb36-1" title="1">charges_r <span class="op">=</span> trexio.read_nucleus_charge(demo_file_r)</a></code></pre></div>
|
||||
<div class="sourceCode" id="cb37"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb37-1" title="1"><span class="bu">print</span>(<span class="ss">f"nucleus_charge from </span><span class="sc">{</span>filename<span class="sc">}</span><span class="ss"> file </span><span class="ch">\n</span><span class="ss">---> </span><span class="sc">{</span>charges_r<span class="sc">}</span><span class="ss">"</span>)</a></code></pre></div>
|
||||
<pre><code>nucleus_charge from benzene_demo.h5 file
|
||||
---> [6. 6. 6. 6. 6. 6. 1. 1. 1. 1. 1. 1.]</code></pre>
|
||||
<p>The values are consistent with each other and with the previously written data. Not bad. What about the format of the output?</p>
|
||||
<div class="sourceCode" id="cb39"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb39-1" title="1"><span class="bu">print</span>(<span class="ss">f"nucleus_label return type: </span><span class="sc">{</span><span class="bu">type</span>(labels_r)<span class="sc">}</span><span class="ss">"</span>)</a></code></pre></div>
|
||||
<pre><code>nucleus_label return type: <class 'list'></code></pre>
|
||||
<p>This makes sense, isn’t it? We have written a <code>list</code> of nuclear labels and have received back a <code>list</code> of values from the file. What about nuclear charges?</p>
|
||||
<div class="sourceCode" id="cb41"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb41-1" title="1"><span class="bu">print</span>(<span class="ss">f"nucleus_charge return type: </span><span class="sc">{</span><span class="bu">type</span>(charges_r)<span class="sc">}</span><span class="ss">"</span>)</a></code></pre></div>
|
||||
<pre><code>nucleus_charge return type: <class 'numpy.ndarray'></code></pre>
|
||||
<p>Looks like <code>trexio.read_nucleus_charge</code> function returns a <code>numpy.ndarray</code> even though we have provided a python-ic <code>list</code> to <code>trexio.write_nucleus_charge</code> in the previous section. Why is it so? As has been mentioned before, TREXIO Python API internally relies on the use of the NumPy package to communicate arrays of <code>float</code>-like or <code>int</code>-like values. This prevents some memory leaks and grants additional flexibility to the API. What kind of flexibility? Check this out:</p>
|
||||
<div class="sourceCode" id="cb43"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb43-1" title="1"><span class="bu">print</span>(<span class="ss">f"return dtype in NumPy notation: ---> </span><span class="sc">{</span>charges_r<span class="sc">.</span>dtype<span class="sc">}</span><span class="ss">"</span>)</a></code></pre></div>
|
||||
<pre><code>return dtype in NumPy notation: ---> float64</code></pre>
|
||||
<p>It means that the default precision of the TREXIO output is double (<code>np.float64</code>) for arrays of floating point numbers like <code>nucleus_charge</code>. But what if you do not need this extra precision and would like to read nuclear charges in single (<code>np.float32</code>) or even reduced (e.g. <code>np.float16</code>) precision? TREXIO Python API provides an additional (optional) argument for this. This argument is called <code>dtype</code> and accepts one of the <a href="https://numpy.org/doc/stable/user/basics.types.html">NumPy data types</a>. For example,</p>
|
||||
<div class="sourceCode" id="cb45"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb45-1" title="1">charges_np <span class="op">=</span> trexio.read_nucleus_charge(demo_file_r, dtype<span class="op">=</span>np.float32)</a></code></pre></div>
|
||||
<div class="sourceCode" id="cb46"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb46-1" title="1"><span class="bu">print</span>(<span class="ss">f"return dtype in NumPy notation: ---> </span><span class="sc">{</span>charges_np<span class="sc">.</span>dtype<span class="sc">}</span><span class="ss">"</span>)</a></code></pre></div>
|
||||
<pre><code>return dtype in NumPy notation: ---> float32</code></pre>
|
||||
<h3 id="reading-multidimensional-arrays">Reading multidimensional arrays</h3>
|
||||
<p>So far, we have only read flat 1D arrays. However, we have also written a 2D array of nuclear coordinates. Let’s now read it back from the file:</p>
|
||||
<div class="sourceCode" id="cb48"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb48-1" title="1">coords_r <span class="op">=</span> trexio.read_nucleus_coord(demo_file_r)</a></code></pre></div>
|
||||
<div class="sourceCode" id="cb49"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb49-1" title="1"><span class="bu">print</span>(<span class="ss">f"nucleus_coord from </span><span class="sc">{</span>filename<span class="sc">}</span><span class="ss"> TREXIO file: </span><span class="ch">\n</span><span class="sc">{</span>coords_r<span class="sc">}</span><span class="ss">"</span>)</a></code></pre></div>
|
||||
<pre><code>nucleus_coord from benzene_demo.h5 TREXIO file:
|
||||
[[ 0. 1.39250319 0. ]
|
||||
[-1.20594314 0.6962516 0. ]
|
||||
[-1.20594314 -0.6962516 0. ]
|
||||
[ 0. -1.39250319 0. ]
|
||||
[ 1.20594314 -0.6962516 0. ]
|
||||
[ 1.20594314 0.6962516 0. ]
|
||||
[-2.14171677 1.23652075 0. ]
|
||||
[-2.14171677 -1.23652075 0. ]
|
||||
[ 0. -2.47304151 0. ]
|
||||
[ 2.14171677 -1.23652075 0. ]
|
||||
[ 2.14171677 1.23652075 0. ]
|
||||
[ 0. 2.47304151 0. ]]</code></pre>
|
||||
<div class="sourceCode" id="cb51"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb51-1" title="1"><span class="bu">print</span>(<span class="ss">f"return shape: ---> </span><span class="sc">{</span>coords_r<span class="sc">.</span>shape<span class="sc">}</span><span class="ss">"</span>)</a></code></pre></div>
|
||||
<pre><code>return shape: ---> (12, 3)</code></pre>
|
||||
<p>We can see that TREXIO returns a 2D array with 12 rows and 3 columns, which is consistent with the <code>nucleus_coord</code> dimensions <code>[nucleus_num, 3]</code>. What this means is that <strong>by default TREXIO reshapes the output flat array into a multidimensional one</strong> whenever applicable. This is done based on the shape specified in the TREX configuration file.</p>
|
||||
<p>In some cases, it might be a good idea to explicitly check that the data exists in the file before reading it. This can be achieved using <code>has_</code>-suffixed functions of the API. For example,</p>
|
||||
<div class="sourceCode" id="cb53"><pre class="sourceCode python"><code class="sourceCode python"><a class="sourceLine" id="cb53-1" title="1"><span class="cf">if</span> trexio.has_nucleus_coord(demo_file_r):</a>
|
||||
<a class="sourceLine" id="cb53-2" title="2"> coords_safer <span class="op">=</span> trexio.read_nucleus_coord(demo_file_r)</a></code></pre></div>
|
||||
<h2 id="conclusion">Conclusion</h2>
|
||||
<p>In this Tutorial, you have created a TREXIO file using HDF5 back end and have written the number of atoms, point group, nuclear charges, labels and coordinates, which correspond to benzene molecule. You have also learned how to read this data back from the TREXIO file and how to handle some TREXIO errors.</p>
|
||||
@ -1,414 +0,0 @@
|
||||
# TREXIO Tutorial
|
||||
|
||||
This interactive Tutorial covers some basic use cases of the TREXIO library based on the Python API. At this point, it is assumed that the TREXIO Python package has been sucessfully installed on the user machine or in the virtual environment. If this is not the case, feel free to follow the [installation guide](https://github.com/TREX-CoE/trexio/blob/master/python/README.md).
|
||||
|
||||
## Importing TREXIO
|
||||
|
||||
First of all, let's import the TREXIO package.
|
||||
|
||||
|
||||
```python
|
||||
try:
|
||||
import trexio
|
||||
except ImportError:
|
||||
raise Exception("Unable to import trexio. Please check that trexio is properly instaled.")
|
||||
```
|
||||
|
||||
If no error occurs, then it means that the TREXIO package has been sucessfully imported. Within the current import, TREXIO attributes can be accessed using the corresponding `trexio.attribute` notation. If you prefer to bound a shorter name to the imported module (as commonly done by the NumPy users with `import numpy as np`), this is also possible. To do so, replace `import trexio` with `import trexio as tr` for example. To learn more about importing modules, see the corresponding page of the [Python documentation](https://docs.python.org/3/tutorial/modules.html#more-on-modules).
|
||||
|
||||
## Creating a new TREXIO file
|
||||
|
||||
TREXIO currently supports two back ends for file I/O:
|
||||
|
||||
1. `TREXIO_HDF5`, which relies on extensive use of the [HDF5 library](https://portal.hdfgroup.org/display/HDF5/HDF5) and the associated binary file format. This back end is optimized for high performance but it requires HDF5 to be installed on the user machine.
|
||||
|
||||
2. `TREXIO_TEXT`, which relies on basic I/O operations that are available in the standard C library. This back end is not optimized for performance but it is supposed to work "out-of-the-box" since there are no external dependencies.
|
||||
|
||||
Armed with these new definitions, let's proceed with the tutorial. The first task is to create a TREXIO file called `benzene_demo.h5`. But first we have to remove the file if it exists in the current directory
|
||||
|
||||
|
||||
```python
|
||||
filename = 'benzene_demo.h5'
|
||||
|
||||
import os
|
||||
try:
|
||||
os.remove(filename)
|
||||
except:
|
||||
print(f"File {filename} does not exist.")
|
||||
```
|
||||
|
||||
File benzene_demo.h5 does not exist.
|
||||
|
||||
|
||||
We are now ready to create a new TREXIO file:
|
||||
|
||||
|
||||
```python
|
||||
demo_file = trexio.File(filename, mode='w', back_end=trexio.TREXIO_HDF5)
|
||||
```
|
||||
|
||||
This creates an instance of the `trexio.File` class, which we refer to as `demo_file` in this tutorial. You can check that the corresponding file called `benzene_demo.h5` exists in the root directory. It is now open for writing as indicated by the user-supplied argument `mode='w'`. The file has been initiated using `TREXIO_HDF5` back end and will be accessed accordingly from now on. The information about back end is stored internally by TREXIO, which means that there is no need to specify it every time the I/O operation is performed. If the file named `benzene_demo.h5` already exists, then it is re-opened for writing (and not truncated to prevent data loss).
|
||||
|
||||
## Writing data in the TREXIO file
|
||||
|
||||
Prior to any work with TREXIO library, we highly recommend users to read about [TREXIO internal configuration](https://trex-coe.github.io/trexio/trex.html), which explains the structure of the wavefunction file. The reason is that TREXIO API has a naming convention, which is based on the groups and variables names that are pre-defined by the developers. In this Tutorial, we will only cover contents of the `nucleus` group. Note that custom groups and variables can be added to the TREXIO API.
|
||||
|
||||
In this Tutorial, we consider benzene molecule (C6H6) as an example. Since benzene has 12 atoms, let's specify it in the previously created `demo_file`. In order to do so, one has to call `trexio.write_nucleus_num` function, which accepts an instance of the `trexio.File` class as a first argument and an `int` value corresponding to the number of nuclei as a second argument.
|
||||
|
||||
|
||||
```python
|
||||
nucleus_num = 12
|
||||
```
|
||||
|
||||
|
||||
```python
|
||||
trexio.write_nucleus_num(demo_file, nucleus_num)
|
||||
```
|
||||
|
||||
In fact, all API functions that contain `write_` prefix can be used in a similar way.
|
||||
Variables that contain `_num` suffix are important part of the TREXIO file because some of them define dimensions of arrays. For example, `nucleus_num` variable corresponds to the number of atoms, which will be internally used to write/read the `nucleus_coord` array of nuclear coordinates. In order for TREXIO files to be self-consistent, overwriting num-suffixed variables is currently disabled.
|
||||
|
||||
The number of atoms is not sufficient to define a molecule. Let's first create a list of nuclear charges, which correspond to benzene.
|
||||
|
||||
|
||||
```python
|
||||
charges = [6., 6., 6., 6., 6., 6., 1., 1., 1., 1., 1., 1.]
|
||||
```
|
||||
|
||||
According to the TREX configuration file, there is a `charge` attribute of the `nucleus` group, which has `float` type and `[nucleus_num]` dimension. The `charges` list defined above fits nicely in the description and can be written as follows
|
||||
|
||||
|
||||
```python
|
||||
trexio.write_nucleus_charge(demo_file, charges)
|
||||
```
|
||||
|
||||
**Note: TREXIO function names only contain parts in singular form.** This means that, both `write_nucleus_charges` and `write_nuclear_charges` are invalid API calls. These functions simply do not exist in the `trexio` Python package and the corresponding error message should appear.
|
||||
|
||||
Alternatively, one can provide a list of nuclear labels (chemical elements from the periodic table) that correspond to the aforementioned charges. There is a `label` attribute of the `nucleus` group, which has `str` type and `[nucleus_num]` dimension. Let's create a list of 12 strings, which correspond to 6 carbon and 6 hydrogen atoms:
|
||||
|
||||
|
||||
```python
|
||||
labels = [
|
||||
'C',
|
||||
'C',
|
||||
'C',
|
||||
'C',
|
||||
'C',
|
||||
'C',
|
||||
'H',
|
||||
'H',
|
||||
'H',
|
||||
'H',
|
||||
'H',
|
||||
'H']
|
||||
```
|
||||
|
||||
This can now be written using the corresponding `trexio.write_nucleus_label` function:
|
||||
|
||||
|
||||
```python
|
||||
trexio.write_nucleus_label(demo_file, labels)
|
||||
```
|
||||
|
||||
Two examples above demonstrate how to write arrays of numbers or strings in the file. TREXIO also supports I/O operations on single numerical or string attributes. In fact, in this Tutorial you have already written one numerical attribute: `nucleus_num`. Let's now write a string `'D6h'`, which indicates a point group of benzene molecule. According to the TREX configuration file, `point_group` is a `str` attribute of the `nucleus` group, thus it can be written in the `demo_file` as follows
|
||||
|
||||
|
||||
```python
|
||||
point_group = 'D6h'
|
||||
```
|
||||
|
||||
|
||||
```python
|
||||
trexio.write_nucleus_point_group(demo_file, point_group)
|
||||
```
|
||||
|
||||
### Writing NumPy arrays (float or int types)
|
||||
|
||||
The aforementioned examples cover the majority of the currently implemented functionality related to writing data in the file. It is worth mentioning that I/O of numerical arrays in TREXIO Python API relies on extensive use of the [NumPy package](https://numpy.org/). This will be discussed in more details in the [section about reading data](#Reading-data-from-the-TREXIO-file). However, TREXIO `write_` functions that work with numerical arrays also accept `numpy.ndarray` objects. For example, consider a `coords` list of nuclear coordinates that correspond to benzene molecule
|
||||
|
||||
|
||||
```python
|
||||
coords = [
|
||||
[0.00000000 , 1.39250319 , 0.00000000 ],
|
||||
[-1.20594314 , 0.69625160 , 0.00000000 ],
|
||||
[-1.20594314 , -0.69625160 , 0.00000000 ],
|
||||
[0.00000000 , -1.39250319 , 0.00000000 ],
|
||||
[1.20594314 , -0.69625160 , 0.00000000 ],
|
||||
[1.20594314 , 0.69625160 , 0.00000000 ],
|
||||
[-2.14171677 , 1.23652075 , 0.00000000 ],
|
||||
[-2.14171677 , -1.23652075 , 0.00000000 ],
|
||||
[0.00000000 , -2.47304151 , 0.00000000 ],
|
||||
[2.14171677 , -1.23652075 , 0.00000000 ],
|
||||
[2.14171677 , 1.23652075 , 0.00000000 ],
|
||||
[0.00000000 , 2.47304151 , 0.00000000 ],
|
||||
]
|
||||
```
|
||||
|
||||
Let's take advantage of using NumPy arrays with fixed precision for floating point numbers. But first, try to import the `numpy` package
|
||||
|
||||
|
||||
```python
|
||||
try:
|
||||
import numpy as np
|
||||
except ImportError:
|
||||
raise Exception("Unable to import numpy. Please check that numpy is properly instaled.")
|
||||
```
|
||||
|
||||
You can now convert the previously defined `coords` list into a numpy array with fixed `float64` type as follows
|
||||
|
||||
|
||||
```python
|
||||
coords_np = np.array(coords, dtype=np.float64)
|
||||
```
|
||||
|
||||
TREXIO functions that write numerical arrays accept both lists and numpy arrays as a second argument. That is, both `trexio.write_nucleus_coord(demo_file, coords)` and `trexio.write_nucleus_coord(demo_file, coords_np)` are valid API calls. Let's use the latter and see if it works
|
||||
|
||||
|
||||
```python
|
||||
trexio.write_nucleus_coord(demo_file, coords_np)
|
||||
```
|
||||
|
||||
Congratulations, you have just completed the `nucleus` section of the TREXIO file for benzene molecule! Note that TREXIO API is rather permissive and do not impose any strict ordering on the I/O operations. The only requirement is that dimensioning (`_num` suffixed) variables have to be written in the file **before** writing arrays that depend on these variables. For example, attempting to write `nucleus_charge` or `nucleus_coord` fails if `nucleus_num` has not been written.
|
||||
|
||||
### TREXIO error handling
|
||||
|
||||
TREXIO Python API provides the `trexio.Error` class which simplifies exception handling in the Python scripts. This class wraps up TREXIO return codes and propagates them all the way from the C back end to the Python front end. Let's try to write a negative number of basis set shells `basis_num` in the TREXIO file.
|
||||
|
||||
|
||||
```python
|
||||
try:
|
||||
trexio.write_basis_num(demo_file, -256)
|
||||
except trexio.Error as e:
|
||||
print(f"TREXIO error message: {e.message}")
|
||||
```
|
||||
|
||||
TREXIO error message: Invalid argument 2
|
||||
|
||||
|
||||
The error message says **Invalid argument 2**, which indicates that the user-provided value `-256` is not valid.
|
||||
|
||||
As mentioned before, `_num`-suffixed variables cannot be overwritten in the file. But what happens if you accidentally attempt to do so? Let's have a look at the `write_nucleus_num` function as an example:
|
||||
|
||||
|
||||
```python
|
||||
try:
|
||||
trexio.write_nucleus_num(demo_file, 24)
|
||||
except trexio.Error as e:
|
||||
print(f"TREXIO error message: {e.message}")
|
||||
```
|
||||
|
||||
TREXIO error message: Attribute already exists
|
||||
|
||||
|
||||
The API rightfully complains that the target attribute already exists and cannot be overwritten.
|
||||
|
||||
Alternatively, the aforementioned case can be handled using `trexio.has_nucleus_num` function as follows
|
||||
|
||||
|
||||
```python
|
||||
if not trexio.has_nucleus_num:
|
||||
trexio.write_nucleus_num(demo_file, 24)
|
||||
```
|
||||
|
||||
TREXIO functions with `has_` prefix return `True` if the corresponding variable exists and `False` otherwise.
|
||||
|
||||
What about writing arrays? Let's try to write an list of 48 nuclear indices instead of 12
|
||||
|
||||
|
||||
```python
|
||||
indices = [i for i in range(nucleus_num*4)]
|
||||
```
|
||||
|
||||
|
||||
```python
|
||||
try:
|
||||
trexio.write_basis_nucleus_index(demo_file, indices)
|
||||
except trexio.Error as e:
|
||||
print(f"TREXIO error message: {e.message}")
|
||||
```
|
||||
|
||||
TREXIO error message: Access to memory beyond allocated
|
||||
|
||||
|
||||
According to the TREX configuration file, `nucleus_index` attribute of a `basis` group is supposed to have `[nucleus_num]` elements. In the example above, we have tried to write 4 times more elements, which might lead to memory and/or file corruption. Luckily, TREXIO internally checks the array dimensions and returns an error in case of inconsistency.
|
||||
|
||||
## Closing the TREXIO file
|
||||
|
||||
It is good practice to close the TREXIO file at the end of the session. In fact, `trexio.File` class has a destructor, which normally takes care of that. However, if you intend to re-open the TREXIO file, it has to be closed explicitly before. This can be done using the `close` method, i.e.
|
||||
|
||||
|
||||
```python
|
||||
demo_file.close()
|
||||
```
|
||||
|
||||
Good! You are now ready to inspect the contents of the `benzene_demo.h5` file using the reading functionality of TREXIO.
|
||||
|
||||
## Reading data from the TREXIO file
|
||||
|
||||
First, let's try to open an existing TREXIO file in read-only mode. This can be done by creating a new instance of the `trexio.File` class but this time with `mode='r'` argument. Back end has to be specified as well.
|
||||
|
||||
|
||||
```python
|
||||
demo_file_r = trexio.File(filename, mode='r', back_end=trexio.TREXIO_HDF5)
|
||||
```
|
||||
|
||||
When reading data from the TREXIO file, the only required argument is a previously created instance of `trexio.File` class. In our case, it is `demo_file_r`. TREXIO functions with `read_` prefix return the desired variable as an output. For example, `nucleus_num` value can be read from the file as follows
|
||||
|
||||
|
||||
```python
|
||||
nucleus_num_r = trexio.read_nucleus_num(demo_file_r)
|
||||
```
|
||||
|
||||
|
||||
```python
|
||||
print(f"nucleus_num from {filename} file ---> {nucleus_num_r}")
|
||||
```
|
||||
|
||||
nucleus_num from benzene_demo.h5 file ---> 12
|
||||
|
||||
|
||||
The function call assigns `nucleus_num_r` to 12, which is consistent with the number of atoms in benzene that we wrote in the previous section.
|
||||
|
||||
All calls to functions that read data can be done in a very similar way. The key point here is a function name, which in turn defines the output format. Hopefully by now you got used to the TREXIO naming convention and the contents of the `nucleus` group. Which function would you call to read a `point_group` attribute of the `nucleus` group? What type does it return? See the answer below:
|
||||
|
||||
|
||||
```python
|
||||
point_group_r = trexio.read_nucleus_point_group(demo_file_r)
|
||||
```
|
||||
|
||||
|
||||
```python
|
||||
print(f"nucleus_point_group from {filename} TREXIO file ---> {point_group_r}\n")
|
||||
print(f"Is return type of read_nucleus_point_group a string? ---> {isinstance(point_group_r, str)}")
|
||||
```
|
||||
|
||||
nucleus_point_group from benzene_demo.h5 TREXIO file ---> D6h
|
||||
|
||||
Is return type of read_nucleus_point_group a string? ---> True
|
||||
|
||||
|
||||
The `trexio.read_nucleus_point_group` function call returns a string `D6h`, which is exactly what we provided in the previous section. Now, let's read nuclear charges and labels.
|
||||
|
||||
|
||||
```python
|
||||
labels_r = trexio.read_nucleus_label(demo_file_r)
|
||||
```
|
||||
|
||||
|
||||
```python
|
||||
print(f"nucleus_label from {filename} file \n---> {labels_r}")
|
||||
```
|
||||
|
||||
nucleus_label from benzene_demo.h5 file
|
||||
---> ['C', 'C', 'C', 'C', 'C', 'C', 'H', 'H', 'H', 'H', 'H', 'H']
|
||||
|
||||
|
||||
|
||||
```python
|
||||
charges_r = trexio.read_nucleus_charge(demo_file_r)
|
||||
```
|
||||
|
||||
|
||||
```python
|
||||
print(f"nucleus_charge from {filename} file \n---> {charges_r}")
|
||||
```
|
||||
|
||||
nucleus_charge from benzene_demo.h5 file
|
||||
---> [6. 6. 6. 6. 6. 6. 1. 1. 1. 1. 1. 1.]
|
||||
|
||||
|
||||
The values are consistent with each other and with the previously written data. Not bad. What about the format of the output?
|
||||
|
||||
|
||||
```python
|
||||
print(f"nucleus_label return type: {type(labels_r)}")
|
||||
```
|
||||
|
||||
nucleus_label return type: <class 'list'>
|
||||
|
||||
|
||||
This makes sense, isn't it? We have written a `list` of nuclear labels and have received back a `list` of values from the file. What about nuclear charges?
|
||||
|
||||
|
||||
```python
|
||||
print(f"nucleus_charge return type: {type(charges_r)}")
|
||||
```
|
||||
|
||||
nucleus_charge return type: <class 'numpy.ndarray'>
|
||||
|
||||
|
||||
Looks like `trexio.read_nucleus_charge` function returns a `numpy.ndarray` even though we have provided a python-ic `list` to `trexio.write_nucleus_charge` in the previous section. Why is it so? As has been mentioned before, TREXIO Python API internally relies on the use of the NumPy package to communicate arrays of `float`-like or `int`-like values. This prevents some memory leaks and grants additional flexibility to the API. What kind of flexibility? Check this out:
|
||||
|
||||
|
||||
```python
|
||||
print(f"return dtype in NumPy notation: ---> {charges_r.dtype}")
|
||||
```
|
||||
|
||||
return dtype in NumPy notation: ---> float64
|
||||
|
||||
|
||||
It means that the default precision of the TREXIO output is double (`np.float64`) for arrays of floating point numbers like `nucleus_charge`. But what if you do not need this extra precision and would like to read nuclear charges in single (`np.float32`) or even reduced (e.g. `np.float16`) precision? TREXIO Python API provides an additional (optional) argument for this. This argument is called `dtype` and accepts one of the [NumPy data types](https://numpy.org/doc/stable/user/basics.types.html). For example,
|
||||
|
||||
|
||||
```python
|
||||
charges_np = trexio.read_nucleus_charge(demo_file_r, dtype=np.float32)
|
||||
```
|
||||
|
||||
|
||||
```python
|
||||
print(f"return dtype in NumPy notation: ---> {charges_np.dtype}")
|
||||
```
|
||||
|
||||
return dtype in NumPy notation: ---> float32
|
||||
|
||||
|
||||
### Reading multidimensional arrays
|
||||
|
||||
So far, we have only read flat 1D arrays. However, we have also written a 2D array of nuclear coordinates. Let's now read it back from the file:
|
||||
|
||||
|
||||
```python
|
||||
coords_r = trexio.read_nucleus_coord(demo_file_r)
|
||||
```
|
||||
|
||||
|
||||
```python
|
||||
print(f"nucleus_coord from {filename} TREXIO file: \n{coords_r}")
|
||||
```
|
||||
|
||||
nucleus_coord from benzene_demo.h5 TREXIO file:
|
||||
[[ 0. 1.39250319 0. ]
|
||||
[-1.20594314 0.6962516 0. ]
|
||||
[-1.20594314 -0.6962516 0. ]
|
||||
[ 0. -1.39250319 0. ]
|
||||
[ 1.20594314 -0.6962516 0. ]
|
||||
[ 1.20594314 0.6962516 0. ]
|
||||
[-2.14171677 1.23652075 0. ]
|
||||
[-2.14171677 -1.23652075 0. ]
|
||||
[ 0. -2.47304151 0. ]
|
||||
[ 2.14171677 -1.23652075 0. ]
|
||||
[ 2.14171677 1.23652075 0. ]
|
||||
[ 0. 2.47304151 0. ]]
|
||||
|
||||
|
||||
|
||||
```python
|
||||
print(f"return shape: ---> {coords_r.shape}")
|
||||
```
|
||||
|
||||
return shape: ---> (12, 3)
|
||||
|
||||
|
||||
We can see that TREXIO returns a 2D array with 12 rows and 3 columns, which is consistent with the `nucleus_coord` dimensions `[nucleus_num, 3]`. What this means is that **by default TREXIO reshapes the output flat array into a multidimensional one** whenever applicable. This is done based on the shape specified in the TREX configuration file.
|
||||
|
||||
In some cases, it might be a good idea to explicitly check that the data exists in the file before reading it. This can be achieved using `has_`-suffixed functions of the API. For example,
|
||||
|
||||
|
||||
```python
|
||||
if trexio.has_nucleus_coord(demo_file_r):
|
||||
coords_safer = trexio.read_nucleus_coord(demo_file_r)
|
||||
```
|
||||
|
||||
## Conclusion
|
||||
|
||||
In this Tutorial, you have created a TREXIO file using HDF5 back end and have written the number of atoms, point group, nuclear charges, labels and coordinates, which correspond to benzene molecule. You have also learned how to read this data back from the TREXIO file and how to handle some TREXIO errors.
|
||||
@ -4,6 +4,7 @@
|
||||
|
||||
------------------
|
||||
|
||||
- [[./tutorial_benzene.html][Tutorial]]
|
||||
- [[./trex.html][Data stored with TREXIO]]
|
||||
- [[./templator_front.html][Front end API]]
|
||||
- [[./templator_hdf5.html][HDF5 back end]]
|
||||
|
||||
@ -48,22 +48,11 @@
|
||||
*/
|
||||
%cstring_output_maxsize(char* const str_out, const int32_t max_str_len);
|
||||
|
||||
/* [WIP] TREXIO back ends and exit codes can be redefined in the SWIG target language
|
||||
using %ignore and further #define statements (instead of disabling the type cast in the trexio.h file)
|
||||
*/
|
||||
/*
|
||||
%ignore TREXIO_HDF5; // Ignore a macro in the header file
|
||||
%ignore TREXIO_TEST; // Ignore a macro in the header file
|
||||
#define TREXIO_HDF5 0
|
||||
#define TREXIO_TEXT 0
|
||||
*/
|
||||
|
||||
/* This is an attempt to make SWIG treat double * dset_out|_in, int64_t dim_out|_in pattern
|
||||
/* This block is needed make SWIG treat (double * dset_out|_in, int64_t dim_out|_in) pattern
|
||||
as a special case in order to return the NumPy array to Python from C pointer to array
|
||||
provided by trexio_read_safe_[dset_num] function.
|
||||
NOTE: numpy.i is currently not part of SWIG but included in the numpy distribution (under numpy/tools/swig/numpy.i)
|
||||
This means that the interface file have to be provided to SWIG during compilation either by
|
||||
copying it to the local working directory or by providing -l/path/to/numpy.i flag upon SWIG compilation
|
||||
*/
|
||||
%include "numpy.i"
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user