mirror of
https://github.com/TREX-CoE/qmckl.git
synced 2024-09-28 20:41:21 +02:00
703031b16f
Line breaks in the lists of the documentation appeared in the file preview. They have been removed so that all of its text belongs correctly to eac bullet point.
313 lines
15 KiB
Org Mode
313 lines
15 KiB
Org Mode
#+TITLE: Introduction
|
|
#+PROPERTY: comments org
|
|
#+SETUPFILE: ../tools/theme.setup
|
|
# -*- mode: org -*-
|
|
|
|
* Using QMCkl
|
|
|
|
The =qmckl.h= header file installed in the =${prefix}/include= directory
|
|
has to be included in C codes when QMCkl functions are used:
|
|
|
|
#+begin_src c :tangle no
|
|
#include "qmckl.h"
|
|
#+end_src
|
|
|
|
In Fortran programs, the =qmckl_f.f90= installed in
|
|
=${prefix}/share/qmckl/fortran= interface file should be copied in the source
|
|
code using the library, and the Fortran codes should use the ~qmckl~ module as
|
|
|
|
#+begin_src f90 :tangle no
|
|
use qmckl
|
|
#+end_src
|
|
|
|
Both files are located in the =include/= directory.
|
|
|
|
* Developing in QMCkl
|
|
|
|
** Literate programming
|
|
|
|
In a traditional source code, most of the lines of source files of a program
|
|
are code, scripts, Makefiles, and only a few lines are comments explaining
|
|
parts of the code that are non-trivial to understand. The documentation of
|
|
the prorgam is usually written in a separate directory, and is often outdated
|
|
compared to the code.
|
|
|
|
Literate programming is a different approach to programming,
|
|
where the program is considered as a publishable-quality document. Most of
|
|
the lines of the source files are text, mathematical formulas, tables,
|
|
figures, /etc/, and the lines of code are just the translation in a computer
|
|
language of the ideas and algorithms expressed in the text. More importantly,
|
|
the "document" is structured like a text document with sections, subsections,
|
|
a bibliography, a table of contents /etc/, and the place where pieces of code
|
|
appear are the places where they should belong for the reader to understand
|
|
the logic of the program, not the places where the compiler expects to find
|
|
them. Both the publishable-quality document and the binary executable are
|
|
produced from the same source files.
|
|
|
|
Literate programming is particularly well adapted in this context, as the
|
|
central part of this project is the documentation of an API. The
|
|
implementation of the algorithms is just an expression of the algorithms in a
|
|
language that can be compiled, so that the correctness of the algorithms can
|
|
be tested.
|
|
|
|
We have chosen to write the source files in [[https://karl-voit.at/2017/09/23/orgmode-as-markup-only/][org-mode]] format,
|
|
as any text editor can be used to edit org-mode files. To
|
|
produce the documentation, there exists multiple possibilities to convert
|
|
org-mode files into different formats such as HTML or PDF. The source code is
|
|
easily extracted from the org-mode files invoking the Emacs text editor from
|
|
the command-line in the =Makefile=, and then the produced files are compiled.
|
|
Moreover, within the Emacs text editor the source code blocks can be executed
|
|
interactively, in the same spirit as Jupyter notebooks.
|
|
|
|
** Source code editing
|
|
|
|
For a tutorial on literate programming with org-mode, follow [[http://www.howardism.org/Technical/Emacs/literate-programming-tutorial.html][this link]].
|
|
|
|
Any text editor can be used to edit org-mode files. For a better
|
|
user experience Emacs is recommended. For users hating Emacs, it
|
|
is good to know that Emacs can behave like Vim when switched into
|
|
``Evil'' mode.
|
|
|
|
In the =tools/init.el= file, we provide a minimal Emacs configuration
|
|
file for vim users. This file should be copied into =.emacs.d/init.el=.
|
|
|
|
For users with a preference for Jupyter notebooks, we also provide the =tools/nb_to_org.sh= script can convert jupyter notebooks into org-mode
|
|
files.
|
|
|
|
Note that pandoc can be used to convert multiple markdown formats into
|
|
org-mode.
|
|
|
|
** Choice of the programming language
|
|
|
|
Most of the codes of the [[https://trex-coe.eu][TREX CoE]] are written in Fortran with some scripts in
|
|
Bash and Python. Outside of the CoE, Fortran is also important (Casino, Amolqc),
|
|
and other important languages used by the community are C and C++ (QMCPack,
|
|
QWalk), and Julia is gaining in popularity. The library we design should be
|
|
compatible with all of these languages. The QMCkl API has to be compatible
|
|
with the C language since libraries with a C-compatible API can be used in
|
|
every other language.
|
|
|
|
High-performance versions of the QMCkl, with the same API, will be rewritten by
|
|
the experts in HPC. These optimized libraries will be tuned for specific
|
|
architectures, among which we can cite x86 based processors, and GPU
|
|
accelerators. Nowadays, the most efficient software tools to take advantage of
|
|
low-level features of the processor (intrinsics) and of GPUs are for C++
|
|
developers. It is highly probable that the optimized implementations will be
|
|
written in C++, and this is agreement with our choice to make the API
|
|
C-compatible.
|
|
|
|
Fortran is one of the most common languages used by the community, and is simple
|
|
enough to make the algorithms readable both by experts in QMC, and experts in
|
|
HPC. Hence we propose in this pedagogical implementation of QMCkl to use Fortran
|
|
to express the QMC algorithms. As the main languages of the library is C, this
|
|
implies that the exposed C functions call the Fortran routine. However, for
|
|
internal functions related to system programming, the C language is more natural
|
|
than Fortran.
|
|
|
|
The Fortran source files should provide a C interface using the ~iso_c_binding~ module. The name of the Fortran source files should end with =_f.f90= to be properly handled by the =Makefile=. The names of the functions
|
|
defined in Fortran should be the same as those exposed in the API suffixed by =_f=.
|
|
|
|
For more guidelines on using Fortran to generate a C interface, see
|
|
[[http://fortranwiki.org/fortran/show/Generating+C+Interfaces][this link]].
|
|
|
|
** Coding rules
|
|
|
|
The authors should follow the recommendations of the C99
|
|
[[https://wiki.sei.cmu.edu/confluence/display/c/SEI+CERT+C+Coding+Standard][SEI+CERT C Coding Standard]].
|
|
|
|
Compliance can be checked with =cppcheck= as:
|
|
|
|
#+begin_src bash
|
|
cppcheck --addon=cert --enable=all *.c &> cppcheck.out
|
|
#+end_src
|
|
|
|
** Design of the library
|
|
|
|
The proposed API should allow the library to: deal with memory transfers
|
|
between CPU and accelerators, and to use different levels of floating-point
|
|
precision. We chose a multi-layered design with low-level and high-level
|
|
functions (see below).
|
|
|
|
** Naming conventions
|
|
|
|
To avoid namespace collisions, we use =qmckl_= as a prefix for all exported
|
|
functions and variables. All exported header files should have a file name
|
|
prefixed with =qmckl_=.
|
|
|
|
If the name of the org-mode file is =xxx.org=, the name of the
|
|
produced C files should be =xxx.c= and =xxx.h= and the name of the
|
|
produced Fortran file should be =xxx.f90=.
|
|
|
|
Arrays are in uppercase and scalars are in lowercase.
|
|
|
|
In the names of the variables and functions, only the singular
|
|
form is allowed.
|
|
|
|
** Application programming interface
|
|
|
|
In the C language, the number of bits used by the integer types can change
|
|
from one architecture to another one. To circumvent this problem, we choose to
|
|
use the integer types defined in ~<stdint.h>~ where the number of bits used for
|
|
the integers are fixed.
|
|
|
|
To ensure that the library will be easily usable in /any/ other language
|
|
than C, we restrict the data types in the interfaces to the following:
|
|
- 32-bit and 64-bit integers, scalars and and arrays (~int32_t~ and ~int64_t~)
|
|
- 32-bit and 64-bit floats, scalars and and arrays (~float~ and ~double~)
|
|
- Pointers are always casted into 64-bit integers, even on legacy 32-bit architectures
|
|
- ASCII strings are represented as a pointers to character arrays
|
|
and terminated by a ~'\0'~ character (C convention).
|
|
- Complex numbers can be represented by an array of 2 floats.
|
|
- Boolean variables are stored as integers, ~1~ for ~true~ and ~0~ for ~false~
|
|
- Floating point variables should be by default ~double~ unless explicitly mentioned
|
|
- integers used for counting should always be ~int64_t~
|
|
|
|
To facilitate the use in other languages than C, we will provide some
|
|
bindings in other languages in other repositories.
|
|
|
|
# TODO : Link to repositories for bindings
|
|
# To facilitate the use in other languages than C, we provide some
|
|
# bindings in other languages in other repositories.
|
|
|
|
** Global state
|
|
|
|
Global variables should be avoided in the library, because it is
|
|
possible that one single program needs to use multiple instances
|
|
of the library. To solve this problem we propose to use a pointer
|
|
to a [[./qmckl_context.html][=context=]] variable, built by the library with the =qmckl_context_create= function. The <<<=context=>>> contains the global
|
|
state of the library, and is used as the first argument of many
|
|
QMCkl functions.
|
|
|
|
The internal structure of the context is not specified, to give a
|
|
maximum of freedom to the different implementations. Modifying
|
|
the state is done by setters and getters, prefixed by =qmckl_set_= an =qmckl_get_=.
|
|
|
|
** Headers
|
|
|
|
A single =qmckl.h= header to be distributed by the library
|
|
is built by concatenating some of the produced header files.
|
|
To facilitate building the =qmckl.h= file, we separate types from
|
|
function declarations in headers. Types should be defined in header
|
|
files suffixed by =_type.h=, and functions in files suffixed by =_func.h=.
|
|
As these files will be concatenated in a single file, they should
|
|
not be guarded by ~#ifndef *_H~, and they should not include other
|
|
produced headers.
|
|
|
|
Some particular types that are not exported need to be known by the
|
|
context, and there are some functions to update instances of these
|
|
types contained inside the context. For example, a ~qmckl_numprec_struct~ is present in the context, and the function ~qmckl_set_numprec_range~ takes a context as a parameter, and set a
|
|
value in the ~qmckl_numprec_struct~ contained in the context.
|
|
Because of these intricate dependencies, a private header is
|
|
created, containing the ~qmckl_numprec_struct~. This header is
|
|
included in the private header file which defines the type of the
|
|
context. Header files for private types are suffixed by =_private_type.h=
|
|
and header files for private functions are suffixed by =_private_func.h=.
|
|
Fortran interfaces should also be written in the =*fh_func.f90= file,
|
|
and the types definitions should be written in the =*fh_type.f90= file.
|
|
|
|
| File | Scope | Description |
|
|
|--------------------+---------+------------------------------|
|
|
| =*_type.h= | Public | Type definitions |
|
|
| =*_func.h= | Public | Function definitions |
|
|
| =*_private_type.h= | Private | Type definitions |
|
|
| =*_private_func.h= | Private | Function definitions |
|
|
| =*fh_type.f90= | Public | Fortran type definitions |
|
|
| =*fh_func.f90= | Public | Fortran function definitions |
|
|
|
|
** Low-level functions
|
|
|
|
Low-level functions are very simple functions which are leaves of
|
|
the function call tree (they don't call any other QMCkl function).
|
|
|
|
These functions are /pure/, and unaware of the QMCkl =context=. They are not allowed to allocate/deallocate memory, and
|
|
if they need temporary memory it should be provided in input.
|
|
|
|
** High-level functions
|
|
|
|
High-level functions are at the top of the function call tree.
|
|
They are able to choose which lower-level function to call
|
|
depending on the required precision, and do the corresponding type
|
|
conversions. These functions are also responsible for allocating
|
|
temporary storage, to simplify the use of accelerators.
|
|
|
|
The high-level functions should be pure, unless the introduction
|
|
of non-purity is justified. All the side effects should be made in
|
|
the =context= variable.
|
|
|
|
# TODO : We need an identifier for impure functions
|
|
# Suggestion (VJ): using *_unsafe_* for impure functions ?
|
|
|
|
** Numerical precision
|
|
|
|
The number of bits of precision required for a function should be
|
|
given as an input of low-level computational functions. This input
|
|
will be used to define the values of the different thresholds that
|
|
might be used to avoid computing unnecessary noise. High-level
|
|
functions will use the precision specified in the =context=
|
|
variable.
|
|
|
|
In order to automatize numerical accuracy tests, QMCkl uses
|
|
[[https://github.com/verificarlo/verificarlo][Verificarlo]] and
|
|
its CI functionality. You can read Verificarlo CI's documentation
|
|
at the [[https://github.com/verificarlo/verificarlo/blob/master/doc/06-Postprocessing.md#verificarlo-ci][following link]].
|
|
Reading it is advised to understand the remainder of this section.
|
|
|
|
To enable support for Verificarlo CI tests when building the
|
|
library, use the following configure command :
|
|
|
|
#+begin_src bash
|
|
QMCKL_DEVEL=1 ./configure --prefix=$PWD/_install --enable-silent-rules --enable-maintainer-mode CC=verificarlo-f FC=verificarlo-f --host=x86_64 --enable-vfc_ci
|
|
#+end_src
|
|
|
|
Note that this does require an install of Verificarlo *with
|
|
Fortran support*. Enabling the support for CI will define the ~VFC_CI~ preprocessor variable which use will be explained now.
|
|
|
|
As explained in the documentation, Verificarlo CI uses a probes
|
|
system to export variables from test programs to the tools itself.
|
|
To make tests easier to use, QMCkl has its own interface to the
|
|
probes system. To make the system very easy to use, these functions
|
|
are always defined, but will behave differently depending on the ~VFC_CI~ preprocessor variable. There are 3 functions at your
|
|
disposal. When the CI is enabled, they will place a ~vfc_ci~ probe
|
|
as if calling ~vfc_probes~ directly. Otherwise, they will either do
|
|
nothing or perform a check on the tested value and return its result
|
|
as a boolean that you are free to use or ignore.
|
|
Here are these 3 functions :
|
|
|
|
- ~qmckl_probe~ : place a normal probe witout any check. Won't do anything when ~vfc_ci~ is disabled.
|
|
- ~qmckl_probe_check~ : place a probe with a relative check. If ~vfc_ci~ is disabled, this will return the result of a relative check (|val - ref| / ref < accuracy target?).
|
|
- ~qmckl_probe_check~ : place a probe with an absolute check. If ~vfc_ci~ is disabled, this will return the result of an absolute check (|val - ref| < accuracy target ?)
|
|
|
|
If you need more details on these functions or their Fortran
|
|
interfaces, have a look at the ~tools/qmckl_probes~ files.
|
|
|
|
Finally, if you need to add a QMCkl kernel to the CI tests
|
|
or modify an existing one, you should pay attention to the
|
|
following points :
|
|
|
|
- you should add the new kernel to the ~vfc_tests_config.json~ file, which controls the backends and repetitions for each executable. More details can be found in the ~vfc_ci~ documentation.
|
|
|
|
- in order to call the ~qmckl_probes~ functions from Fortran, import the ~qmckl_probes_f~ module. All Fortran functions are appended with the ~_f~ suffix.
|
|
|
|
- if your tests include some asserts that rely on accurate FP values, you should probably wrap them inside a ~#ifndef VFC_CI~ statement, as the asserts would otherwise risk to fail when executed with the different Verificarlo backends.
|
|
|
|
** Algorithms
|
|
|
|
Reducing the scaling of an algorithm usually implies also reducing
|
|
its arithmetic complexity (number of flops per byte). Therefore,
|
|
for small sizes \(\mathcal{O}(N^3)\) and \(\mathcal{O}(N^2)\)
|
|
algorithms are better adapted than linear scaling algorithms. As
|
|
QMCkl is a general purpose library, multiple algorithms should be
|
|
implemented adapted to different problem sizes.
|