#+TITLE: QMCkl source code documentation #+EXPORT_FILE_NAME: index.html #+PROPERTY: comments org #+SETUPFILE: https://fniessen.github.io/org-html-themes/org/theme-readtheorg.setup * Introduction The ultimate goal of QMCkl is to provide a high-performance implementation of the main kernels of QMC. In this particular repository, we focus on the definition of the API and the tests, and on a /pedagogical/ presentation of the algorithms. We expect the HPC experts to use this repository as a reference for re-writing optimized libraries. Literate programming is particularly adapted in this context. Source files are written in [[https://karl-voit.at/2017/09/23/orgmode-as-markup-only/][org-mode]] format, to provide useful comments and LaTex formulas close to the code. There exists multiple possibilities to convert org-mode files into different formats such as HTML or pdf. For a tutorial on literate programming with org-mode, follow [[http://www.howardism.org/Technical/Emacs/literate-programming-tutorial.html][this link]]. The code is extracted from the org files using Emacs as a command-line tool in the =Makefile=, and then the produced files are compiled. ** Language used Fortran is one of the most common languages used by the community, and is simple enough to make the algorithms readable. Hence we propose in this pedagogical implementation of QMCkl to use Fortran to express the algorithms. For specific internal functions where the C language is more natural, C is used. As Fortran modules generate compiler-dependent files, the use of modules is restricted to the internal use of the library, otherwise the compliance with C is violated. The external dependencies should be kept as small as possible, so external libraries should be used /only/ if their used is strongly justified. ** Source code editing Any text editor can be used to edit org-mode files. For a better user experience Emacs is recommended. For users hating Emacs, it is good to know that Emacs can behave like Vim when switched into ``Evil'' mode. There also exists [[https://www.spacemacs.org][Spacemacs]] which helps the transition for Vim users. For users with a preference for Jupyter notebooks, the following script can convert jupyter notebooks to org-mode files: #+BEGIN_SRC sh tangle: nb_to_org.sh #!/bin/bash # $ nb_to_org.sh notebook.ipynb # produces the org-mode file notebook.org set -e nb=$(basename $1 .ipynb) jupyter nbconvert --to markdown ${nb}.ipynb --output ${nb}.md pandoc ${nb}.md -o ${nb}.org rm ${nb}.md #+END_SRC And pandoc can convert multiple markdown formats into org-mode. ** Writing in Fortran The Fortran source files should provide a C interface using =iso_c_binding=. The name of the Fortran source files should end with =_f.f90= to be properly handled by the Makefile. The names of the functions defined in fortran should be the same as those exposed in the API suffixed by =_f=. Fortran interface files should also be written in the =qmckl_f.f90= file. For more guidelines on using Fortran to generate a C interface, see [[http://fortranwiki.org/fortran/show/Generating+C+Interfaces][this link]]. ** Coding style # TODO: decide on a coding style To improve readability, we maintain a consistent coding style in the library. - For C source files, we will use __(decide on a coding style)__ - For Fortran source files, we will use __(decide on a coding style)__ Coding style can be automatically checked with [[https://clang.llvm.org/docs/ClangFormat.html][clang-format]]. ** Design of the library The proposed API should allow the library to: - deal with memory transfers between CPU and accelerators - use different levels of floating-point precision We chose a multi-layered design with low-level and high-level functions (see below). *** Naming conventions Use =qmckl_= as a prefix for all exported functions and variables. All exported header files should have a filename with the prefix =qmckl_=. If the name of the org-mode file is =xxx.org=, the name of the produced C files should be =xxx.c= and =xxx.h= and the name of the produced Fortran files should be =xxx.f90= Arrays are in uppercase and scalars are in lowercase. In the names of the variables and functions, only the singular form is allowed. *** Application programming interface The application programming interface (API) is designed to be compatible with the C programming language (not C++), to ensure that the library will be easily usable in /any/ language. This implies that only the following data types are allowed in the API: - 32-bit and 64-bit floats and arrays (=real= and =double=) - 32-bit and 64-bit integers and arrays (=int32_t= and =int64_t=) - Pointers should be represented as 64-bit integers (even on 32-bit architectures) - ASCII strings are represented as a pointers to a character arrays and terminated by a zero character (C convention). Complex numbers can be represented by an array of 2 floats. # TODO : Link to repositories for bindings To facilitate the use in other languages than C, we provide some bindings in other languages in other repositories. *** Global state Global variables should be avoided in the library, because it is possible that one single program needs to use multiple instances of the library. To solve this problem we propose to use a pointer to a =context= variable, built by the library with the =qmckl_context_create= function. The =context= contains the global state of the library, and is used as the first argument of many QMCkl functions. The internal structure of the context is not specified, to give a maximum of freedom to the different implementations. Modifying the state is done by setters and getters, prefixed by =qmckl_context_set_= an =qmckl_context_get_=. When a context variable is modified by a setter, a copy of the old data structure is made and updated, and the pointer to the new data structure is returned, such that the old contexts can still be accessed. It is also possible to modify the state in an impure fashion, using the =qmckl_context_update_= functions. The context and its old versions can be destroyed with =qmckl_context_destroy=. *** Low-level functions Low-level functions are very simple functions which are leaves of the function call tree (they don't call any other QMCkl function). These functions are /pure/, and unaware of the QMCkl =context=. They are not allowed to allocate/deallocate memory, and if they need temporary memory it should be provided in input. *** High-level functions High-level functions are at the top of the function call tree. They are able to choose which lower-level function to call depending on the required precision, and do the corresponding type conversions. These functions are also responsible for allocating temporary storage, to simplify the use of accelerators. The high-level functions should be pure, unless the introduction of non-purity is justified. All the side effects should be made in the =context= variable. # TODO : We need an identifier for impure functions *** Numerical precision The number of bits of precision required for a function should be given as an input of low-level computational functions. This input will be used to define the values of the different thresholds that might be used to avoid computing unnecessary noise. High-level functions will use the precision specified in the =context= variable. ** Algorithms Reducing the scaling of an algorithm usually implies also reducing its arithmetic complexity (number of flops per byte). Therefore, for small sizes \(\mathcal{O}(N^3)\) and \(\mathcal{O}(N^2)\) algorithms are better adapted than linear scaling algorithms. As QMCkl is a general purpose library, multiple algorithms should be implemented adapted to different problem sizes. ** Rules for the API - =stdint= should be used for integers (=int32_t=, =int64_t=) - integers used for counting should always be =int64_t= - floats should be by default =double=, unless explicitly mentioned - pointers are converted to =int64_t= to increase portability * Documentation