UP | HOME

Front end API

Table of Contents

break;

1 Coding conventions

  • integer types will be defined using types given in stdint.h
  • pointers are always initialized to NULL
  • when memory is freed, the pointer is set to NULL
  • assert.h should be used extensively
  • variable names are in lower case
  • #define constants are in upper case
  • structs are suffixed by _s
  • types are suffixed by _t
  • API calls return trexio_exit_code (except for trexio_open function)

1.1 Memory allocation

Memory allocation of structures can be facilitated by using the following macro, which ensures that the size of the allocated object is the same as the size of the data type pointed by the pointer.

#define MALLOC(T) (T*) malloc (sizeof(T))
#define CALLOC(N,T) (T*) calloc ( (N) , sizeof(T) )

When a pointer is freed, it should be set to NULL. This can be facilitated by the use of the following macro:

#define FREE(X) { free(X) ; (X)=NULL; }

The maximum string size for the filenames is 4096 characters.

#define TREXIO_MAX_FILENAME_LENGTH 4096

2 Front end

All calls to TREXIO are thread-safe. TREXIO front end is modular, which simplifies implementation of new back ends.

2.1 Error handling

Macro Code Description
TREXIO_FAILURE -1 'Unknown failure'
TREXIO_SUCCESS 0 'Success'
TREXIO_INVALID_ARG_1 1 'Invalid argument 1'
TREXIO_INVALID_ARG_2 2 'Invalid argument 2'
TREXIO_INVALID_ARG_3 3 'Invalid argument 3'
TREXIO_INVALID_ARG_4 4 'Invalid argument 4'
TREXIO_INVALID_ARG_5 5 'Invalid argument 5'
TREXIO_END 6 'End of file'
TREXIO_READONLY 7 'Read-only file'
TREXIO_ERRNO 8 strerror(errno)
TREXIO_INVALID_ID 9 'Invalid ID'
TREXIO_ALLOCATION_FAILED 10 'Allocation failed'
TREXIO_HAS_NOT 11 'Element absent'
TREXIO_INVALID_NUM 12 'Invalid (negative or 0) dimension'
TREXIO_ATTR_ALREADY_EXISTS 13 'Attribute already exists'
TREXIO_DSET_ALREADY_EXISTS 14 'Dataset already exists'
TREXIO_OPEN_ERROR 15 'Error opening file'
TREXIO_LOCK_ERROR 16 'Error locking file'
TREXIO_UNLOCK_ERROR 17 'Error unlocking file'
TREXIO_FILE_ERROR 18 'Invalid file handle'
TREXIO_GROUP_READ_ERROR 19 'Error reading group'
TREXIO_GROUP_WRITE_ERROR 20 'Error writing group'
TREXIO_ELEM_READ_ERROR 21 'Error reading element'
TREXIO_ELEM_WRITE_ERROR 22 'Error writing element'
TREXIO_UNSAFE_ARRAY_DIM 23 'Access to memory beyond allocated'
TREXIO_ATTR_MISSING 24 'Attribute does not exist in the file'
TREXIO_DSET_MISSING 25 'Dataset does not exist in the file'
TREXIO_BACK_END_MISSING 26 'Requested back end is disabled'
TREXIO_INVALID_STR_LEN 30 'Invalid maxstrlen'
TREXIO_INT_SIZE_OVERFLOW 31 'Possible integer overflow'

IMPORTANT! The code below has to be executed within Emacs each time a new error code is added to the table above. Otherwise, the codes and the corresponding message are not propagated to the source code.

""" This script generates the C and Fortran constants for the error
    codes from the org-mode table.
"""

result = [ "#+begin_src c :tangle prefix_front.h :exports none" ]
for (text, code,_) in table:
  text=text.replace("~","")
  result += [ f"#define {text:30s} ((trexio_exit_code) {code:d})" ]
result += [ "#+end_src" ]

result += [ "" ]

result += [ "#+begin_src f90 :tangle prefix_fortran.f90 :exports none" ]
for (text, code,_) in table:
  text=text.replace("~","")
  result += [ f"   integer(trexio_exit_code), parameter :: {text:30s} = {code:d}" ]
result += [ "#+end_src" ]

result += [ "" ]

result += [ "#+begin_src python :tangle prefix_python.py :exports none" ]
result += [ "# define TREXIO exit codes" ]
for (text, code,_) in table:
  text=text.replace("~","")
  result += [ f"{text:30s} = {code:d}" ]
result += [ "#+end_src" ]

return '\n'.join(result)

2.1.1 Decoding errors

The trexio_string_of_error converts an exit code into a string. The string is assumed to be large enough to contain the error message (typically 128 characters).

To decode the error messages, trexio_string_of_error converts an error code into a string.

128

IMPORTANT! The code below has to be executed within Emacs each time a new error code is added to the table above. Otherwise, the codes and the corresponding message are not propagated to the source code.

2.1.1.1 C source code
const char*
trexio_string_of_error (const trexio_exit_code error)
{
  switch (error) {
  case TREXIO_FAILURE:
    return "Unknown failure";
    break;
  case TREXIO_SUCCESS:
    return "Success";
    break;
  case TREXIO_INVALID_ARG_1:
    return "Invalid argument 1";
    break;
  case TREXIO_INVALID_ARG_2:
    return "Invalid argument 2";
    break;
  case TREXIO_INVALID_ARG_3:
    return "Invalid argument 3";
    break;
  case TREXIO_INVALID_ARG_4:
    return "Invalid argument 4";
    break;
  case TREXIO_INVALID_ARG_5:
    return "Invalid argument 5";
    break;
  case TREXIO_END:
    return "End of file";
    break;
  case TREXIO_READONLY:
    return "Read-only file";
    break;
  case TREXIO_ERRNO:
    return strerror(errno);
    break;
  case TREXIO_INVALID_ID:
    return "Invalid ID";
    break;
  case TREXIO_ALLOCATION_FAILED:
    return "Allocation failed";
    break;
  case TREXIO_HAS_NOT:
    return "Element absent";
    break;
  case TREXIO_INVALID_NUM:
    return "Invalid (negative or 0) dimension";
    break;
  case TREXIO_ATTR_ALREADY_EXISTS:
    return "Attribute already exists";
    break;
  case TREXIO_DSET_ALREADY_EXISTS:
    return "Dataset already exists";
    break;
  case TREXIO_OPEN_ERROR:
    return "Error opening file";
    break;
  case TREXIO_LOCK_ERROR:
    return "Error locking file";
    break;
  case TREXIO_UNLOCK_ERROR:
    return "Error unlocking file";
    break;
  case TREXIO_FILE_ERROR:
    return "Invalid file handle";
    break;
  case TREXIO_GROUP_READ_ERROR:
    return "Error reading group";
    break;
  case TREXIO_GROUP_WRITE_ERROR:
    return "Error writing group";
    break;
  case TREXIO_ELEM_READ_ERROR:
    return "Error reading element";
    break;
  case TREXIO_ELEM_WRITE_ERROR:
    return "Error writing element";
    break;
  case TREXIO_UNSAFE_ARRAY_DIM:
    return "Access to memory beyond allocated";
    break;
  case TREXIO_ATTR_MISSING:
    return "Attribute does not exist in the file";
    break;
  case TREXIO_DSET_MISSING:
    return "Dataset does not exist in the file";
    break;
  case TREXIO_BACK_END_MISSING:
    return "Requested back end is disabled";
    break;
  case TREXIO_INVALID_STR_LEN:
    return "Invalid max_str_len";
    break;
  case TREXIO_INT_SIZE_OVERFLOW:
    return "Possible integer overflow";
    break;
  }
  return "Unknown error";
}

void
trexio_string_of_error_f (const trexio_exit_code error, char result[128])
{
  strncpy(result, trexio_string_of_error(error), 128);
}
2.1.1.2 Fortran interface
interface
   subroutine trexio_string_of_error (error, string) bind(C, name='trexio_string_of_error_f')
     use, intrinsic :: iso_c_binding
     import
     integer (trexio_exit_code), intent(in), value :: error
     character, intent(out) :: string(128)
   end subroutine trexio_string_of_error
end interface
2.1.1.3 Python interface
class Error(Exception):
    """Base class for TREXIO errors.

    Attributes:
       error: int -- exit code returned by the call to TREXIO library;
       message: str -- decoded string corresponding to trexio_exit_code.

    """

    def __init__(self, trexio_return_code):
        self.error = trexio_return_code
        self.message = string_of_error(trexio_return_code)
        super().__init__(self.message)


def string_of_error(return_code: int) -> str:
    """Decode the TREXIO exit code.

    Argument is an integer return code that correspond to one of the TREXIO errors.

    Returns string that contains description of TREXIO ~return_code~.
    """

    try:
        error_str = pytr.trexio_string_of_error(return_code)
    except:
        raise

    return error_str

2.2 Back ends

TREXIO has several back ends:

  1. TREXIO_HDF5 relies on extensive use of the HDF5 library and the associated file format. The HDF5 file is binary and tailored to high-performance I/O. This back end is the default one. HDF5 can be compiled with MPI for parallel I/O. Note, that HDF5 has to be downloaded and installed independently of TREXIO, which may cause some obstacles, especially when the user is not allowed to install external software. The produced files usually have .h5 extension.
  2. TREXIO_TEXT relies on basic file I/O in C, namely fopen, fclose, fprintf, fscanf etc. from stdio.h library. This back end is not optimized for performance. It is supposed to be used for debug purposes or, for example, when the user wants to modify some data manually within the file. This back end is supposed to work "out-of-the-box" since there are no external dependencies, which might be useful for users that do not have access to HDF5 library. The produced files usually have .txt extension.

Additional back ends can be implemented thanks to the modular nature of the front end. This can be achieved by adding a new case (corresponding to the desired back end) in the front-end switch. Then the corresponding back-end has/read/write functions has to be implemented. For example, see the commented lines that correspond to the TREXIO_JSON back end (not implemented yet).

Note: It is important to increment the value of TREXIOINVALIDBACKEND when a new back end is added. Otherwise, it will not be available.

2.2.1 C

typedef int32_t back_end_t;

#define TREXIO_HDF5             ( (back_end_t) 0 )
#define TREXIO_TEXT             ( (back_end_t) 1 )
#define TREXIO_INVALID_BACK_END ( (back_end_t) 2 )
/*#define TREXIO_JSON             ( (back_end_t) 2 )*/

#define TREXIO_DELIM            "\n"

The helper function trexio_has_backend returns true if TREXIO compilation includes a back end provided as an argument; false otherwise. This is useful due to the fact that HDF5 back end can be disabled at configure step.

bool trexio_has_backend(back_end_t back_end);
bool trexio_has_backend(back_end_t back_end) {
  switch (back_end) {
    case TREXIO_TEXT:
      return true;
    case TREXIO_HDF5:
#ifdef HAVE_HDF5
      return true;
#else
      return false;
#endif
  }
  return false;
}

2.2.2 Fortran

  integer(trexio_backend), parameter :: TREXIO_HDF5 = 0
  integer(trexio_backend), parameter :: TREXIO_TEXT = 1
! integer(trexio_backend), parameter :: TREXIO_JSON = 2
  integer(trexio_backend), parameter :: TREXIO_INVALID_BACK_END = 2

The function below is a Fortran interface for the aforementioned C-compatible trexio_has_backend function.

interface
   logical function trexio_has_backend (back_end) bind(C)
     use, intrinsic :: iso_c_binding
     integer(4), intent(in), value :: back_end
   end function trexio_has_backend
end interface

2.2.3 Python

# define TREXIO back ends
TREXIO_HDF5 = 0
TREXIO_TEXT = 1
#TREXIO_JSON = 2
TREXIO_INVALID_BACK_END = 2

2.3 Read/write behavior

Every time a reading function is called, the data is read from the disk. If data needs to be cached, this is left to the user of the library.

Writing to TREXIO files is done with transactions (all-or-nothing effect) in a per-group fashion. File writes are attempted by calling explicitly the write (TREXIO_HDF5) or flush (TREXIO_TEXT) function, or when the TREXIO file is closed. If writing is impossible because the data is not valid, no data is written.

The order in which the data is written is not necessarily consistent with the order in which the function calls were made.

The TREXIO files are supposed to be opened by only one program at a time: if the same TREXIO file is modified simultaneously by multiple concurrent programs, the behavior is not specified.

2.4 TREXIO file type

trexio_s is the the main type for TREXIO files, visible to the users of the library. This type is kept opaque, and all modifications to the files will be necessarily done through the use of functions, taking such a type as argument.

File creation and opening functions will return TREXIO file handles, namely pointers to trexio_s types. All functions accessing to the TREXIO files will have as a first argument the TREXIO file handle.

typedef struct trexio_s trexio_t;
struct trexio_s {
  char              file_name[TREXIO_MAX_FILENAME_LENGTH];
  pthread_mutex_t   thread_lock;
  back_end_t        back_end;
  char              mode;
  bool              one_based;
  char              version[16];
  char              padding[6];   /* Ensures the proper alignment of back ends */
};

2.4.1 TREXIOFile Python class

class File:
    """TREXIO File object.

    General information about the TREXIO file.

    Parameters:

    filename: str
        Is a name of the full path to the TREXIO file.
    back_end: int
        One of the currently supported TREXIO back ends.
        For example, TREXIO_HDF5 (0) or TREXIO_TEXT (1).
    mode: str
        One of the currently supported TREXIO open modes.
        For example, 'r' or 'w'.
    isOpen: bool
        Flag indicating whether the current object is still open for I/O
    pytrexio_s:
        A PyObject corresponding to SWIG proxy of the trexio_s struct in C.
        This argument is in fact a TREXIO file handle, which is required for
        communicating with the C back end.
    info: dict
        Dictionary of key-value pairs with additional information about the file.
    """


    def __init__(self, filename, mode='r', back_end=TREXIO_HDF5,
                 pytrexio_s=None, info=None):
        """This is a TREXIO File constructor."""

        self.filename = filename
        self.mode = mode
        self.back_end = back_end

        self.isOpen = False
        if pytrexio_s is None:
            self.pytrexio_s = open(filename, mode, back_end)
            self.isOpen = True
        else:
            self.pytrexio_s = pytrexio_s
            self.isOpen = None

        self.info = info


    def close(self):
        """Close a TREXIO File."""

        if self.isOpen:
            close(self.pytrexio_s)
            self.isOpen = False
        else:
            raise Exception("TREXIO File object has not been opened.")


    def __del__(self):
        """This is a TREXIO File destructor."""

        if self.isOpen:
            close(self.pytrexio_s)
        elif self.isOpen is None:
            raise Exception("[WIP]: TREXIO file handle provided but what if the file is already closed?")
        else:
            pass

2.5 Polymorphism of the file handle

Polymorphism of the trexio_t type is handled by ensuring that the corresponding types for all back ends can be safely casted to trexio_t. This is done by making the back-end structs start with struct trexio_s:

struct trexio_back_end_s {
  trexio_t     parent ;
  /* add below specific back-end data */
}

2.6 File opening

trexio_open creates a new TREXIO file or opens existing one.

input parameters:

  1. file_name - string containing file name
  2. mode - character containing open mode (see below)
    • 'w' - (write) creates a new file as READWRITE (overwrite existing file)
    • 'r' - (read) opens existing file as READONLY
  3. back_end - integer number (or the corresponding global parameter) specifying the back end
    • TREXIO_HDF5 - for HDF5 back end (integer alternative: 0)
    • TREXIO_TEXT - for TEXT back end (integer alternative: 1)

output: trexio_t file handle

Note: the file_name in TEXT back end actually corresponds to the name of the directory where .txt data files are stored. The actual name of each .txt file corresponds to the group name provided in trex.config (e.g. nucleus.txt for nuclei-related data). These names are populated by the generator.py (i.e. they are hard-coded), which is why the user should tend to avoid renaming the .txt data files.

2.6.1 C

trexio_t*
trexio_open(const char* file_name, const char mode,
            const back_end_t back_end, trexio_exit_code* const rc_open)
{

  if (file_name == NULL || file_name[0] == '\0') {
    if (rc_open != NULL) *rc_open = TREXIO_INVALID_ARG_1;
    return NULL;
  }
  /* Check overflow in file_name */

  if (back_end <  0 || back_end >= TREXIO_INVALID_BACK_END) {
    if (rc_open != NULL) *rc_open = TREXIO_INVALID_ARG_3;
    return NULL;
  }

  if (mode != 'r' && mode != 'w') {
    if (rc_open != NULL) *rc_open = TREXIO_INVALID_ARG_2;
    return NULL;
  }

  trexio_t* result = NULL;
  void* result_tmp = NULL;

  /* Allocate data structures */
  switch (back_end) {

  case TREXIO_TEXT:
    result_tmp = malloc(sizeof(trexio_text_t));
    break;

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    result_tmp = malloc(sizeof(trexio_hdf5_t));
    break;
#else
    if (rc_open != NULL) *rc_open = TREXIO_BACK_END_MISSING;
    return NULL;
#endif
/*
  case TREXIO_JSON:
    result = (trexio_t*) malloc (sizeof(trexio_json_t));
    break;
*/
  }
  result = (trexio_t*) result_tmp;

  assert (result != NULL);    /* TODO: Error handling */

  /* Data for the parent type */

  strncpy(result->file_name, file_name, TREXIO_MAX_FILENAME_LENGTH);
  if (result->file_name[TREXIO_MAX_FILENAME_LENGTH-1] != '\0') {
    if (rc_open != NULL) *rc_open = TREXIO_INVALID_ARG_1;
    free(result);
    return NULL;
  }

  strncpy(result->version, TREXIO_PACKAGE_VERSION, 16);
  if (result->version[15] != '\0') {
    if (rc_open != NULL) *rc_open = TREXIO_FAILURE;
    free(result);
    return NULL;
  }

  result->back_end    = back_end;
  result->mode        = mode;
  result->one_based   = false;  // Need to be flipped in Fortran interface
  int irc = pthread_mutex_init ( &(result->thread_lock), NULL);
  if (irc != 0) {
    if (rc_open != NULL) *rc_open = TREXIO_FAILURE;
    free(result);
    return NULL;
  }
  // assert (irc == 0);

  trexio_exit_code rc;

  /* Back end initialization */

  rc = TREXIO_OPEN_ERROR;

  switch (back_end) {

  case TREXIO_TEXT:
    rc = trexio_text_init(result);
    break;

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    rc = trexio_hdf5_init(result);
    break;
#else
    if (rc_open != NULL) *rc_open = TREXIO_BACK_END_MISSING;
    return NULL;
#endif
/*
  case TREXIO_JSON:
    rc = trexio_json_init(result);
    break;
*/
  }

  if (rc != TREXIO_SUCCESS) {
    if (rc_open != NULL) *rc_open = TREXIO_OPEN_ERROR;
    free(result);
    return NULL;
  }


  /* File locking */

  rc = TREXIO_LOCK_ERROR;

  switch (back_end) {

  case TREXIO_TEXT:
    rc = trexio_text_lock(result);
    break;
  /* HDF5 v.>=1.10 has file locking activated by default */
  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    rc = TREXIO_SUCCESS;
    break;
#else
    if (rc_open != NULL) *rc_open = TREXIO_BACK_END_MISSING;
    return NULL;
#endif
/*
  case TREXIO_JSON:
    rc = trexio_json_lock(result);
    break;
*/
  }

  if (rc != TREXIO_SUCCESS) {
    if (rc_open != NULL) *rc_open = TREXIO_LOCK_ERROR;
    free(result);
    return NULL;
  }

  /* Write metadata (i.e. package version) upon creation */
  rc = trexio_has_metadata_package_version(result);
  if (rc == TREXIO_FAILURE) {
    if (rc_open != NULL) *rc_open = TREXIO_OPEN_ERROR;
    free(result);
    return NULL;
  }

  if (rc == TREXIO_HAS_NOT) {
    switch (back_end) {

    case TREXIO_TEXT:
      rc = trexio_text_write_metadata_package_version(result, TREXIO_PACKAGE_VERSION);
      break;

    case TREXIO_HDF5:
#ifdef HAVE_HDF5
      rc = trexio_hdf5_write_metadata_package_version(result, TREXIO_PACKAGE_VERSION);
      break;
#else
      if (rc_open != NULL) *rc_open = TREXIO_BACK_END_MISSING;
      return NULL;
#endif

    }
  }

  if (rc != TREXIO_SUCCESS) {
    if (rc_open != NULL) *rc_open = TREXIO_OPEN_ERROR;
    free(result);
    return NULL;
  }

  /* Exit upon success */
  if (rc_open != NULL) *rc_open = TREXIO_SUCCESS;

  return result;
}

2.6.2 Fortran

interface
   integer(8) function trexio_open_c (filename, mode, backend, rc_open) bind(C, name="trexio_open")
     use, intrinsic :: iso_c_binding
     import
     character(kind=c_char), dimension(*)       :: filename
     character, intent(in), value               :: mode
     integer(trexio_backend), intent(in), value :: backend
     integer(trexio_exit_code), intent(out)     :: rc_open
   end function trexio_open_c
end interface

2.6.3 Python

def open(file_name: str, mode: str, back_end: int):
    """Create TREXIO file or open existing one.

    Parameters:

    file_name: str
        Name of the TREXIO file

    mode: str
        One of the currently supported ~open~ modes (e.g. 'w', 'r')

    back_end: int
        One of the currently supported TREXIO back ends (e.g. TREXIO_HDF5, TREXIO_TEXT)

    Return:
        SWIG object of type trexio_s.

    Examples:
    >>> from trexio import open as tr_open
    >>> trex_file = tr_open("example.h5", "w", TREXIO_HDF5)
    """

    # The new trexio_open function is capable of returning error code which SWIG can append to the output trexio_s file struct
    # However, if trexio_s* == NULL, then SWIG returns only an error code rc_open instead of a list [trexio_s, rc_open]
    # Thus, the following try/except sequence is needed
    try:
        return_obj = pytr.trexio_open(file_name, mode, back_end)
        assert return_obj is not None
        if isinstance(return_obj, int):
            raise Error(return_obj)
        else:
            rc_open = return_obj[1]
            # this is a sanity check in case the code evolves and SWIG issue is patched
            if rc_open == TREXIO_SUCCESS:
                trexio_file = return_obj[0]
                assert trexio_file is not None
    except AssertionError:
        raise Exception(f"Could not open TREXIO file {file_name} using trexio_open function. The return value is None (NULL pointer).")

    return trexio_file

2.6.4 Zero-based versus one-based arrays of indices

Because arrays are zero-based in Fortran, we need to set a flag to know if we need to shift by 1 arrays of indices.

trexio_exit_code trexio_set_one_based(trexio_t* file)
{
  if (file == NULL)
    return TREXIO_FILE_ERROR;

  file->one_based = true;

  return TREXIO_SUCCESS;
}
interface
   integer function trexio_set_one_based(trex_file) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
   end function trexio_set_one_based
end interface

2.7 File closing

trexio_close closes an existing trexio_t file.

input parameters: file – TREXIO file handle.

output: trexio_exit_code exit code.

2.7.1 C

trexio_exit_code
trexio_close (trexio_t* file)
{

  if (file == NULL) return TREXIO_FILE_ERROR;

  trexio_exit_code rc = TREXIO_FAILURE;

  assert(file->back_end < TREXIO_INVALID_BACK_END);

  /* Terminate the back end */
  switch (file->back_end) {

  case TREXIO_TEXT:
    rc = trexio_text_deinit(file);
    break;

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    rc = trexio_hdf5_deinit(file);
    break;
#else
    FREE(file);
    return TREXIO_BACK_END_MISSING;
#endif
/*
  case TREXIO_JSON:
    rc = trexio_json_deinit(file);
    break;
*/
  }

  if (rc != TREXIO_SUCCESS) {
    FREE(file);
    return rc;
  }

  /* File unlocking */

  rc = TREXIO_UNLOCK_ERROR;

  switch (file->back_end) {

  case TREXIO_TEXT:
    rc = trexio_text_unlock(file);
    break;

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    rc = TREXIO_SUCCESS;
    break;
#else
    FREE(file);
    return TREXIO_BACK_END_MISSING;
#endif
/*
  case TREXIO_JSON:
    rc = trexio_json_unlock(file);
    break;
*/
  }

  /* Terminate front end */

  int irc = pthread_mutex_destroy( &(file->thread_lock) );

  free(file);

  if (irc != 0) return TREXIO_ERRNO;
  if (rc != TREXIO_SUCCESS) return rc;

  return TREXIO_SUCCESS;
}

2.7.2 Fortran

interface
   integer function trexio_close (trex_file) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
   end function trexio_close
end interface

2.7.3 Python

def close(trexio_file):
    """Close TREXIO file.

    Parameter is a ~trexio_file~ object that has been created by a call to ~open~ function.
    """

    try:
        rc = pytr.trexio_close(trexio_file)
        if rc != TREXIO_SUCCESS:
            raise Error(rc)
    except:
        raise

3 Templates for front end

3.1 Description

Consider the following block of trex.json:

{
  "nucleus": {
      "num"                : [ "int"  , [                     ] ]
    , "charge"             : [ "float", [ "nucleus.num"       ] ]
    , "coord"              : [ "float", [ "nucleus.num", "3"  ] ]
    , "label"              : [ "str" ,  [ "nucleus.num"       ] ]
    , "point_group"        : [ "str"  , [                     ] ]
  }
}

TREXIO is generated automatically by the generator.py Python script based on the tree-like configuration provided in the trex.json file. Because of that, generalized templates can be implemented and re-used. This approach minimizes the number of bugs as compared with manual copy-paste-modify scheme.

All templates presented below use the $var$ notation to indicate the variable, which will be replaced by the generator.py. Sometimes the upper case is used, i.e. $VAR$ (for example, in #define statements). More detailed description of each variable can be found below:

Template variable Description Example
$group$ Name of the group nucleus
$group_num$ Name of the numerical attribute (scalar) nucleus_num
$group_str$ Name of the string attribute (scalar) nucleus_point_group
$group_dset$ Name of the dataset (vector/matrix/tensor) nucleus_coord
$group_dset_rank$ Rank of the dataset 2
$group_dset_dim$ Selected dimension of the dataset nucleus_num
$group_dset_dim_list$ All dimensions of the dataset {nucleus_num, 3}
$group_dset_dtype$ Basic type of the dataset (int/float/char) float
$group_dset_h5_dtype$ Type of the dataset in HDF5 double
$group_dset_format_scanf$ Input type of the dataset in TEXT [fscanf] %lf
$group_dset_format_printf$ Output type of the dataset in TEXT [fprintf] %24.16e
$group_dset_dtype_default$ Default datatype of the dataset [C] double/int32_t
$group_dset_dtype_single$ Single precision datatype of the dataset [C] float/int32_t
$group_dset_dtype_double$ Double precision datatype of the dataset [C] double/int64_t
$group_dset_f_dtype_default$ Default datatype of the dataset [Fortran] real(8)/integer(4)
$group_dset_f_dtype_single$ Single precision datatype of the dataset [Fortran] real(4)/integer(4)
$group_dset_f_dtype_double$ Double precision datatype of the dataset [Fortran] real(8)/integer(8)
$group_dset_f_dims$ Dimensions in Fortran (:,:)
$group_dset_py_dtype$ Standard datatype of the dataset [Python] float/int
$default_prec$ Default precision for read/write without suffix [C] 64/32
$is_index$ Expands to true if dataset has a type index [C] true/false

Some of the aforementioned template variables with group_dset prefix are duplicated with group_num prefix, e.g. you might find \(group_num_dtype_double\) in the templates corresponding to numerical attributes. The expanding values are the same as for group_dset and thus are not listed in the table above.

Note: parent group name is always added to the child objects upon construction of TREXIO (e.g. num of nucleus group becomes nucleus_num and should be accessed accordingly within TREXIO).

TREXIO generator parses the trex.json file. TREXIO operates with names of variables based on the 1-st (parent group) and 2-nd (child object) levels of trex.json . The parsed data is divided in 2 parts:

  1. Single attributes. These can be numerical values or strings.
  2. Datasets. These can be vectors, matrices or tensors. The types are indicated in trex.json. Currently supported data types: int, float and strings.

For each of the aforementioned objects, TREXIO provides has, read and write functionality. TREXIO supports I/O with single or double precision for integer and floating point numbers.

Note: single integer attributes that contain num in their name (e.g. nucleus_num) are considered dimensioning variables and cannot be negative or 0. An attempt to write negative or 0 value will result in TREXIO_INVALID_ARG_2 exit code.

3.2 Templates for front end has/read/write a single numerical attribute

3.2.1 Introduction

This section concerns API calls related to numerical attributes, namely single value of int/float types.

Function name Description Precision
trexio_has_$group_num$ Check if an attribute exists in a file ---
trexio_read_$group_num$ Read a attribute Single
trexio_write_$group_num$ Write a attribute Single
trexio_read_$group_num$_32 Read a attribute Single
trexio_write_$group_num$_32 Write a attribute Single
trexio_read_$group_num$_64 Read a attribute Double
trexio_write_$group_num$_64 Write a attribute Double

3.2.2 C templates for front end

The C templates that correspond to each of the abovementioned functions can be found below. First parameter is the TREXIO file handle. Second parameter is the variable to be written/read to/from the TREXIO file (except for trexio_has_ functions). Suffixes _32 and _64 correspond to API calls dealing with single and double precision, respectively. The basic (non-suffixed) API call on dimensioning variables deals with single precision (see Table above).

3.2.2.1 Function declarations
3.2.2.2 Source code for double precision functions
trexio_exit_code
trexio_read_$group_num$_64 (trexio_t* const file, $group_num_dtype_double$* const num)
{
  if (file == NULL) return TREXIO_INVALID_ARG_1;
  if (trexio_has_$group_num$(file) != TREXIO_SUCCESS) return TREXIO_ATTR_MISSING;

  switch (file->back_end) {

  case TREXIO_TEXT:
    return trexio_text_read_$group_num$(file, num);

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    return trexio_hdf5_read_$group_num$(file, num);
#else
    return TREXIO_BACK_END_MISSING;
#endif
/*
  case TREXIO_JSON:
    return trexio_json_read_$group_num$(file, num);
*/
  }

  return TREXIO_FAILURE;
}
trexio_exit_code
trexio_write_$group_num$_64 (trexio_t* const file, const $group_num_dtype_double$ num)
{
  if (file == NULL) return TREXIO_INVALID_ARG_1;
  //if (num <= 0L) return TREXIO_INVALID_NUM; /* this line is uncommented by the generator for dimensioning variables; do NOT remove! */
  if (trexio_has_$group_num$(file) == TREXIO_SUCCESS) return TREXIO_ATTR_ALREADY_EXISTS;

  switch (file->back_end) {

  case TREXIO_TEXT:
    return trexio_text_write_$group_num$(file, num);

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    return trexio_hdf5_write_$group_num$(file, num);
#else
    return TREXIO_BACK_END_MISSING;
#endif
/*
  case TREXIO_JSON:
    return trexio_json_write_$group_num$(file, num);
*/
  }

  return TREXIO_FAILURE;
}
3.2.2.3 Source code for single precision functions
trexio_exit_code
trexio_read_$group_num$_32 (trexio_t* const file, $group_num_dtype_single$* const num)
{
  if (file == NULL) return TREXIO_INVALID_ARG_1;
  if (trexio_has_$group_num$(file) != TREXIO_SUCCESS) return TREXIO_ATTR_MISSING;

  $group_num_dtype_double$ num_64 = 0;
  trexio_exit_code rc = TREXIO_GROUP_READ_ERROR;

  switch (file->back_end) {

  case TREXIO_TEXT:
    rc = trexio_text_read_$group_num$(file, &num_64);
    break;

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    rc = trexio_hdf5_read_$group_num$(file, &num_64);
    break;
#else
    rc = TREXIO_BACK_END_MISSING;
    break ;
#endif
/*
  case TREXIO_JSON:
    rc =trexio_json_read_$group_num$(file, &num_64);
    break;
*/
  }

  if (rc != TREXIO_SUCCESS) return rc;

  *num = ($group_num_dtype_single$) num_64;
  return TREXIO_SUCCESS;
}
trexio_exit_code
trexio_write_$group_num$_32 (trexio_t* const file, const $group_num_dtype_single$ num)
{

  if (file == NULL) return TREXIO_INVALID_ARG_1;
  //if (num <= 0) return TREXIO_INVALID_NUM; /* this line is uncommented by the generator for dimensioning variables; do NOT remove! */
  if (trexio_has_$group_num$(file) == TREXIO_SUCCESS) return TREXIO_ATTR_ALREADY_EXISTS;

  switch (file->back_end) {

  case TREXIO_TEXT:
    return trexio_text_write_$group_num$(file, ($group_num_dtype_double$) num);

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    return trexio_hdf5_write_$group_num$(file, ($group_num_dtype_double$) num);
#else
    return TREXIO_BACK_END_MISSING;
#endif
/*
  case TREXIO_JSON:
    return trexio_json_write_$group_num$(file, ($group_num_dtype_double$) num);
    break;
*/
  }

  return TREXIO_FAILURE;
}
3.2.2.4 Source code for default functions
trexio_exit_code
trexio_read_$group_num$ (trexio_t* const file, $group_num_dtype_default$* const num)
{
  return trexio_read_$group_num$_$default_prec$(file, num);
}
trexio_exit_code
trexio_write_$group_num$ (trexio_t* const file, const $group_num_dtype_default$ num)
{
  return trexio_write_$group_num$_$default_prec$(file, num);
}
trexio_exit_code
trexio_has_$group_num$ (trexio_t* const file)
{

  if (file == NULL) return TREXIO_INVALID_ARG_1;

  assert(file->back_end < TREXIO_INVALID_BACK_END);

  switch (file->back_end) {

  case TREXIO_TEXT:
    return trexio_text_has_$group_num$(file);

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    return trexio_hdf5_has_$group_num$(file);
#else
    return TREXIO_BACK_END_MISSING;
#endif
/*
  case TREXIO_JSON:
    return trexio_json_has_$group_num$(file);
    break;
*/
  }

  return TREXIO_FAILURE;
}

3.2.3 Fortran templates for front end

The Fortran templates that provide an access to the C API calls from Fortran. These templates are based on the use of iso_c_binding. Pointers have to be passed by value.

interface
   integer function trexio_write_$group_num$_64 (trex_file, num) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     $group_num_f_dtype_double$, intent(in), value :: num
   end function trexio_write_$group_num$_64
end interface
interface
   integer function trexio_read_$group_num$_64 (trex_file, num) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     $group_num_f_dtype_double$, intent(out) :: num
   end function trexio_read_$group_num$_64
end interface
interface
   integer function trexio_write_$group_num$_32 (trex_file, num) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     $group_num_f_dtype_single$, intent(in), value :: num
   end function trexio_write_$group_num$_32
end interface
interface
   integer function trexio_read_$group_num$_32 (trex_file, num) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     $group_num_f_dtype_single$, intent(out) :: num
   end function trexio_read_$group_num$_32
end interface
interface
   integer function trexio_write_$group_num$ (trex_file, num) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     $group_num_f_dtype_default$, intent(in), value :: num
   end function trexio_write_$group_num$
end interface
interface
   integer function trexio_read_$group_num$ (trex_file, num) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     $group_num_f_dtype_default$, intent(out) :: num
   end function trexio_read_$group_num$
end interface
interface
   integer function trexio_has_$group_num$ (trex_file) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
   end function trexio_has_$group_num$
end interface

3.2.4 Python templates for front end

def write_$group_num$(trexio_file, num_w: $group_num_py_dtype$) -> None:
    """Write the $group_num$ variable in the TREXIO file.

    Parameters:

    trexio_file:
        TREXIO File object.

    num_w: int
        Value of the $group_num$ variable to be written.

    Raises:
        - Exception from AssertionError if TREXIO return code ~rc~ is different from TREXIO_SUCCESS and prints the error message using trexio_string_of_error.
        - Exception from some other error (e.g. RuntimeError).
    """

    try:
        rc = pytr.trexio_write_$group_num$(trexio_file.pytrexio_s, num_w)
        if rc != TREXIO_SUCCESS:
            raise Error(rc)
    except:
        raise
def read_$group_num$(trexio_file) -> $group_num_py_dtype$:
    """Read the $group_num$ variable from the TREXIO file.

    Parameter is a ~TREXIO File~ object that has been created by a call to ~open~ function.

    Returns:
        ~num_r~: int
        Integer value of $group_num$ variable read from ~trexio_file~.

    Raises:
        - Exception from AssertionError if TREXIO return code ~rc~ is different from TREXIO_SUCCESS and prints the error message using trexio_string_of_error.
        - Exception from some other error (e.g. RuntimeError).
    """

    try:
        rc, num_r = pytr.trexio_read_$group_num$(trexio_file.pytrexio_s)
        if rc != TREXIO_SUCCESS:
            raise Error(rc)
    except:
        raise

    return num_r
def has_$group_num$(trexio_file) -> bool:
    """Check that $group_num$ variable exists in the TREXIO file.

    Parameter is a ~TREXIO File~ object that has been created by a call to ~open~ function.

    Returns:
          True if the variable exists, False otherwise

    Raises:
        - Exception from trexio.Error class if TREXIO return code ~rc~ is TREXIO_FAILURE and prints the error message using string_of_error.
        - Exception from some other error (e.g. RuntimeError).
    """

    try:
        rc = pytr.trexio_has_$group_num$(trexio_file.pytrexio_s)
        if rc == TREXIO_FAILURE:
            raise Error(rc)
    except:
        raise

    if rc == TREXIO_SUCCESS:
        return True
    else:
        return False

3.3 Templates for front end has/read/write a dataset of numerical data

3.3.1 Introduction

This section concerns API calls related to datasets.

Function name Description Precision
trexio_has_$group_dset$ Check if a dataset exists in a file ---
trexio_read_$group_dset$ Read a dataset in default precision Double/Single for float/int
trexio_write_$group_dset$ Write a dataset in default precision Double/Single for float/int
trexio_read_safe_$group_dset$ Read a bounded dataset Double
trexio_write_safe_$group_dset$ Write a bounded dataset Double
trexio_read_$group_dset$_32 Read a dataset in single precision Single
trexio_write_$group_dset$_32 Write a dataset in single precision Single
trexio_read_$group_dset$_64 Read a dataset in double precision Double
trexio_write_$group_dset$_64 Write a dataset in double precision Double

3.3.2 C templates for front end

The C templates that correspond to each of the abovementioned functions can be found below. First parameter is the TREXIO file handle. Second parameter is the variable to be written/read to/from the TREXIO file (except for trexio_has_ functions). Suffixes _32 and _64 correspond to API calls dealing with single and double precision, respectively. The basic (non-suffixed) API call on datasets deals with double precision (see Table above).

3.3.2.1 Function declarations
3.3.2.2 Source code for double precision functions
trexio_exit_code
trexio_read_$group_dset$_64 (trexio_t* const file, $group_dset_dtype_double$* const $group_dset$)
{

  if (file  == NULL) return TREXIO_INVALID_ARG_1;
  if ($group_dset$ == NULL) return TREXIO_INVALID_ARG_2;
  if (trexio_has_$group_dset$(file) != TREXIO_SUCCESS) return TREXIO_DSET_MISSING;

  trexio_exit_code rc;
  int64_t $group_dset_dim$ = 0;

  /* Error handling for this call is added by the generator */
  rc = trexio_read_$group_dset_dim$_64(file, &($group_dset_dim$));

  if ($group_dset_dim$ == 0L) return TREXIO_INVALID_NUM;

  uint32_t rank = $group_dset_rank$;
  uint64_t dims[$group_dset_rank$] = {$group_dset_dim_list$};

  assert(file->back_end < TREXIO_INVALID_BACK_END);

  rc = TREXIO_FAILURE;
  switch (file->back_end) {

  case TREXIO_TEXT:
    rc = trexio_text_read_$group_dset$(file, $group_dset$, rank, dims);
    break;

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    rc = trexio_hdf5_read_$group_dset$(file, $group_dset$, rank, dims);
    break;
#else
    rc = TREXIO_BACK_END_MISSING;
    break;
#endif
/*
  case TREXIO_JSON:
    rc = trexio_json_read_$group_dset$(file, $group_dset$, rank, dims);
    break;
*/
  }
  if (rc != TREXIO_SUCCESS) return rc;

  /* Handle index type */
  if ($is_index$) {
    uint64_t dim_size = 1;
    for (uint32_t i=0; i<rank; ++i){
        dim_size *= dims[i];
    }
    for (uint64_t i=0; i<dim_size; ++i){
       $group_dset$[i] += ($group_dset_dtype_double$) 1;
    }
  }

  return TREXIO_SUCCESS;
}
trexio_exit_code
trexio_write_$group_dset$_64 (trexio_t* const file, const $group_dset_dtype_double$* $group_dset$)
{

  if (file  == NULL) return TREXIO_INVALID_ARG_1;
  assert(file->back_end < TREXIO_INVALID_BACK_END);

  if ($group_dset$ == NULL) return TREXIO_INVALID_ARG_2;
  if (trexio_has_$group_dset$(file) == TREXIO_SUCCESS) return TREXIO_DSET_ALREADY_EXISTS;


  trexio_exit_code rc;
  int64_t $group_dset_dim$ = 0;

  /* Error handling for this call is added by the generator */
  rc = trexio_read_$group_dset_dim$_64(file, &($group_dset_dim$));

  if ($group_dset_dim$ == 0L) return TREXIO_INVALID_NUM;

  uint32_t rank = $group_dset_rank$;
  uint64_t dims[$group_dset_rank$] = {$group_dset_dim_list$};

  if ($is_index$) {
    /* Handle index type : is_index = $is_index$ */
    uint64_t dim_size = 1;
    for (uint32_t i=0; i<rank; ++i){
      dim_size *= dims[i];
    }

    $group_dset_dtype_double$* $group_dset$_p =
       CALLOC(dim_size, $group_dset_dtype_double$);

    if ($group_dset$_p == NULL) return TREXIO_ALLOCATION_FAILED;

    for (uint64_t i=0; i<dim_size; ++i){
       $group_dset$_p[i] = $group_dset$[i] - ($group_dset_dtype_double$) 1;
    }

    rc = TREXIO_FAILURE;
    switch (file->back_end) {

    case TREXIO_TEXT:
      rc = trexio_text_write_$group_dset$(file,
                                          (const $group_dset_dtype_double$*) $group_dset$_p,
                                          rank, dims);
      break;

    case TREXIO_HDF5:
#ifdef HAVE_HDF5
      rc = trexio_hdf5_write_$group_dset$(file, 
                                          (const $group_dset_dtype_double$*) $group_dset$_p,
                                          rank, dims);
      break;
#else
      rc = TREXIO_BACK_END_MISSING;
      break;
#endif
      /*
        case TREXIO_JSON:
        rc = trexio_json_write_$group_dset$(file, $group_dset$_p, rank, dims);
        break;
      */
      FREE($group_dset$_p);

    }

  } else {

    rc = TREXIO_FAILURE;
    switch (file->back_end) {

    case TREXIO_TEXT:
      rc = trexio_text_write_$group_dset$(file, $group_dset$, rank, dims);
      break;

    case TREXIO_HDF5:
#ifdef HAVE_HDF5
      rc = trexio_hdf5_write_$group_dset$(file, $group_dset$, rank, dims);
      break;
#else
      rc = TREXIO_BACK_END_MISSING;
      break;
#endif
      /*
        case TREXIO_JSON:
        rc = trexio_json_write_$group_dset$(file, $group_dset$, rank, dims);
        break;
      */
    }
  }

  return rc;
}
3.3.2.3 Source code for single precision functions
trexio_exit_code
trexio_read_$group_dset$_32 (trexio_t* const file, $group_dset_dtype_single$* const $group_dset$)
{

  if (file  == NULL) return TREXIO_INVALID_ARG_1;
  if ($group_dset$ == NULL) return TREXIO_INVALID_ARG_2;
  if (trexio_has_$group_dset$(file) != TREXIO_SUCCESS) return TREXIO_DSET_MISSING;

  trexio_exit_code rc;
  int64_t $group_dset_dim$ = 0;

  /* Error handling for this call is added by the generator */
  rc = trexio_read_$group_dset_dim$_64(file, &($group_dset_dim$));

  if ($group_dset_dim$ == 0L) return TREXIO_INVALID_NUM;

  uint32_t rank = $group_dset_rank$;
  uint64_t dims[$group_dset_rank$] = {$group_dset_dim_list$};

  uint64_t dim_size = 1;
  for (uint32_t i=0; i<rank; ++i){
    dim_size *= dims[i];
  }

  $group_dset_dtype_double$* $group_dset$_64 = CALLOC(dim_size, $group_dset_dtype_double$);
  if ($group_dset$_64 == NULL) return TREXIO_ALLOCATION_FAILED;

  assert(file->back_end < TREXIO_INVALID_BACK_END);

  rc = TREXIO_FAILURE;

  switch (file->back_end) {

  case TREXIO_TEXT:
    rc = trexio_text_read_$group_dset$(file, $group_dset$_64, rank, dims);
    break;

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    rc = trexio_hdf5_read_$group_dset$(file, $group_dset$_64, rank, dims);
    break;
#else
    rc = TREXIO_BACK_END_MISSING;
    break;
#endif
/*
  case TREXIO_JSON:
    rc = trexio_json_read_$group_dset$(file, $group_dset$_64, rank, dims);
    break;
*/
  }

  if (rc != TREXIO_SUCCESS){
    FREE($group_dset$_64);
    return rc;
  }

  if ($is_index$) {
    for (uint64_t i=0; i<dim_size; ++i){
      $group_dset$[i] = ($group_dset_dtype_single$) $group_dset$_64[i] + 1;
    }
  } else {
    for (uint64_t i=0; i<dim_size; ++i){
      $group_dset$[i] = ($group_dset_dtype_single$) $group_dset$_64[i];
    }
  }

  FREE($group_dset$_64);
  return TREXIO_SUCCESS;
}
trexio_exit_code
trexio_write_$group_dset$_32 (trexio_t* const file, const $group_dset_dtype_single$* $group_dset$)
{

  if (file  == NULL) return TREXIO_INVALID_ARG_1;
  if ($group_dset$ == NULL) return TREXIO_INVALID_ARG_2;
  if (trexio_has_$group_dset$(file) == TREXIO_SUCCESS) return TREXIO_DSET_ALREADY_EXISTS;

  trexio_exit_code rc;
  int64_t $group_dset_dim$ = 0;

  /* Error handling for this call is added by the generator */
  rc = trexio_read_$group_dset_dim$_64(file, &($group_dset_dim$));

  if ($group_dset_dim$ == 0L) return TREXIO_INVALID_NUM;

  uint32_t rank = $group_dset_rank$;
  uint64_t dims[$group_dset_rank$] = {$group_dset_dim_list$};

  uint64_t dim_size = 1;
  for (uint32_t i=0; i<rank; ++i){
    dim_size *= dims[i];
  }

  $group_dset_dtype_double$* $group_dset$_64 = CALLOC(dim_size, $group_dset_dtype_double$);
  if ($group_dset$_64 == NULL) return TREXIO_ALLOCATION_FAILED;

  /* A type conversion from single precision to double reqired since back end only accepts 64-bit data */
  if ($is_index$) {
    for (uint64_t i=0; i<dim_size; ++i){
      $group_dset$_64[i] = ($group_dset_dtype_double$) $group_dset$[i] - ($group_dset_dtype_double$) 1;
    }
  } else {
    for (uint64_t i=0; i<dim_size; ++i){
      $group_dset$_64[i] = ($group_dset_dtype_double$) $group_dset$[i];
    }
  }

  assert(file->back_end < TREXIO_INVALID_BACK_END);

  rc = TREXIO_FAILURE;
  switch (file->back_end) {

  case TREXIO_TEXT:
    rc = trexio_text_write_$group_dset$(file, $group_dset$_64, rank, dims);
    break;

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    rc = trexio_hdf5_write_$group_dset$(file, $group_dset$_64, rank, dims);
    break;
#else
    rc = TREXIO_BACK_END_MISSING;
    break;
#endif
/*
  case TREXIO_JSON:
    rc = trexio_json_write_$group_dset$(file, $group_dset$_64, rank, dims);
    break;
*/
  }

  FREE($group_dset$_64);

  if (rc != TREXIO_SUCCESS) return rc;

  return TREXIO_SUCCESS;
}
3.3.2.4 Source code for memory-safe functions
trexio_exit_code rc;
int64_t $group_dset_dim$ = 0;

/* Error handling for this call is added by the generator */
rc = trexio_read_$group_dset_dim$_64(file, &($group_dset_dim$));

if ($group_dset_dim$ == 0L) return TREXIO_INVALID_NUM;

uint32_t rank = $group_dset_rank$;
uint64_t dims[$group_dset_rank$] = {$group_dset_dim_list$};

/* The block below is specific to safe API as it checks the boundaries */
uint64_t dim_size = 1;
for (uint32_t i=0; i<rank; ++i){
    dim_size *= dims[i];
}
trexio_exit_code
trexio_read_safe_$group_dset$_32 (trexio_t* const file, $group_dset_dtype_single$* const dset_out, const int64_t dim_out)
{

  if (file  == NULL) return TREXIO_INVALID_ARG_1;
  if (dset_out == NULL) return TREXIO_INVALID_ARG_2;
  if (trexio_has_$group_dset$(file) != TREXIO_SUCCESS) return TREXIO_DSET_MISSING;

trexio_exit_code rc;
int64_t $group_dset_dim$ = 0;

/* Error handling for this call is added by the generator */
rc = trexio_read_$group_dset_dim$_64(file, &($group_dset_dim$));

if ($group_dset_dim$ == 0L) return TREXIO_INVALID_NUM;

uint32_t rank = $group_dset_rank$;
uint64_t dims[$group_dset_rank$] = {$group_dset_dim_list$};

/* The block below is specific to safe API as it checks the boundaries */
uint64_t dim_size = 1;
for (uint32_t i=0; i<rank; ++i){
    dim_size *= dims[i];
}

  if (dim_out > (int64_t) dim_size) return TREXIO_UNSAFE_ARRAY_DIM;

  return trexio_read_$group_dset$_32(file, dset_out);
}
trexio_exit_code
trexio_write_safe_$group_dset$_32 (trexio_t* const file, const $group_dset_dtype_single$* dset_in, const int64_t dim_in)
{

  if (file  == NULL) return TREXIO_INVALID_ARG_1;
  if (dset_in == NULL) return TREXIO_INVALID_ARG_2;
  if (trexio_has_$group_dset$(file) == TREXIO_SUCCESS) return TREXIO_DSET_ALREADY_EXISTS;

trexio_exit_code rc;
int64_t $group_dset_dim$ = 0;

/* Error handling for this call is added by the generator */
rc = trexio_read_$group_dset_dim$_64(file, &($group_dset_dim$));

if ($group_dset_dim$ == 0L) return TREXIO_INVALID_NUM;

uint32_t rank = $group_dset_rank$;
uint64_t dims[$group_dset_rank$] = {$group_dset_dim_list$};

/* The block below is specific to safe API as it checks the boundaries */
uint64_t dim_size = 1;
for (uint32_t i=0; i<rank; ++i){
    dim_size *= dims[i];
}

  if (dim_in > (int64_t) dim_size) return TREXIO_UNSAFE_ARRAY_DIM;

  return trexio_write_$group_dset$_32(file, dset_in);
}
trexio_exit_code
trexio_read_safe_$group_dset$_64 (trexio_t* const file, $group_dset_dtype_double$* const dset_out, const int64_t dim_out)
{

  if (file  == NULL) return TREXIO_INVALID_ARG_1;
  if (dset_out == NULL) return TREXIO_INVALID_ARG_2;
  if (trexio_has_$group_dset$(file) != TREXIO_SUCCESS) return TREXIO_DSET_MISSING;

trexio_exit_code rc;
int64_t $group_dset_dim$ = 0;

/* Error handling for this call is added by the generator */
rc = trexio_read_$group_dset_dim$_64(file, &($group_dset_dim$));

if ($group_dset_dim$ == 0L) return TREXIO_INVALID_NUM;

uint32_t rank = $group_dset_rank$;
uint64_t dims[$group_dset_rank$] = {$group_dset_dim_list$};

/* The block below is specific to safe API as it checks the boundaries */
uint64_t dim_size = 1;
for (uint32_t i=0; i<rank; ++i){
    dim_size *= dims[i];
}

  if (dim_out > (int64_t) dim_size) return TREXIO_UNSAFE_ARRAY_DIM;

  return trexio_read_$group_dset$_64(file, dset_out);
}
trexio_exit_code
trexio_write_safe_$group_dset$_64 (trexio_t* const file, const $group_dset_dtype_double$* dset_in, const int64_t dim_in)
{

  if (file  == NULL) return TREXIO_INVALID_ARG_1;
  if (dset_in == NULL) return TREXIO_INVALID_ARG_2;
  if (trexio_has_$group_dset$(file) == TREXIO_SUCCESS) return TREXIO_DSET_ALREADY_EXISTS;

trexio_exit_code rc;
int64_t $group_dset_dim$ = 0;

/* Error handling for this call is added by the generator */
rc = trexio_read_$group_dset_dim$_64(file, &($group_dset_dim$));

if ($group_dset_dim$ == 0L) return TREXIO_INVALID_NUM;

uint32_t rank = $group_dset_rank$;
uint64_t dims[$group_dset_rank$] = {$group_dset_dim_list$};

/* The block below is specific to safe API as it checks the boundaries */
uint64_t dim_size = 1;
for (uint32_t i=0; i<rank; ++i){
    dim_size *= dims[i];
}

  if (dim_in > (int64_t) dim_size) return TREXIO_UNSAFE_ARRAY_DIM;

  return trexio_write_$group_dset$_64(file, dset_in);
}
3.3.2.5 Source code for default functions
trexio_exit_code
trexio_read_safe_$group_dset$ (trexio_t* const file, $group_dset_dtype_default$* const $group_dset$, const int64_t dim_out)
{
  return trexio_read_safe_$group_dset$_$default_prec$(file, $group_dset$, dim_out);
}
trexio_exit_code
trexio_write_safe_$group_dset$ (trexio_t* const file, const $group_dset_dtype_default$* $group_dset$, const int64_t dim_in)
{
  return trexio_write_safe_$group_dset$_$default_prec$(file, $group_dset$, dim_in);
}
trexio_exit_code
trexio_read_$group_dset$ (trexio_t* const file, $group_dset_dtype_default$* const $group_dset$)
{
  return trexio_read_$group_dset$_$default_prec$(file, $group_dset$);
}
trexio_exit_code
trexio_write_$group_dset$ (trexio_t* const file, const $group_dset_dtype_default$* $group_dset$)
{
  return trexio_write_$group_dset$_$default_prec$(file, $group_dset$);
}
trexio_exit_code
trexio_has_$group_dset$ (trexio_t* const file)
{

  if (file  == NULL) return TREXIO_INVALID_ARG_1;

  assert(file->back_end < TREXIO_INVALID_BACK_END);

  switch (file->back_end) {

  case TREXIO_TEXT:
    return trexio_text_has_$group_dset$(file);

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    return trexio_hdf5_has_$group_dset$(file);
#else
    return TREXIO_BACK_END_MISSING;
#endif
/*
  case TREXIO_JSON:
    return trexio_json_has_$group_dset$(file);
*/
  }
  return TREXIO_FAILURE;
}

3.3.3 Fortran templates for front end

The Fortran templates that provide an access to the C API calls from Fortran. These templates are based on the use of iso_c_binding. Pointers have to be passed by value.

interface
   integer function trexio_write_$group_dset$_64 (trex_file, dset) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     $group_dset_f_dtype_double$, intent(in) :: dset$group_dset_f_dims$
   end function trexio_write_$group_dset$_64
end interface
interface
   integer function trexio_read_$group_dset$_64 (trex_file, dset) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     $group_dset_f_dtype_double$, intent(out) :: dset$group_dset_f_dims$
   end function trexio_read_$group_dset$_64
end interface
interface
   integer function trexio_write_$group_dset$_32 (trex_file, dset) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     $group_dset_f_dtype_single$, intent(in) :: dset$group_dset_f_dims$
   end function trexio_write_$group_dset$_32
end interface
interface
   integer function trexio_read_$group_dset$_32 (trex_file, dset) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     $group_dset_f_dtype_single$, intent(out) :: dset$group_dset_f_dims$
   end function trexio_read_$group_dset$_32
end interface
interface
   integer function trexio_write_$group_dset$ (trex_file, dset) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     $group_dset_f_dtype_default$, intent(in) :: dset$group_dset_f_dims$
   end function trexio_write_$group_dset$
end interface
interface
   integer function trexio_read_$group_dset$ (trex_file, dset) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     $group_dset_f_dtype_default$, intent(out) :: dset$group_dset_f_dims$
   end function trexio_read_$group_dset$
end interface
interface
   integer function trexio_has_$group_dset$ (trex_file) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
   end function trexio_has_$group_dset$
end interface

3.3.4 Python templates for front end

def write_$group_dset$(trexio_file, dset_w) -> None:
    """Write the $group_dset$ array of numbers in the TREXIO file.

    Parameters:

    trexio_file:
        TREXIO File object.

    dset_w: list OR numpy.ndarray
        Array of $group_dset$ values to be written. If array data type does not correspond to int64 or float64, the conversion is performed.

    Raises:
        - Exception from AssertionError if TREXIO return code ~rc~ is different from TREXIO_SUCCESS and prints the error message using trexio_string_of_error.
        - Exception from some other error (e.g. RuntimeError).
    """

    try:
        import numpy as np
    except ImportError:
        raise Exception("NumPy cannot be imported.")

    doConversion = False
    doFlatten = False
    if not isinstance(dset_w, (list, tuple)):
       # if input array is not a list or tuple then it is probably a numpy array
       if isinstance(dset_w, np.ndarray) and (not dset_w.dtype==np.int64 or not dset_w.dtype==np.float64):
           doConversion = True

       if len(dset_w.shape) > 1:
           doFlatten = True
           if doConversion:
               dset_64 = np.$group_dset_py_dtype$64(dset_w).flatten()
           else:
               dset_flat = np.array(dset_w, dtype=np.$group_dset_py_dtype$64).flatten()
       else:
           if doConversion:
               dset_64 = np.$group_dset_py_dtype$64(dset_w)

    else:
       # if input array is a multidimensional list or tuple, we have to convert it
       try:
           doFlatten = True
           ncol = len(dset_w[0])
           dset_flat = np.array(dset_w, dtype=np.$group_dset_py_dtype$64).flatten()
       except TypeError:
           doFlatten = False
           pass


    if doConversion:
        rc = pytr.trexio_write_safe_$group_dset$_64(trexio_file.pytrexio_s, dset_64)
    elif doFlatten:
        rc = pytr.trexio_write_safe_$group_dset$_64(trexio_file.pytrexio_s, dset_flat)
    else:
        rc = pytr.trexio_write_safe_$group_dset$_64(trexio_file.pytrexio_s, dset_w)

    if rc != TREXIO_SUCCESS:
        raise Error(rc)
def read_$group_dset$(trexio_file, dim = None, doReshape = None, dtype = None):
    """Read the $group_dset$ array of numbers from the TREXIO file.

    Parameters:

    trexio_file:
        TREXIO File object.

    dim (Optional): int
        Size of the block to be read from the file (i.e. how many items of $group_dset$ will be returned)
        If None, the function will read all necessary array dimensions from the file.

    dtype (Optional): type
        NumPy data type of the output (e.g. np.int32|int16 or np.float32|float16). If specified, the output array will be converted from the default double precision.

    doReshape (Optional): bool
        Flag to determine whether the output NumPy array has be reshaped or not. Be default, reshaping is performed
        based on the dimensions from the ~trex.json~ file. Otherwise, ~shape~ array (list or tuple) is used if provided by the user.

    Returns:
        ~dset_64~ if dtype is None or ~dset_converted~ otherwise: numpy.ndarray
        1D NumPy array with ~dim~ elements corresponding to $group_dset$ values read from the TREXIO file.

    Raises:
        - Exception from AssertionError if TREXIO return code ~rc~ is different from TREXIO_SUCCESS and prints the error message using trexio_string_of_error.
        - Exception from some other error (e.g. RuntimeError).
"""

    try:
        import numpy as np
    except ImportError:
        raise Exception("NumPy cannot be imported.")

    if doReshape is None:
        doReshape = True

    # if dim is not specified, read dimensions from the TREXIO file
    dims_list = None
    if dim is None or doReshape:
        $group_dset_dim$ = read_$group_dset_dim$(trexio_file)

        dims_list = [$group_dset_dim_list$]
        dim = 1
        for i in range($group_dset_rank$):
            dim *= dims_list[i]


    shape = tuple(dims_list)
    if shape is None and doReshape:
        raise ValueError("Reshaping failure: shape is None.")

    try:
        rc, dset_64 = pytr.trexio_read_safe_$group_dset$_64(trexio_file.pytrexio_s, dim)

        if rc != TREXIO_SUCCESS:
            raise Error(rc)
    except:
        raise


    isConverted = False
    dset_converted = None
    if dtype is not None:

        try:
            assert isinstance(dtype, type)
        except AssertionError:
            raise TypeError("dtype argument has to be an instance of the type class (e.g. np.float32).")


        if not dtype==np.int64 or not dtype==np.float64:
            try:
                dset_converted = np.array(dset_64, dtype=dtype)
            except:
                raise

            isConverted = True

    # additional assert can be added here to check that read_safe functions returns numpy array of proper dimension

    if doReshape:
        try:
            # in-place reshaping did not work so I have to make a copy
            if isConverted:
                dset_reshaped = np.reshape(dset_converted, shape, order='C')
            else:
                dset_reshaped = np.reshape(dset_64, shape, order='C')
        except:
            raise

    if isConverted:
        return dset_converted
    elif doReshape:
        return dset_reshaped
    else:
        return dset_64
def has_$group_dset$(trexio_file) -> bool:
    """Check that $group_dset$ variable exists in the TREXIO file.

    Parameter is a ~TREXIO File~ object that has been created by a call to ~open~ function.

    Returns:
          True if the variable exists, False otherwise

    Raises:
        - Exception from trexio.Error class if TREXIO return code ~rc~ is TREXIO_FAILURE and prints the error message using string_of_error.
        - Exception from some other error (e.g. RuntimeError).
    """

    try:
        rc = pytr.trexio_has_$group_dset$(trexio_file.pytrexio_s)
        if rc == TREXIO_FAILURE:
            raise Error(rc)
    except:
        raise

    if rc == TREXIO_SUCCESS:
        return True
    else:
        return False

3.4 Templates for front end has/read/write a dataset of sparse data

3.4.1 Introduction

Sparse data structures are used typically for large tensors such as two-electron integrals. For example, in the trex.json file sparse arrays appear as for the eri :

"ao_2e_int"  : {
  "eri"      : [ "float sparse", [ "ao.num", "ao.num", "ao.num", "ao.num" ] ]
}

The electron repulsion integral (eri) \(\langle ij | kl \rangle\) is represented as a quartet of integers \((i,j,k,l)\) and a floating point value.

To store \(N\) integrals in the file, we store

  • An array of quartets of integers
  • An array of values (floats)

Both arrays have the same size, \(N\), the number of non-zero integrals. Knowing the maximum dimensions allows to check that the integers are in a valid range, and also lets the library choose the smallest integer representation to compress the storage.

Fortran uses 1-based array indexing, while C uses 0-based indexing. Internally, we use a 0-based representation but the Fortran binding does the appropriate conversion when reading or writing.

As the number of integrals to store can be prohibitively large, we provide the possibility to read/write the integrals in chunks. So the functions take two extra parameters:

  • offset_file : how many integrals in the file should be skipped when reading/writing. An offset of zero implies to read the first integral.
  • buffer_size : the number of integrals to read/write. If EOF is encountered upon reading, the buffer_size is overwritten with the number of integrals that have been read before EOF and the trexio_read_ function return TREXIO_END exit code instead of TREXIO_SUCCESS.

The storage of int indices is internally compressed based on the maximum possible value of an index, which is derived from the corresponding dimension of the sparse array (e.g. ao_num is the upper bound of indices in the aforementioned ao_2e_int_eri dataset). The upper bounds for different int types (e.g. uint16_t) can be found in the in the stdint.h C library. Currently implemented list of compressions based on the upper bound of indices can be found below:

Max value of indices Internal representation (in the TREXIO file)
UINT8_MAX (e.g. \(< 255\)) 8-bit unsigned int
UINT16_MAX (e.g. \(< 65535\)) 16-bit unsigned int
Otherwise (e.g. \(\ge 65535\)) 32-bit signed int

This section concerns API calls related to sparse data structures.

Function name Description Precision
trexio_has_$group_dset$ Check if a sparse dset is present in a file ---
trexio_read_$group_dset$ Read indices and values of a sparse dset Single/Double for indices/values
trexio_read_$group_dset$_size Read the number of sparse data elements stored in the file Double for size
trexio_write_$group_dset$ Write indices and values of a sparse dset Single/Double for indices/values
trexio_read_safe_$group_dset$ Safe (bounded) read of indices and values (for Python API) Single/Double for indices/values
trexio_write_safe_$group_dset$ Safe (bounded) write of indices and values (for Python API) Single/Double for indices/values

3.4.2 C templates for front end

3.4.2.1 Function declarations
3.4.2.2 Source code for default functions
trexio_exit_code trexio_read_safe_$group_dset$(trexio_t* const file,
                                               const int64_t offset_file,
                                               int64_t* const buffer_size_read,
                                               int32_t* const index_sparse_read,
                                               const int64_t size_index_read,
                                               double* const value_sparse_read,
                                               const int64_t size_value_read
                                               )
{
  return trexio_read_$group_dset$(file, offset_file, buffer_size_read, index_sparse_read, value_sparse_read);
}

trexio_exit_code
trexio_read_$group_dset$(trexio_t* const file,
                         const int64_t offset_file,
                         int64_t* const buffer_size,
                         int32_t* const index_sparse,
                         double* const value_sparse
                         )
{
  if (file  == NULL) return TREXIO_INVALID_ARG_1;
  if (offset_file < 0L) return TREXIO_INVALID_ARG_2;
  if (*buffer_size <= 0L) return TREXIO_INVALID_ARG_3;
  if (index_sparse == NULL) return TREXIO_INVALID_ARG_4;
  if (value_sparse == NULL) return TREXIO_INVALID_ARG_5;
  if (trexio_has_$group_dset$(file) != TREXIO_SUCCESS) return TREXIO_DSET_MISSING;

  const uint32_t rank = $group_dset_rank$;  // To be set by generator : number of indices

  int64_t size_max;         // Max number of integrals (already in the file)
  trexio_exit_code rc;

  /* Read the max number of integrals stored in the file */
  rc = trexio_read_$group_dset$_size(file, &size_max);
  if (rc != TREXIO_SUCCESS) return rc;

  int64_t num;
  rc = trexio_read_$group_dset_sparse_dim$_64(file, &num);
  if (rc != TREXIO_SUCCESS) return rc;

  // introduce a new variable which will be modified with the number of integrals being read if EOF is encountered
  int64_t eof_read_size = 0UL;

  switch (file->back_end) {

  case TREXIO_TEXT:
    rc = trexio_text_read_$group_dset$(file, offset_file, *buffer_size, num, &eof_read_size, index_sparse, value_sparse);
    break;

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    rc = trexio_hdf5_read_$group_dset$(file, offset_file, *buffer_size, num, &eof_read_size, index_sparse, value_sparse);
    break;
#else
    rc = TREXIO_BACK_END_MISSING;
    break;
#endif
/*
  case TREXIO_JSON:
    return trexio_json_read_$group_dset$(...);
    break;
*/
  default:
    rc = TREXIO_FAILURE;  /* Impossible case */
    break;
  }

  if (rc != TREXIO_SUCCESS && rc != TREXIO_END) return rc;

  if (rc == TREXIO_END) *buffer_size = eof_read_size;

  // shift indices to be one-based if Fortran API is used
  if (file->one_based) {
    // if EOF is reached - shift only indices that have been read, not an entire buffer
    uint64_t index_size = rank*(*buffer_size) ;
    for (uint64_t i=0; i<index_size; ++i){
      index_sparse[i] += 1;
    }
  }

  return rc;
}
trexio_exit_code
trexio_read_$group_dset$_size(trexio_t* const file, int64_t* const size_max)
{
  if (file  == NULL) return TREXIO_INVALID_ARG_1;
  if (trexio_has_$group_dset$(file) != TREXIO_SUCCESS) return TREXIO_DSET_MISSING;

  switch (file->back_end) {

  case TREXIO_TEXT:
    return trexio_text_read_$group_dset$_size(file, size_max);
    break;

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    return trexio_hdf5_read_$group_dset$_size(file, size_max);
    break;
#else
    return TREXIO_BACK_END_MISSING;
#endif
/*
  case TREXIO_JSON:
    return trexio_json_read_$group_dset$_size(...);
    break;
*/
  default:
    return TREXIO_FAILURE;  /* Impossible case */
  }
}
trexio_exit_code trexio_write_safe_$group_dset$(trexio_t* const file,
                                                const int64_t offset_file,
                                                const int64_t buffer_size,
                                                const int32_t* index_sparse_write,
                                                const int64_t size_index_write,
                                                const double* value_sparse_write,
                                                const int64_t size_value_write
                                                )
{
  return trexio_write_$group_dset$(file, offset_file, buffer_size, index_sparse_write, value_sparse_write);
}

trexio_exit_code
trexio_write_$group_dset$(trexio_t* const file,
                          const int64_t offset_file,
                          const int64_t buffer_size,
                          const int32_t* index_sparse,
                          const double* value_sparse
                          )
{
  if (file  == NULL) return TREXIO_INVALID_ARG_1;
  if (offset_file < 0L) return TREXIO_INVALID_ARG_2;
  if (buffer_size <= 0L) return TREXIO_INVALID_ARG_3;
  if (index_sparse == NULL) return TREXIO_INVALID_ARG_4;
  if (value_sparse == NULL) return TREXIO_INVALID_ARG_5;

  const uint32_t rank = $group_dset_rank$;  // To be set by generator : number of indices

  int64_t size_max=0L;         // Max number of integrals (already in the file)
  trexio_exit_code rc;

  /* Read the max number of integrals stored in the file */
  rc = trexio_read_$group_dset$_size(file, &size_max);
  if (rc != TREXIO_SUCCESS && rc != TREXIO_DSET_MISSING) return rc;
  if (rc == TREXIO_DSET_MISSING) size_max = 0L;

  int64_t num;
  rc = trexio_read_$group_dset_sparse_dim$_64(file, &num);
  if (rc != TREXIO_SUCCESS) return rc;

  // shift indices to be zero-based if Fortran API is used
  if (file->one_based) {

    uint64_t index_size = rank * buffer_size;

    int32_t* index_sparse_p = CALLOC(index_size, int32_t);
    if (index_sparse_p == NULL) return TREXIO_ALLOCATION_FAILED;

    for (uint64_t i=0; i<index_size; ++i){
      index_sparse_p[i] = index_sparse[i] - 1;
    }

    switch (file->back_end) {

    case TREXIO_TEXT:
      rc = trexio_text_write_$group_dset$(file, offset_file, buffer_size, num,
                                          size_max, index_sparse_p, value_sparse);
      break;

    case TREXIO_HDF5:
#ifdef HAVE_HDF5
      rc = trexio_hdf5_write_$group_dset$(file, offset_file, buffer_size, num,
                                          index_sparse_p, value_sparse);
      break;
#else
      rc = TREXIO_BACK_END_MISSING;
      break;
#endif
      /*
        case TREXIO_JSON:
        rc = trexio_json_write_$group_dset$(...);
        break;
      */
    default:
      rc = TREXIO_FAILURE;  /* Impossible case */
      break;
    }

    FREE(index_sparse_p);

  }
  else {

    switch (file->back_end) {

    case TREXIO_TEXT:
      rc = trexio_text_write_$group_dset$(file, offset_file, buffer_size, num,
                                          size_max, index_sparse, value_sparse);
      break;

    case TREXIO_HDF5:
#ifdef HAVE_HDF5
      rc = trexio_hdf5_write_$group_dset$(file, offset_file, buffer_size, num,
                                          index_sparse, value_sparse);
      break;
#else
      rc = TREXIO_BACK_END_MISSING;
      break;
#endif
      /*
        case TREXIO_JSON:
        rc = trexio_json_write_$group_dset$(...);
        break;
      */
    default:
      rc = TREXIO_FAILURE;  /* Impossible case */
      break;
    }

  }

  return rc;
}
trexio_exit_code
trexio_has_$group_dset$ (trexio_t* const file)
{

  if (file  == NULL) return TREXIO_INVALID_ARG_1;

  assert(file->back_end < TREXIO_INVALID_BACK_END);

  switch (file->back_end) {

  case TREXIO_TEXT:
    return trexio_text_has_$group_dset$(file);

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    return trexio_hdf5_has_$group_dset$(file);
#else
    return TREXIO_BACK_END_MISSING;
#endif
/*
  case TREXIO_JSON:
    return trexio_json_has_$group_dset$(file);
    break;
*/
  }

  return TREXIO_FAILURE;
}

3.4.3 Fortran templates for front end

The Fortran templates that provide an access to the C API calls from Fortran. These templates are based on the use of iso_c_binding. Pointers have to be passed by value.

interface
   integer function trexio_write_$group_dset$ (trex_file, &
                                               offset_file, buffer_size, &
                                               index_sparse, value_sparse) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     integer(8), intent(in), value :: offset_file
     integer(8), intent(in), value :: buffer_size
     integer(4), intent(in)        :: index_sparse(*)
     double precision, intent(in)  :: value_sparse(*)
   end function trexio_write_$group_dset$
end interface

interface
   integer function trexio_write_safe_$group_dset$ (trex_file, &
                                                    offset_file, buffer_size, &
                                                    index_sparse, index_size, &
                                                    value_sparse, value_size) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     integer(8), intent(in), value :: offset_file
     integer(8), intent(in), value :: buffer_size
     integer(4), intent(in)        :: index_sparse(*)
     integer(8), intent(in), value :: index_size
     double precision, intent(in)  :: value_sparse(*)
     integer(8), intent(in), value :: value_size
   end function trexio_write_safe_$group_dset$
end interface
interface
   integer function trexio_read_$group_dset$ (trex_file, &
                                              offset_file, buffer_size, &
                                              index_sparse, value_sparse) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     integer(8), intent(in), value :: offset_file
     integer(8), intent(inout)     :: buffer_size
     integer(4), intent(out)       :: index_sparse(*)
     double precision, intent(out) :: value_sparse(*)
   end function trexio_read_$group_dset$
end interface

interface
   integer function trexio_read_safe_$group_dset$ (trex_file, &
                                                   offset_file, buffer_size, &
                                                   index_sparse, index_size, &
                                                   value_sparse, value_size) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     integer(8), intent(in), value :: offset_file
     integer(8), intent(inout)     :: buffer_size
     integer(4), intent(out)       :: index_sparse(*)
     integer(8), intent(in), value :: index_size
     double precision, intent(out) :: value_sparse(*)
     integer(8), intent(in), value :: value_size
   end function trexio_read_safe_$group_dset$
end interface
interface
   integer function trexio_read_$group_dset$_size (trex_file, &
                                                   size_max) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     integer(8), intent(out) :: size_max
   end function trexio_read_$group_dset$_size
end interface
interface
   integer function trexio_has_$group_dset$ (trex_file) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
   end function trexio_has_$group_dset$
end interface

3.4.4 Python templates for front end

def write_$group_dset$(trexio_file: File, offset_file: int, buffer_size: int, indices: list, values: list) -> None:
    """Write the $group_dset$ indices and values in the TREXIO file.

    Parameters:

    trexio_file:
        TREXIO File object.

    offset_file: int
        The number of integrals to be skipped in the file when writing.

    buffer_size: int
        The number of integrals to write in the file from the provided sparse arrays.

    values: list OR numpy.ndarray
        Array of $group_dset$ indices to be written. If array data type does not correspond to int32, the conversion is performed.

    values: list OR numpy.ndarray
        Array of $group_dset$ values to be written. If array data type does not correspond to float64, the conversion is performed.

    Raises:
        - Exception from AssertionError if TREXIO return code ~rc~ is different from TREXIO_SUCCESS and prints the error message using trexio_string_of_error.
        - Exception from some other error (e.g. RuntimeError).
    """

    try:
        import numpy as np
    except ImportError:
        raise Exception("NumPy cannot be imported.")

    if not isinstance(offset_file, int):
        raise TypeError("offset_file argument has to be an integer.")
    if not isinstance(buffer_size, int):
        raise TypeError("buffer_size argument has to be an integer.")
    if not isinstance(indices, (list, tuple, np.ndarray)):
        raise TypeError("indices argument has to be an array (list, tuple or NumPy ndarray).")
    if not isinstance(values, (list, tuple, np.ndarray)):
        raise TypeError("values argument has to be an array (list, tuple or NumPy ndarray).")

    convertIndices = False
    convertValues = False
    flattenIndices = False
    if isinstance(indices, np.ndarray):
       # convert to int32 if input indices are in a different precision
       if not indices.dtype==np.int32:
           convertIndices = True

       if len(indices.shape) > 1:
           flattenIndices = True
           if convertIndices:
               indices_32 = np.int32(indices).flatten()
           else:
               indices_32 = np.array(indices, dtype=np.int32).flatten()
       else:
           if convertIndices:
               indices_32 = np.int32(indices)
    else:
       # if input array is a multidimensional list or tuple, we have to convert it
       try:
           doFlatten = True
           # if list of indices is flat - the attempt to compute len(indices[0]) will raise a TypeError
           ncol = len(indices[0])
           indices_32 = np.array(indices, dtype=np.int32).flatten()
       except TypeError:
           doFlatten = False
           pass

    if isinstance(values, np.ndarray):
       # convert to float64 if input values are in a different precision
       if not values.dtype==np.float64:
           convertValues = True
       if convertValues:
           values_64 = np.float64(values)

    if (convertIndices or flattenIndices) and convertValues:
        rc = pytr.trexio_write_safe_$group_dset$(trexio_file.pytrexio_s, offset_file, buffer_size, indices_32, values_64)
    elif (convertIndices or flattenIndices) and not convertValues:
        rc = pytr.trexio_write_safe_$group_dset$(trexio_file.pytrexio_s, offset_file, buffer_size, indices_32, values)
    elif not (convertIndices or flattenIndices) and convertValues:
        rc = pytr.trexio_write_safe_$group_dset$(trexio_file.pytrexio_s, offset_file, buffer_size, indices, values_64)
    else:
        rc = pytr.trexio_write_safe_$group_dset$(trexio_file.pytrexio_s, offset_file, buffer_size, indices, values)

    if rc != TREXIO_SUCCESS:
        raise Error(rc)
def read_$group_dset$(trexio_file: File, offset_file: int, buffer_size: int) -> tuple:
    """Read the $group_dset$ indices and values from the TREXIO file.

    Parameters:

    trexio_file:
        TREXIO File object.

    offset_file: int
        The number of integrals to be skipped in the file when reading.

    buffer_size: int
        The number of integrals to read from the file.

    Returns:
        (indices, values, n_int_read, eof_flag) tuple where
          - indices and values are NumPy arrays [numpy.ndarray] with the default int32 and float64 precision, respectively;
          - n_int_read [int] is the number of integrals read from the trexio_file
            (either strictly equal to buffer_size or less than buffer_size if EOF has been reached);
          - eof_flag [bool] is True when EOF has been reached (i.e. when call to low-level pytrexio API returns TREXIO_END)
                               False otherwise.

    Raises:
        - Exception from AssertionError if TREXIO return code ~rc~ is different from TREXIO_SUCCESS
            and prints the error message using trexio_string_of_error.
        - Exception from some other error (e.g. RuntimeError).
    """

    try:
        import numpy as np
    except ImportError:
        raise Exception("NumPy cannot be imported.")

    if not isinstance(offset_file, int):
        raise TypeError("offset_file argument has to be an integer.")
    if not isinstance(buffer_size, int):
        raise TypeError("buffer_size argument has to be an integer.")


    # read the number of integrals already in the file
    integral_num = read_$group_dset$_size(trexio_file)

    # additional modification needed to avoid allocating more memory than needed if EOF will be reached during read
    overflow = offset_file + buffer_size - integral_num
    eof_flag = False
    if overflow > 0:
        verified_size = buffer_size - overflow
        eof_flag = True
    else:
        verified_size = buffer_size

    # main call to the low-level (SWIG-wrapped) trexio_read function, which also requires the sizes of the output to be provided
    # as the last 2 arguments (for numpy arrays of indices and values, respectively)
    # read_buf_size contains the number of elements being read from the file, useful when EOF has been reached
    rc, n_int_read, indices, values = pytr.trexio_read_safe_$group_dset$(trexio_file.pytrexio_s,
                                                                         offset_file,
                                                                         verified_size,
                                                                         verified_size * $group_dset_rank$,
                                                                         verified_size)
    if rc != TREXIO_SUCCESS:
        raise Error(rc)
    if n_int_read == 0:
        raise ValueError("No integrals have been read from the file.")
    if indices is None or values is None:
        raise ValueError("Returned NULL array from the low-level pytrexio API.")

    # conversion to custom types can be performed on the user side, here we only reshape the returned flat array of indices according to group_dset_rank
    shape = tuple([verified_size, $group_dset_rank$])
    indices_reshaped = np.reshape(indices, shape, order='C')

    return (indices_reshaped, values, n_int_read, eof_flag)


def read_$group_dset$_size(trexio_file) -> int:
    """Read the number of $group_dset$ integrals stored in the TREXIO file.

    Parameter is a ~TREXIO File~ object that has been created by a call to ~open~ function.

    Returns:
        ~num_integral~: int
        Integer value of corresponding to the size of the $group_dset$ sparse array from ~trexio_file~.

    Raises:
        - Exception from AssertionError if TREXIO return code ~rc~ is different from TREXIO_SUCCESS and prints the error message using trexio_string_of_error.
        - Exception from some other error (e.g. RuntimeError).
    """

    try:
        rc, num_integral = pytr.trexio_read_$group_dset$_size(trexio_file.pytrexio_s)
        if rc != TREXIO_SUCCESS:
            raise Error(rc)
    except:
        raise

    return num_integral
def has_$group_dset$(trexio_file) -> bool:
    """Check that $group_dset$ variable exists in the TREXIO file.

    Parameter is a ~TREXIO File~ object that has been created by a call to ~open~ function.

    Returns:
          True if the variable exists, False otherwise

    Raises:
        - Exception from trexio.Error class if TREXIO return code ~rc~ is TREXIO_FAILURE and prints the error message using string_of_error.
        - Exception from some other error (e.g. RuntimeError).
    """

    try:
        rc = pytr.trexio_has_$group_dset$(trexio_file.pytrexio_s)
        if rc == TREXIO_FAILURE:
            raise Error(rc)
    except:
        raise

    if rc == TREXIO_SUCCESS:
        return True
    else:
        return False

3.5 Templates for front end has/read/write a dataset of strings

3.5.1 Introduction

This section concerns API calls related to datasets of strings.

Function name Description
trexio_has_$group_dset$ Check if a dataset exists in a file
trexio_read_$group_dset$ Read a dataset
trexio_write_$group_dset$ Write a dataset

3.5.2 C templates for front end

First parameter is the TREXIO file handle. Second parameter is the variable to be written/read to/from the TREXIO file (except for trexio_has_ functions).

3.5.2.1 Function declarations
3.5.2.2 Source code for default functions
trexio_exit_code
trexio_read_$group_dset$_low (trexio_t* const file, char* dset_out, const int32_t max_str_len)
{

  if (file  == NULL) return TREXIO_INVALID_ARG_1;
  if (dset_out == NULL) return TREXIO_INVALID_ARG_2;
  if (max_str_len <= 0) return TREXIO_INVALID_ARG_3;
  if (trexio_has_$group_dset$(file) != TREXIO_SUCCESS) return TREXIO_DSET_MISSING;

  trexio_exit_code rc;
  int64_t $group_dset_dim$ = 0;

  /* Error handling for this call is added by the generator */
  rc = trexio_read_$group_dset_dim$_64(file, &($group_dset_dim$));

  if ($group_dset_dim$ == 0L) return TREXIO_INVALID_NUM;

  uint32_t rank = $group_dset_rank$;
  uint64_t dims[$group_dset_rank$] = {$group_dset_dim_list$};

  assert(file->back_end < TREXIO_INVALID_BACK_END);
  switch (file->back_end) {

  case TREXIO_TEXT:
    return trexio_text_read_$group_dset$(file, dset_out, rank, dims, (uint32_t) max_str_len);

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    return trexio_hdf5_read_$group_dset$(file, dset_out, rank, dims, (uint32_t) max_str_len);
#else
    return TREXIO_BACK_END_MISSING;
#endif
/*
  case TREXIO_JSON:
    rc = trexio_json_read_$group_dset$(file, dset_out, rank, dims);
*/
  }

  return TREXIO_FAILURE;
}

trexio_exit_code
trexio_read_$group_dset$ (trexio_t* const file, char** dset_out, const int32_t max_str_len)
{

  if (file  == NULL) return TREXIO_INVALID_ARG_1;
  if (dset_out == NULL) return TREXIO_INVALID_ARG_2;
  if (max_str_len <= 0) return TREXIO_INVALID_ARG_3;
  if (trexio_has_$group_dset$(file) != TREXIO_SUCCESS) return TREXIO_DSET_MISSING;

  assert(file->back_end < TREXIO_INVALID_BACK_END);

  trexio_exit_code rc;
  int64_t dset_dim = 0;

  /* Error handling for this call is added by the generator */
  rc = trexio_read_$group_dset_dim$_64(file, &(dset_dim));

  if (dset_dim == 0L) return TREXIO_INVALID_NUM;

  char* str_compiled = CALLOC(dset_dim*(max_str_len+1) + 1, char);
  if (str_compiled == NULL) return TREXIO_ALLOCATION_FAILED;

  rc = trexio_read_$group_dset$_low(file, str_compiled, max_str_len);
  if (rc != TREXIO_SUCCESS) {
    FREE(str_compiled);
    return rc;
  }

  for (uint64_t i=0; i < (uint64_t) dset_dim; i++) {

    char * pch;
    pch = i == 0 ? strtok(str_compiled, TREXIO_DELIM) : strtok(NULL, TREXIO_DELIM) ;
    if (pch == NULL) {
      FREE(str_compiled);
      return TREXIO_FAILURE;
    }

    strcpy(dset_out[i], "");
    strcat(dset_out[i], pch);

  }

  FREE(str_compiled);
  return TREXIO_SUCCESS;

}
trexio_exit_code
trexio_write_$group_dset$_low (trexio_t* const file, char* dset_in, const int32_t max_str_len)
{

  if (file  == NULL) return TREXIO_INVALID_ARG_1;
  if (dset_in == NULL) return TREXIO_INVALID_ARG_2;
  if (max_str_len <= 0) return TREXIO_INVALID_ARG_3;
  if (trexio_has_$group_dset$(file) == TREXIO_SUCCESS) return TREXIO_DSET_ALREADY_EXISTS;

  trexio_exit_code rc;
  int64_t $group_dset_dim$ = 0;

  /* Error handling for this call is added by the generator */
  rc = trexio_read_$group_dset_dim$_64(file, &($group_dset_dim$));

  if ($group_dset_dim$ == 0L) return TREXIO_INVALID_NUM;

  uint32_t rank = $group_dset_rank$;
  uint64_t dims[$group_dset_rank$] = {$group_dset_dim_list$};

  assert(file->back_end < TREXIO_INVALID_BACK_END);

  char*  tmp_str  = CALLOC(dims[0]*(max_str_len+1), char);
  if (tmp_str == NULL) return TREXIO_ALLOCATION_FAILED;
  char** dset_str = CALLOC(dims[0], char*);
  if (dset_str == NULL) {
    FREE(tmp_str);
    return TREXIO_ALLOCATION_FAILED;
  }

  /* parse the string using strtok */
  for(uint64_t i=0; i<dims[0]; i++) {

    char* pch;
    pch = i == 0 ? strtok(dset_in, TREXIO_DELIM) : strtok(NULL, TREXIO_DELIM) ;

    if (pch == NULL) {
      FREE(dset_str[0]);
      FREE(dset_str);
      return TREXIO_FAILURE;
    }

    size_t pch_len = strlen(pch) + 1;

    if (pch_len > (size_t) max_str_len) {
      FREE(dset_str[0]);
      FREE(dset_str);
      return TREXIO_INVALID_STR_LEN;
    }

    dset_str[i] = tmp_str;
    strncpy(tmp_str, pch, pch_len);
    tmp_str += pch_len + 1;
  }

  rc = TREXIO_FAILURE;
  switch (file->back_end) {

  case TREXIO_TEXT:
    rc = trexio_text_write_$group_dset$(file, (const char**) dset_str, rank, dims);
    break;

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    rc = trexio_hdf5_write_$group_dset$(file, (const char**) dset_str, rank, dims);
    break;
#else
    rc =TREXIO_BACK_END_MISSING;
    break;
#endif
/*
  case TREXIO_JSON:
    rc = trexio_json_write_$group_dset$(file, dset, rank, dims);
    break;
*/
  }

  FREE(dset_str[0]);
  FREE(dset_str);

  return rc;

}

trexio_exit_code
trexio_write_$group_dset$ (trexio_t* const file, const char** dset_in, const int32_t max_str_len)
{

  if (file  == NULL) return TREXIO_INVALID_ARG_1;
  if (dset_in == NULL) return TREXIO_INVALID_ARG_2;
  if (max_str_len <= 0) return TREXIO_INVALID_ARG_3;
  if (trexio_has_$group_dset$(file) == TREXIO_SUCCESS) return TREXIO_DSET_ALREADY_EXISTS;

  assert(file->back_end < TREXIO_INVALID_BACK_END);

  trexio_exit_code rc;
  int64_t dset_dim = 0;

  /* Error handling for this call is added by the generator */
  rc = trexio_read_$group_dset_dim$_64(file, &(dset_dim));

  if (dset_dim == 0L) return TREXIO_INVALID_NUM;

  char* str_compiled = CALLOC(dset_dim*max_str_len + 1, char);
  if (str_compiled == NULL) return TREXIO_ALLOCATION_FAILED;

  strcpy(str_compiled, "");
  for (uint64_t i=0; i < (uint64_t) dset_dim; i++) {
    strcat(str_compiled, dset_in[i]);
    strcat(str_compiled, TREXIO_DELIM);
  }

  rc = trexio_write_$group_dset$_low(file, str_compiled, max_str_len);

  FREE(str_compiled);

  return rc;
}
trexio_exit_code
trexio_has_$group_dset$ (trexio_t* const file)
{

  if (file  == NULL) return TREXIO_INVALID_ARG_1;

  assert(file->back_end < TREXIO_INVALID_BACK_END);

  switch (file->back_end) {

  case TREXIO_TEXT:
    return trexio_text_has_$group_dset$(file);

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    return trexio_hdf5_has_$group_dset$(file);
#else
    return TREXIO_BACK_END_MISSING;
#endif
/*
  case TREXIO_JSON:
    return trexio_json_has_$group_dset$(file);
    break;
*/
  }

  return TREXIO_FAILURE;
}

3.5.3 Fortran templates for front end

The Fortran templates that provide an access to the C API calls from Fortran. These templates are based on the use of iso_c_binding. Pointers have to be passed by value.

interface
   integer function trexio_write_$group_dset$_low (trex_file, dset, max_str_len) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     character, intent(in) :: dset(*)
     integer(4), intent(in), value :: max_str_len
   end function trexio_write_$group_dset$_low
end interface
interface
   integer function trexio_read_$group_dset$_low (trex_file, dset, max_str_len) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     character, intent(out) :: dset(*)
     integer(4), intent(in), value :: max_str_len
   end function trexio_read_$group_dset$_low
end interface
interface
   integer function trexio_has_$group_dset$ (trex_file) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
   end function trexio_has_$group_dset$
end interface
integer function trexio_read_$group_dset$ (trex_file, dset, max_str_len)
  implicit none
  integer(8), intent(in), value :: trex_file
  integer(4), intent(in), value :: max_str_len
  character(len=*), intent(inout) :: dset(*)

  character, allocatable :: str_compiled(:)
  integer(8) :: $group_dset_dim$
  integer :: rc

  rc = trexio_read_$group_dset_dim$_64(trex_file, $group_dset_dim$)
  if (rc /= TREXIO_SUCCESS) trexio_read_$group_dset$ = rc

  allocate(str_compiled($group_dset_dim$*(max_str_len+1)+1))

  rc = trexio_read_$group_dset$_low(trex_file, str_compiled, max_str_len)
  if (rc /= TREXIO_SUCCESS) then
    deallocate(str_compiled)
    trexio_read_$group_dset$ = rc
  else
    call trexio_str2strarray(str_compiled, $group_dset_dim$, max_str_len, dset)
    deallocate(str_compiled)
    trexio_read_$group_dset$ = TREXIO_SUCCESS
  endif

end function trexio_read_$group_dset$
integer function trexio_write_$group_dset$ (trex_file, dset, max_str_len)
  implicit none
  integer(8), intent(in), value :: trex_file
  integer(4), intent(in), value :: max_str_len
  character(len=*), intent(in) :: dset(*)

  character(len=:), allocatable :: str_compiled
  integer(8) :: $group_dset_dim$
  integer :: rc

  rc = trexio_read_$group_dset_dim$_64(trex_file, $group_dset_dim$)
  if (rc /= TREXIO_SUCCESS) then
    trexio_write_$group_dset$ = rc
  else
    call trexio_strarray2str(dset, $group_dset_dim$, max_str_len, str_compiled)
    trexio_write_$group_dset$ = trexio_write_$group_dset$_low(trex_file, str_compiled, max_str_len)
  endif

end function trexio_write_$group_dset$

3.5.4 Python templates for front end

def write_$group_dset$(trexio_file, dset_w: list) -> None:
    """Write the $group_dset$ array of strings in the TREXIO file.

    Parameters:

    trexio_file:
        TREXIO File object.

    dset_w: list
        Array of $group_dset$ strings to be written.

    Raises:
        - Exception from AssertionError if TREXIO return code ~rc~ is different from TREXIO_SUCCESS and prints the error message using trexio_string_of_error.
        - Exception from some other error (e.g. RuntimeError).
    """

    max_str_length = len(max(dset_w, key=len)) + 1

    try:
        rc = pytr.trexio_write_$group_dset$(trexio_file.pytrexio_s, dset_w, max_str_length)

        if rc != TREXIO_SUCCESS:
            raise Error(rc)
    except:
        raise

def read_$group_dset$(trexio_file, dim = None) -> list:
    """Read the $group_dset$ array of strings from the TREXIO file.

    Parameters:

    trexio_file:
        TREXIO File object.

    dim (Optional): int
        Size of the block to be read from the file (i.e. how many items of $group_dset$ will be returned)
        If None, the function will read all necessary array dimensions from the file.

    Returns:
        ~dset_r~: list
        1D list with ~dim~ elements corresponding to $group_dset$ strings read from the TREXIO file.

    Raises:
        - Exception from AssertionError if TREXIO return code ~rc~ is different from TREXIO_SUCCESS and prints the error message using trexio_string_of_error.
        - Exception from some other error (e.g. RuntimeError).
    """

    # if dim is not specified, read dimensions from the TREXIO file
    if dim is None:
        $group_dset_dim$ = read_$group_dset_dim$(trexio_file)

        dims_list = [$group_dset_dim_list$]
        dim = 1
        for i in range($group_dset_rank$):
            dim *= dims_list[i]


    try:
        rc, dset_1d_r = pytr.trexio_read_$group_dset$_low(trexio_file.pytrexio_s, PYTREXIO_MAX_STR_LENGTH)

        if rc != TREXIO_SUCCESS:
            raise Error(rc)
    except:
        raise


    try:
        dset_full = dset_1d_r.split(pytr.TREXIO_DELIM)
        dset_2d_r = [dset_full[i] for i in range(dim) if dset_full[i]]
        if not dset_2d_r:
            raise ValueError(f"Output of {read_$group_dset$.__name__} function cannot be an empty list.")
    except:
        raise

    return dset_2d_r
def has_$group_dset$(trexio_file) -> bool:
    """Check that $group_dset$ variable exists in the TREXIO file.

    Parameter is a ~TREXIO File~ object that has been created by a call to ~open~ function.

    Returns:
          True if the variable exists, False otherwise

    Raises:
        - Exception from trexio.Error class if TREXIO return code ~rc~ is TREXIO_FAILURE and prints the error message using string_of_error.
        - Exception from some other error (e.g. RuntimeError).
    """

    try:
        rc = pytr.trexio_has_$group_dset$(trexio_file.pytrexio_s)
        if rc == TREXIO_FAILURE:
            raise Error(rc)
    except:
        raise

    if rc == TREXIO_SUCCESS:
        return True
    else:
        return False

3.6 Templates for front end has/read/write a single string attribute

3.6.1 Introduction

This section concerns API calls related to string attributes.

Function name Description
trexio_has_$group_str$ Check if a string attribute exists in a file
trexio_read_$group_str$ Read a string attribute
trexio_write_$group_str$ Write a string attribute

3.6.2 C templates for front end

3.6.2.1 Function declarations
3.6.2.2 Source code for default functions
trexio_exit_code
trexio_read_$group_str$ (trexio_t* const file, char* const str_out, const int32_t max_str_len)
{

  if (file == NULL) return TREXIO_INVALID_ARG_1;
  if (str_out  == NULL) return TREXIO_INVALID_ARG_2;
  if (max_str_len <= 0) return TREXIO_INVALID_ARG_3;
  if (trexio_has_$group_str$(file) != TREXIO_SUCCESS) return TREXIO_ATTR_MISSING;

  switch (file->back_end) {

  case TREXIO_TEXT:
    return trexio_text_read_$group_str$(file, str_out, (uint32_t) max_str_len);

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    return trexio_hdf5_read_$group_str$(file, str_out, (uint32_t) max_str_len);
#else
    return TREXIO_BACK_END_MISSING;
#endif
/*
  case TREXIO_JSON:
    return trexio_json_read_$group_str$(file, str);
    break;
*/
  }

  return TREXIO_FAILURE;
}
trexio_exit_code
trexio_write_$group_str$ (trexio_t* const file, const char* str, const int32_t max_str_len)
{

  if (file == NULL) return TREXIO_INVALID_ARG_1;
  if (str  == NULL) return TREXIO_INVALID_ARG_2;
  if (max_str_len <= 0) return TREXIO_INVALID_ARG_3;
  if (trexio_has_$group_str$(file) == TREXIO_SUCCESS) return TREXIO_ATTR_ALREADY_EXISTS;

  size_t len_write = strlen(str);
  if ((size_t) max_str_len < len_write) return TREXIO_INVALID_STR_LEN;

  switch (file->back_end) {

  case TREXIO_TEXT:
    return trexio_text_write_$group_str$(file, str);

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    return trexio_hdf5_write_$group_str$(file, str);
#else
    return TREXIO_BACK_END_MISSING;
#endif
/*
  case TREXIO_JSON:
    return trexio_json_write_$group_str$(file, str);
*/
  }

  return TREXIO_FAILURE;
}
trexio_exit_code
trexio_has_$group_str$ (trexio_t* const file)
{

  if (file == NULL) return TREXIO_INVALID_ARG_1;

  assert(file->back_end < TREXIO_INVALID_BACK_END);

  switch (file->back_end) {

  case TREXIO_TEXT:
    return trexio_text_has_$group_str$(file);

  case TREXIO_HDF5:
#ifdef HAVE_HDF5
    return trexio_hdf5_has_$group_str$(file);
#else
    return TREXIO_BACK_END_MISSING;
#endif
/*
  case TREXIO_JSON:
    return trexio_json_has_$group_str$(file);
*/
  }

  return TREXIO_FAILURE;
}

3.6.3 Fortran templates for front end

The Fortran templates that provide an access to the C API calls from Fortran. These templates are based on the use of iso_c_binding. Pointers have to be passed by value.

interface
   integer function trexio_write_$group_str$_c (trex_file, str, max_str_len) &
           bind(C, name="trexio_write_$group_str$")
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     character, intent(in) :: str(*)
     integer(4), intent(in), value :: max_str_len
   end function trexio_write_$group_str$_c
end interface
interface
   integer function trexio_read_$group_str$_c (trex_file, str, max_str_len) &
           bind(C, name="trexio_read_$group_str$")
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
     character, intent(out) :: str(*)
     integer(4), intent(in), value :: max_str_len
   end function trexio_read_$group_str$_c
end interface
interface
   integer function trexio_has_$group_str$ (trex_file) bind(C)
     use, intrinsic :: iso_c_binding
     integer(8), intent(in), value :: trex_file
   end function trexio_has_$group_str$
end interface
integer function trexio_read_$group_str$ (trex_file, str, max_str_len)
  implicit none
  integer(8), intent(in), value :: trex_file
  integer(4), intent(in), value :: max_str_len
  character, intent(out) :: str(*)

  trexio_read_$group_str$ = trexio_read_$group_str$_c(trex_file, str, max_str_len)

end function trexio_read_$group_str$
integer function trexio_write_$group_str$ (trex_file, str, max_str_len)
  use, intrinsic :: iso_c_binding, only : c_null_char
  implicit none
  integer(8), intent(in), value :: trex_file
  integer(4), intent(in), value :: max_str_len
  character(len=*), intent(in) :: str

  character(len=len_trim(str)+1) :: str_c

  str_c = trim(str) // c_null_char

  trexio_write_$group_str$ = trexio_write_$group_str$_c(trex_file, str_c, max_str_len)

end function trexio_write_$group_str$

3.6.4 Python templates for front end

def write_$group_str$(trexio_file, str_w: str) -> None:
    """Write the $group_str$ variable in the TREXIO file.

    Parameters:

    trexio_file:
        TREXIO File object.

    str_w: str
        String corresponding to the $group_str$ variable to be written.

    Raises:
        - Exception from AssertionError if TREXIO return code ~rc~ is different from TREXIO_SUCCESS and prints the error message using trexio_string_of_error.
        - Exception from some other error (e.g. RuntimeError).
    """

    max_str_length = len(str_w) + 1

    try:
        rc = pytr.trexio_write_$group_str$(trexio_file.pytrexio_s, str_w, max_str_length)

        if rc != TREXIO_SUCCESS:
            raise Error(rc)
    except:
        raise
def read_$group_str$(trexio_file) -> str:
    """Read the $group_str$ variable from the TREXIO file.

    Parameter is a ~TREXIO File~ object that has been created by a call to ~open~ function.

    Returns:
        ~str_r~: str
        String corresponding to the $group_str$ variable read from ~trexio_file~.

    Raises:
        - Exception from AssertionError if TREXIO return code ~rc~ is different from TREXIO_SUCCESS and prints the error message using trexio_string_of_error.
        - Exception from some other error (e.g. RuntimeError).
    """

    try:
        rc, str_r = pytr.trexio_read_$group_str$(trexio_file.pytrexio_s, PYTREXIO_MAX_STR_LENGTH)

        if rc != TREXIO_SUCCESS:
            raise Error(rc)
    except:
        raise

    return str_r
def has_$group_str$(trexio_file) -> bool:
    """Check that $group_str$ variable exists in the TREXIO file.

    Parameter is a ~TREXIO File~ object that has been created by a call to ~open~ function.

    Returns:
          True if the variable exists, False otherwise

    Raises:
        - Exception from trexio.Error class if TREXIO return code ~rc~ is TREXIO_FAILURE and prints the error message using string_of_error.
        - Exception from some other error (e.g. RuntimeError).
    """

    try:
        rc = pytr.trexio_has_$group_str$(trexio_file.pytrexio_s)
        if rc == TREXIO_FAILURE:
            raise Error(rc)
    except:
        raise

    if rc == TREXIO_SUCCESS:
        return True
    else:
        return False

4 Fortran helper/wrapper functions

The function below adapts the original C-based trexio_open for Fortran. This is needed due to the fact that strings in C are terminated by NULL character \0 unlike strings in Fortran. Note, that Fortran interface calls the main TREXIO API, which is written in C.

contains
   integer(8) function trexio_open (filename, mode, backend, rc_open)
     use, intrinsic :: iso_c_binding, only : c_null_char
     implicit none
     character(len=*), intent(in)                 :: filename
     character, intent(in), value                 :: mode
     integer(trexio_backend), intent(in), value   :: backend
     integer(trexio_exit_code), intent(out)       :: rc_open
     character(len=len_trim(filename)+1) :: filename_c
     integer(trexio_exit_code) :: rc

     filename_c = trim(filename) // c_null_char
     trexio_open = trexio_open_c(filename_c, mode, backend, rc_open)
     if (trexio_open == 0_8 .or. rc_open /= TREXIO_SUCCESS) then
       return
     endif
     rc = trexio_set_one_based(trexio_open)
     if (rc /= TREXIO_SUCCESS) then
        rc = trexio_close(trexio_open)
        trexio_open = 0_8
     endif
   end function trexio_open

The subroutine below transforms an array of Fortran strings into one big string using TREXIO_DELIM symbol as a delimeter and adds NULL character in the end in order to properly pass the desired string to C API. This is needed due to the fact that strings in C are terminated by NULL character \0.

subroutine trexio_strarray2str(str_array, max_num_str, max_len_str, str_res)
  use, intrinsic :: iso_c_binding, only : c_null_char
  implicit none

  integer(8), intent(in), value   :: max_num_str  ! number of elements in strign array
  integer, intent(in), value   :: max_len_str  ! maximum length of a string in an array
  character(len=*), intent(in)  :: str_array(*)
  character(len=:), allocatable, intent(out) :: str_res
  integer :: i

  str_res = ''
  do i = 1, max_num_str
    str_res = str_res // trim(str_array(i)) // TREXIO_DELIM
  enddo
  str_res = str_res // c_null_char

end subroutine trexio_strarray2str

The subroutine below does the reverse tranformation from one big string with delimeters into an array of Fortran strings.

subroutine trexio_str2strarray(str_flat, max_num_str, max_len_str, str_array)
  implicit none

  integer(8), intent(in), value   :: max_num_str  ! number of elements in strign array
  integer, intent(in), value   :: max_len_str  ! maximum length of a string in an array
  character, intent(in) :: str_flat(*)
  character(len=*), intent(inout)  :: str_array(*)

  character(len=max_len_str)  :: tmp_str
  integer :: i, j, k, ind, offset
  integer(8) :: len_flat

  len_flat = (max_len_str+1)*max_num_str + 1

  ind=1
  offset=1
  do i=1,max_num_str
    k = 1
    tmp_str=''
    do j=ind,len_flat
      if (str_flat(j) == TREXIO_DELIM) then
        ind=j+1
        exit
      endif
      tmp_str(k:k) = str_flat(j)
      k = k + 1
    enddo
    str_array(i)=tmp_str
    offset=ind
  enddo

end subroutine trexio_str2strarray

The subroutine is a Fortran analogue of assert in C. It check that the the return code of the TREXIO API call is equal to a given return code. It can optionally print a success message if the two code are identical, i.e. if the assert statement pass.

subroutine trexio_assert(trexio_rc, check_rc, success_message)
  implicit none

  integer, intent(in), value   :: trexio_rc
  integer, intent(in), value   :: check_rc
  character(len=*), intent(in), optional  :: success_message

  character*(128) :: str

  if (trexio_rc == check_rc) then
    if (present(success_message)) write(*,*) success_message
  else
    call trexio_string_of_error(trexio_rc, str)
    print *, trim(str)
    call exit(1)
  endif

end subroutine trexio_assert

Author: TREX-CoE

Created: 2022-01-07 Fri 14:38

Validate