TREX/trexio
1
0
Fork 0
mirror of https://github.com/TREX-CoE/trexio.git synced 2024-12-23 04:43:57 +01:00
Code Issues Projects Releases Wiki Activity

document the HDF5 issue with releasing the free space

Browse Source 
suggest using h5repack after each deletion to get the file size right
This commit is contained in:
q-posev 2022-01-24 11:52:25 +01:00
parent 7ba6924a8a
commit 179e0c8dfc
1 changed files with 7 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
src/templates_hdf5/templator_hdf5.org
View File

@ -925,7 +925,12 @@ trexio_hdf5_has_$group_str$ (trexio_t* const file)
**Note:** in early versions of the HDF5 library (v < 1.10) unlinking an object was not working as expected **Note:** in early versions of the HDF5 library (v < 1.10) unlinking an object was not working as expected
and the associated memory was not necessarily freed (see [[https://stackoverflow.com/questions/1124994/removing-data-from-a-hdf5-file][this StackOverflow discussion]] for example). and the associated memory was not necessarily freed (see [[https://stackoverflow.com/questions/1124994/removing-data-from-a-hdf5-file][this StackOverflow discussion]] for example).
Nevertheless, we highly recommend to reopen the file in SAFE mode (e.g. ~'r'~ or ~'w'~) after deleting the data. Nevertheless, some space might remain occupied even after deleting the associated object in recent version.
To take the best use of the deleted file space, we recommend to write the deleted group within the same session
(i.e. before closing the TREXIO file).
In principle, one can use HDF5-provided ~h5repack~ binary, which copies all existing objects from one file into another.
Thus, any corrupted/lost file space will remain in the first file. The use of ~h5repack~ is highly encouraged.
#+begin_src c :tangle delete_group_hdf5.h :exports none #+begin_src c :tangle delete_group_hdf5.h :exports none
trexio_exit_code trexio_hdf5_delete_$group$ (trexio_t* const file); trexio_exit_code trexio_hdf5_delete_$group$ (trexio_t* const file);
@ -941,6 +946,7 @@ trexio_hdf5_delete_$group$ (trexio_t* const file)
// delete the link to the existing group: this should free the associated space // delete the link to the existing group: this should free the associated space
H5Gclose(f->$group$_group); H5Gclose(f->$group$_group);
f->$group$_group = 0;
herr_t status = H5Ldelete(f->file_id, $GROUP$_GROUP_NAME, H5P_DEFAULT); herr_t status = H5Ldelete(f->file_id, $GROUP$_GROUP_NAME, H5P_DEFAULT);
if (status < 0) return TREXIO_FAILURE; if (status < 0) return TREXIO_FAILURE;