TREX/trexio
1
0
Fork 0
mirror of https://github.com/TREX-CoE/trexio.git synced 2024-07-22 10:47:43 +02:00
Code Issues Projects Releases Wiki Activity
5b1cf03aaf
trexio/tools/generator.py

64 lines
2.9 KiB
Python
Raw Normal View History

Fixed: rc was not checked for multiple dims
2021-03-28 15:43:49 +02:00
#!/usr/bin/env python3
Major refactoring of the generator script (#47) * minor cleaning * create functions for dictionary parsers in the external file * remove files_exclude since post-processing does the job * no need to have group_group_dset when propagatin dsets * oneliner for paths to templates * add dset_per_group dict * add function to iteratively populate string based on triggers list * added recursive replacer for numbers * add recursive replaces for datasets * add function for text groups [iterative+recursive upd] * do not define triggers in the master script * transition to helper dictionaries * cleaning * comment main code blocks * rearrange parameters * add documentation strings to the functions * minor cleaning and changes * adapt build_trexio script * add consisteny check for dimensioning variables
2021-05-26 19:11:46 +02:00
from generator_tools import *
Fixed: rc was not checked for multiple dims
2021-03-28 15:43:49 +02:00
Major refactoring of the generator script (#47) * minor cleaning * create functions for dictionary parsers in the external file * remove files_exclude since post-processing does the job * no need to have group_group_dset when propagatin dsets * oneliner for paths to templates * add dset_per_group dict * add function to iteratively populate string based on triggers list * added recursive replacer for numbers * add recursive replaces for datasets * add function for text groups [iterative+recursive upd] * do not define triggers in the master script * transition to helper dictionaries * cleaning * comment main code blocks * rearrange parameters * add documentation strings to the functions * minor cleaning and changes * adapt build_trexio script * add consisteny check for dimensioning variables
2021-05-26 19:11:46 +02:00
# --------------------- GET CONFIGURATION FROM THE TREX.JSON ---------------- #
config_file = 'trex.json'
trex_config = read_json(config_file)
# --------------------------------------------------------------------------- #
initial generator
2021-03-05 16:50:44 +01:00
Major refactoring of the generator script (#47) * minor cleaning * create functions for dictionary parsers in the external file * remove files_exclude since post-processing does the job * no need to have group_group_dset when propagatin dsets * oneliner for paths to templates * add dset_per_group dict * add function to iteratively populate string based on triggers list * added recursive replacer for numbers * add recursive replaces for datasets * add function for text groups [iterative+recursive upd] * do not define triggers in the master script * transition to helper dictionaries * cleaning * comment main code blocks * rearrange parameters * add documentation strings to the functions * minor cleaning and changes * adapt build_trexio script * add consisteny check for dimensioning variables
2021-05-26 19:11:46 +02:00
# -------------------------------- [WIP] ------------------------------------ #
initial generator
2021-03-05 16:50:44 +01:00
print('Metadata I/O currently not supported')
# TODO, for now remove metadata-related stuff
Major refactoring of the generator script (#47) * minor cleaning * create functions for dictionary parsers in the external file * remove files_exclude since post-processing does the job * no need to have group_group_dset when propagatin dsets * oneliner for paths to templates * add dset_per_group dict * add function to iteratively populate string based on triggers list * added recursive replacer for numbers * add recursive replaces for datasets * add function for text groups [iterative+recursive upd] * do not define triggers in the master script * transition to helper dictionaries * cleaning * comment main code blocks * rearrange parameters * add documentation strings to the functions * minor cleaning and changes * adapt build_trexio script * add consisteny check for dimensioning variables
2021-05-26 19:11:46 +02:00
del trex_config['metadata']
# for now remove rdm from config because it functions are hardcoded
del trex_config['rdm']
initial generator
2021-03-05 16:50:44 +01:00
# TODO, for now remove char-related stuff
minor cleaning in the generator
2021-03-18 21:12:28 +01:00
print('Strings I/O currently not supported')
Major refactoring of the generator script (#47) * minor cleaning * create functions for dictionary parsers in the external file * remove files_exclude since post-processing does the job * no need to have group_group_dset when propagatin dsets * oneliner for paths to templates * add dset_per_group dict * add function to iteratively populate string based on triggers list * added recursive replacer for numbers * add recursive replaces for datasets * add function for text groups [iterative+recursive upd] * do not define triggers in the master script * transition to helper dictionaries * cleaning * comment main code blocks * rearrange parameters * add documentation strings to the functions * minor cleaning and changes * adapt build_trexio script * add consisteny check for dimensioning variables
2021-05-26 19:11:46 +02:00
# --------------------------------------------------------------------------- #
# -------------------- GET ATTRIBUTES FROM THE CONFIGURATION ---------------- #
group_dict = get_group_dict(trex_config)
[WIP] adapt generator for string datasets
2021-05-31 16:39:21 +02:00
detailed_nums = get_detailed_num_dict(trex_config)
# helper dictionaries that contain names of groups, nums or dsets as keys
dsets = get_dset_dict(trex_config)
detailed_dsets_nostr, detailed_dsets_str = split_dset_dict_detailed(dsets)
detailed_dsets = detailed_dsets_nostr.copy()
detailed_dsets.update(detailed_dsets_str)
Major refactoring of the generator script (#47) * minor cleaning * create functions for dictionary parsers in the external file * remove files_exclude since post-processing does the job * no need to have group_group_dset when propagatin dsets * oneliner for paths to templates * add dset_per_group dict * add function to iteratively populate string based on triggers list * added recursive replacer for numbers * add recursive replaces for datasets * add function for text groups [iterative+recursive upd] * do not define triggers in the master script * transition to helper dictionaries * cleaning * comment main code blocks * rearrange parameters * add documentation strings to the functions * minor cleaning and changes * adapt build_trexio script * add consisteny check for dimensioning variables
2021-05-26 19:11:46 +02:00
# consistency check for dimensioning variables
[WIP] adapt generator for string datasets
2021-05-31 16:39:21 +02:00
check_dim_consistency(detailed_nums, dsets)
Major refactoring of the generator script (#47) * minor cleaning * create functions for dictionary parsers in the external file * remove files_exclude since post-processing does the job * no need to have group_group_dset when propagatin dsets * oneliner for paths to templates * add dset_per_group dict * add function to iteratively populate string based on triggers list * added recursive replacer for numbers * add recursive replaces for datasets * add function for text groups [iterative+recursive upd] * do not define triggers in the master script * transition to helper dictionaries * cleaning * comment main code blocks * rearrange parameters * add documentation strings to the functions * minor cleaning and changes * adapt build_trexio script * add consisteny check for dimensioning variables
2021-05-26 19:11:46 +02:00
# --------------------------------------------------------------------------- #
# -------------------- GET TEMPLATED FILES TO BE POPULATED ------------------ #
source = ['front', 'text', 'hdf5']
# build helper dictionaries with paths per source directory
template_paths = get_template_paths(source)
# build helper dictionaries with source files per source directory
source_files = get_source_files(template_paths)
# build helper dictionaries with templated files
files_todo = get_files_todo(source_files)
# --------------------------------------------------------------------------- #
# ----------------------- POPULATE TEMPLATED FILES -------------------------- #
# populate files with iterative scheme, i.e. for unique functions
for fname in files_todo['auxiliary']:
[WIP] adapt generator for string datasets
2021-05-31 16:39:21 +02:00
iterative_populate_file(fname, template_paths, group_dict, detailed_dsets, detailed_nums)
Major refactoring of the generator script (#47) * minor cleaning * create functions for dictionary parsers in the external file * remove files_exclude since post-processing does the job * no need to have group_group_dset when propagatin dsets * oneliner for paths to templates * add dset_per_group dict * add function to iteratively populate string based on triggers list * added recursive replacer for numbers * add recursive replaces for datasets * add function for text groups [iterative+recursive upd] * do not define triggers in the master script * transition to helper dictionaries * cleaning * comment main code blocks * rearrange parameters * add documentation strings to the functions * minor cleaning and changes * adapt build_trexio script * add consisteny check for dimensioning variables
2021-05-26 19:11:46 +02:00
# populate has/read/write_num functions with recursive scheme
for fname in files_todo['num']:
[WIP] adapt generator for string datasets
2021-05-31 16:39:21 +02:00
recursive_populate_file(fname, template_paths, detailed_nums)
Major refactoring of the generator script (#47) * minor cleaning * create functions for dictionary parsers in the external file * remove files_exclude since post-processing does the job * no need to have group_group_dset when propagatin dsets * oneliner for paths to templates * add dset_per_group dict * add function to iteratively populate string based on triggers list * added recursive replacer for numbers * add recursive replaces for datasets * add function for text groups [iterative+recursive upd] * do not define triggers in the master script * transition to helper dictionaries * cleaning * comment main code blocks * rearrange parameters * add documentation strings to the functions * minor cleaning and changes * adapt build_trexio script * add consisteny check for dimensioning variables
2021-05-26 19:11:46 +02:00
[WIP] adapt generator for string datasets
2021-05-31 16:39:21 +02:00
# populate has/read/write_dset (numerical) functions with recursive scheme
split dataset files in _str and _data groups
2021-05-27 15:03:13 +02:00
for fname in files_todo['dset_data']:
[WIP] adapt generator for string datasets
2021-05-31 16:39:21 +02:00
recursive_populate_file(fname, template_paths, detailed_dsets_nostr)
# populate has/read/write_dset (strings) functions with recursive scheme
for fname in files_todo['dset_str']:
recursive_populate_file(fname, template_paths, detailed_dsets_str)
Major refactoring of the generator script (#47) * minor cleaning * create functions for dictionary parsers in the external file * remove files_exclude since post-processing does the job * no need to have group_group_dset when propagatin dsets * oneliner for paths to templates * add dset_per_group dict * add function to iteratively populate string based on triggers list * added recursive replacer for numbers * add recursive replaces for datasets * add function for text groups [iterative+recursive upd] * do not define triggers in the master script * transition to helper dictionaries * cleaning * comment main code blocks * rearrange parameters * add documentation strings to the functions * minor cleaning and changes * adapt build_trexio script * add consisteny check for dimensioning variables
2021-05-26 19:11:46 +02:00
# populate group-related functions with mixed (iterative+recursive) scheme [text backend]
for fname in files_todo['group']:
[WIP] adapt generator for string datasets
2021-05-31 16:39:21 +02:00
special_populate_text_group(fname, template_paths, group_dict, detailed_dsets_nostr, detailed_nums)
Major refactoring of the generator script (#47) * minor cleaning * create functions for dictionary parsers in the external file * remove files_exclude since post-processing does the job * no need to have group_group_dset when propagatin dsets * oneliner for paths to templates * add dset_per_group dict * add function to iteratively populate string based on triggers list * added recursive replacer for numbers * add recursive replaces for datasets * add function for text groups [iterative+recursive upd] * do not define triggers in the master script * transition to helper dictionaries * cleaning * comment main code blocks * rearrange parameters * add documentation strings to the functions * minor cleaning and changes * adapt build_trexio script * add consisteny check for dimensioning variables
2021-05-26 19:11:46 +02:00
# --------------------------------------------------------------------------- #
Reference in New Issue Copy Permalink