Pipeline¶
-
class
prism.
Pipeline
(modellink_obj, *, root_dir=None, working_dir=None, prefix=None, prism_par=None, emul_type=None, comm=None, **kwargs)[source]¶ Defines the
Pipeline
class of the PRISM package.The
Pipeline
class is the main user class of the PRISM package and provides a user-friendly environment that gives access to all operations within the package.-
_Projection__analyze_proj_hcube
(hcube)¶ Analyzes an emulator projection hypercube hcube.
Parameters: hcube (1D array_like of int of length {1, 2}) – Array containing the internal integer identifiers of the main model parameters that require a projection hypercube. Returns: - impl_min_hcube (1D
ndarray
object) – List containing the lowest implausibility value that can be reached in every single grid point on the given hypercube. - impl_los_hcube (1D
ndarray
object) – List containing the fraction of the total amount of evaluated samples in every single grid point on the given hypercube, that still satisfied the implausibility cut-off criterion.
- impl_min_hcube (1D
-
_Projection__draw_2D_proj_fig
(hcube, impl_min, impl_los, proj_res)¶ Draws the 2D projection figure for the provided hcube, given the impl_min and impl_los values.
Parameters: - hcube (1D array_like of int of length 1) – Array containing the internal integer identifiers of the main model parameters that require a projection figure.
- impl_min (1D
ndarray
object) – List containing the lowest implausibility value that can be reached in every single grid point on the given hypercube. - impl_los (1D
ndarray
object) – List containing the fraction of the total amount of evaluated samples in every single grid point on the given hypercube, that still satisfied the implausibility cut-off criterion. - proj_res (int) – Number of emulator evaluations used to generate the grid for the given hypercube.
-
_Projection__draw_3D_proj_fig
(hcube, impl_min, impl_los, proj_res)¶ Draws the 3D projection figure for the provided hcube, given the impl_min and impl_los values.
Parameters: - hcube (1D array_like of int of length 2) – Array containing the internal integer identifiers of the main model parameters that require a projection figure.
- impl_min (1D
ndarray
object) – List containing the lowest implausibility value that can be reached in every single grid point on the given hypercube. - impl_los (1D
ndarray
object) – List containing the fraction of the total amount of evaluated samples in every single grid point on the given hypercube, that still satisfied the implausibility cut-off criterion. - proj_res (int) – Number of emulator evaluations used to generate the grid for the given hypercube.
-
_Projection__get_default_input_arguments
()¶ Generates a dict containing default values for all input arguments.
Returns: kwargs_dict (dict) – Dict containing all default input argument values.
-
_Projection__get_default_parameters
()¶ Generates a dict containing default values for all projection parameters.
Returns: par_dict (dict) – Dict containing all default projection parameter values.
-
_Projection__get_fig_path
(hcube, emul_i=None)¶ Determines the absolute path of a projection figure corresponding to a provided projection hypercube hcube in emulator iteration emul_i and returns it.
Parameters: hcube (1D array_like of int of length {1, 2} or str) – Array containing the internal integer identifiers of the main model parameters that require a projection hypercube. If str, the name of hcube instead ( _Projection__get_hcube_name()
).Other Parameters: emul_i (int or None. Default: None) – If int, number indicating the requested emulator iteration. If None, the last iteration of the emulator will be used. Returns: - fig_path (str) – The absolute path to the requested projection figure.
- fig_path_s (str) – The absolute path to the smoothed version.
-
_Projection__get_hcube_name
(hcube)¶ Determines the name of a provided projection hypercube hcube and returns it.
Parameters: hcube (1D array_like of int of length {1, 2}) – Array containing the internal integer identifiers of the main model parameters that require a projection hypercube. Returns: hcube_name (str) – The name of this projection hypercube.
-
_Projection__get_proj_data
(hcube)¶ Returns the projection data belonging to the provided hypercube hcube.
Parameters: hcube (1D array_like of int of length {1, 2}) – Array containing the internal integer identifiers of the main model parameters that require a projection hypercube. Returns: - impl_min_hcube (1D
ndarray
object) – List containing the lowest implausibility value that can be reached in every single grid point on the given hypercube. - impl_los_hcube (1D
ndarray
object) – List containing the fraction of the total amount of evaluated samples in every single grid point on the given hypercube, that still satisfied the implausibility cut-off criterion. - proj_res (int) – Number of emulator evaluations used to generate the grid for the given hypercube.
- proj_depth (int) – Number of emulator evaluations used to generate the samples in every grid point for the given hypercube.
- impl_min_hcube (1D
-
_Projection__get_proj_hcube
(hcube)¶ Generates a projection hypercube hcube containing emulator evaluation samples The output of this function depends on the requested projection type.
Parameters: hcube (1D array_like of int of length {1, 2}) – Array containing the internal integer identifiers of the main model parameters that require a projection hypercube. Returns: proj_hcube (3D ndarray
object) – 3D projection hypercube of emulator evaluation samples. For 3D projections, the grid uses matrix indexing (second parameter varies the fastest).
-
_Projection__get_req_hcubes
(proj_par)¶ Determines which projection hypercubes have been requested by the user. Also checks if these projection hypercubes have been calculated before, and depending on the value of
force
, either skips them or recreates them.Parameters: proj_par (1D array_like of {int, str} or None) – For which model parameters to construct the projection figures. If 1D array_like, construct projection figures for all combinations of provided model parameters that are active, with a string referring to the name of the model parameter and an integer referring to the position in which the model parameter is shown in the details()
method. If None, projection figures are made for all active model parameters.Generates
- hcubes : list of lists
- List containing the parameter indices of the requested projection hypercubes.
- create_hcubes : list of lists
- List containing the parameter indices of the requested projection hypercubes that need to be created first.
-
_Projection__prepare_projections
(emul_i, proj_par, **kwargs)¶ Prepares the pipeline for the creation of the requested projections.
Parameters: - emul_i (int or None) – Number indicating the requested emulator iteration.
- proj_par (1D array_like of {int, str} or None) – For which model parameters to construct the projection figures.
If 1D array_like, construct projection figures for all combinations
of provided model parameters that are active, with a string
referring to the name of the model parameter and an integer
referring to the position in which the model parameter is shown in
the
details()
method. If None, projection figures are made for all active model parameters. - kwargs (dict) – Dict of keyword arguments that were provided to
project()
.
-
_Projection__process_input_arguments
(emul_i, **kwargs)¶ Processes the input arguments given to the
project()
method.Parameters: - emul_i (int or None) – Number indicating the requested emulator iteration.
- kwargs (dict) – Dict of keyword arguments that were provided to
project()
.
Generates
All default and provided arguments are saved to their respective properties.
-
_Projection__save_data
(data_dict)¶ Saves a given data dict
{keyword: data}
at the emulator iteration this class was initialized for, to the HDF5-file.Parameters: data_dict (dict) – Dict containing the data that needs to be saved to the HDF5-file.
Keyword Arguments: - keyword ({‘nD_proj_hcube’}) – String specifying the type of data that needs to be saved.
- data ({int, float, str, array_like} or dict) – The actual data that needs to be saved at data keyword keyword. If dict, save every item individually.
Generates
The specified data is saved to the HDF5-file.
-
_Projection__set_parameters
()¶ Sets the Projection parameters from the
prism_dict
property and saves them in the currentProjection
instance.
-
__init__
(modellink_obj, *, root_dir=None, working_dir=None, prefix=None, prism_par=None, emul_type=None, comm=None, **kwargs)[source]¶ Initialize an instance of the
Pipeline
class.Parameters: modellink_obj (
ModelLink
object) – Instance of theModelLink
subclass that links the emulated model to thisPipeline
instance.Other Parameters: - root_dir (str or None. Default: None) – String containing the absolute path of the root directory where all working directories are stored. If None, root directory will be set to the directory this class was initialized in.
- working_dir (str, bool or None. Default: None) – String containing the name of the working directory of the emulator in root_dir. If True, a new working directory will be created in root_dir. If None or False, working directory is set to the last one that was created in root_dir that starts with the given prefix. Note that providing an integer instead of a bool will not work here. If no directories are found, one will be created.
- prefix (str or None. Default: None) – String containing a prefix that is used for naming new working directories or scan for existing ones. If None, all directories in root_dir are considered working directories and ‘prism_’ will be used as a prefix for new ones.
- prism_par (array_like, dict, str or None. Default: None) – A dict containing the values for the PRISM parameters that need to be changed from their default values. If array_like, dict(prism_par) must generate a dict with the correct lay-out. If str, the string is the absolute or relative path to the file that contains the keys in the first column and the dict values in the second column. If a relative path is given, its path must be relative to root_dir or the current directory. If None, no changes will be made to the default parameters.
- emul_type (
Emulator
subclass or None. Default: None) – The type ofEmulator
to use in thisPipeline
instance. If None, use the default emulator instead. - comm (
Intracomm
object or None. Default: None) – The MPI intra-communicator to use in thisPipeline
instance orMPI.COMM_WORLD
if comm is None.
-
_call_model
(emul_i, par_set, data_idx)[source]¶ Obtain the output corresponding to the provided data_idx that is generated by the model for a given model parameter sample par_set. The current emulator iteration emul_i is also provided in case it is required by the
ModelLink
subclass.Parameters: - emul_i (int) – Number indicating the requested emulator iteration.
- par_set (1D array_like) – Model parameter sample to calculate the model output for.
- data_idx (list of tuples) – The list of data identifiers for which the model is requested to return the corresponding data values.
Returns: mod_out (1D
ndarray
object) – Model output corresponding to given par_set.
-
_compile_code_snippets
()[source]¶ Compiles all pre-defined built-in code snippets to code objects and saves them to
code_objects
. These code objects are used for performing standard operations in the_evaluate_sam_set()
method.
-
_do_impl_check
(emul_i, uni_impl_val)[source]¶ Performs an implausibility cut-off check on the provided implausibility values uni_impl_val at emulator iteration emul_i.
Parameters: - emul_i (int) – Number indicating the requested emulator iteration.
- uni_impl_val (1D array_like) – Array containing all univariate implausibility values corresponding to a certain parameter set for all data points.
Returns: - result (bool) – 1 if check was successful, 0 if it was not.
- impl_cut_val (float) – Implausibility value at the first real implausibility cut-off.
-
_evaluate_model
(emul_i, sam_set, data_idx)[source]¶ Evaluates the model for provided evaluation sample set sam_set at given data points data_idx.
This method automatically distributes the samples according to the various flags set in the
ModelLink
subclass.Parameters: - emul_i (int) – Number indicating the requested emulator iteration.
- sam_set (1D or 2D array_like) – Parameter/sample set to evaluate in the model.
- data_idx (list of tuples) – The list of data identifiers for which the model is requested to return the corresponding data values.
Returns: mod_set (2D
ndarray
object of shape(n_sam, n_data)
) – Array containing the data values corresponding to the requested data points.
-
_evaluate_sam_set
(emul_i, sam_set, exec_code)[source]¶ Evaluates a provided set of emulator evaluation samples sam_set at a given emulator iteration emul_i. The provided tuple of code snippets exec_code are executed using Python’s
exec()
function at specific points during the analysis.Parameters: - emul_i (int) – Number indicating the requested emulator iteration.
- sam_set (2D
ndarray
object) – Array containing model parameter value sets to be evaluated in all emulator systems in emulator iteration emul_i. - exec_code ({‘analyze’, ‘evaluate’, ‘hybrid’, ‘project’} or tuple) – Tuple of five code snippets
(pre_code, eval_code, anal_code, post_code, exit_code)
to be executed at specific points during the analysis. If string, use one of the built-in tuples incode_objects
instead.
Other Parameters: - pre_code (str or code object) – Code snippet to be executed before the evaluation of sam_set starts.
- eval_code (str or code object) – Code snippet to be executed after the evaluation of each sample in sam_set.
- anal_code (str or code object) – Code snippet to be executed after the analysis of each sample in sam_set. This code snippet is only executed by the controller.
- post_code (str or code object) – Code snippet to be executed after the evaluation of sam_set ends.
- exit_code (str or code object) – Code snippet to be executed before returning the results of the evaluation of sam_set. This code snippet is only executed by the controller.
Returns: results (object) – The object that is assigned to
results
, which is defaulted to None if no code snippet changes it. Preferably, the execution of post_code and/or exit_code modifiesresults
. All MPI ranks return it.Notes
If any of the code snippets is provided as a string, it will be compiled into a code object before starting the evaluation.
-
_get_default_parameters
()[source]¶ Generates a dict containing default values for all pipeline parameters.
Returns: par_dict (dict) – Dict containing all default pipeline parameter values.
-
_get_eval_sam_set
(emul_i)[source]¶ Generates an emulator evaluation sample set to be used for analyzing an emulator iteration. Currently uses the
lhd()
function.Parameters: emul_i (int) – Number indicating the requested emulator iteration. Returns: eval_sam_set (2D ndarray
object) – Array containing the evaluation samples.
-
_get_ext_real_set
(ext_real_set)[source]¶ Processes an externally provided model realization set ext_real_set, containing the used sample set and the corresponding data value set.
Parameters: ext_real_set (list, dict or None) – List of arrays containing an externally calculated set of model evaluation samples and its data values, a dict with keys ['sam_set', 'mod_set']
containing these arrays or None if no external set needs to be used.Returns:
-
_get_iteration_data
(emul_i, sam_set, ext_sam_set, ext_mod_set)[source]¶ Obtains the model realization data for given emulator iteration emul_i by evaluating the provided sam_set in the model and distributing model outputs to the correct emulator systems.
Parameters: - emul_i (int) – Number indicating the requested emulator iteration.
- sam_set (2D
ndarray
object) – Array containing the model evaluation samples. - ext_sam_set (1D or 2D
ndarray
object) – Array containing the externally provided model evaluation samples. - ext_mod_set (1D or 2D
ndarray
object) – Array containing the model outputs of all specified externally provided model evaluation samples.
Generates
-
_get_md_var
(emul_i, par_set)[source]¶ Retrieves the model discrepancy variances, which includes all variances that are created by the model provided by the
ModelLink
instance. This method tries to call theget_md_var()
method, and assumes a default model discrepancy variance of1/6th
the data value if it cannot be called. If the data value space is not linear, then this default value is calculated such to reflect that.Parameters: - emul_i (int) – Number indicating the requested emulator iteration.
- par_set (1D
ndarray
object) – Model parameter value set to calculate the model discrepancy variances for.
Returns: var_md (2D
ndarray
object) – Variance of the model discrepancy.
-
_get_mock_data
(mock_par)[source]¶ Generates mock data and loads it into the
ModelLink
object that was provided during class initialization. This function overwrites theModelLink
properties holding the parameter estimates, data values and data errors.Parameters: mock_par (1D array_like or None) – If 1D array_like, use the provided parameter estimates to create the mock data. If None, a random parameter set will be generated as parameter estimates. Generates
Overwrites the corresponding
ModelLink
class properties with the generated values.
-
_get_n_eval_sam
(emul_i)[source]¶ This function calculates the total number of emulator evaluation samples at a given emulator iteration emul_i from
base_eval_sam
.Parameters: emul_i (int) – Number indicating the requested emulator iteration. Returns: n_eval_sam (int) – Total number of emulator evaluation samples.
-
_get_paths
(root_dir, working_dir, prefix)[source]¶ Obtains the path for the root directory, working directory and parameters file for PRISM.
Parameters: - root_dir (str or None) – String containing the absolute path of the root directory where all working directories are stored. If None, root directory will be set to the directory this class was initialized in.
- working_dir (str, bool or None) – String containing the name of the working directory of the emulator in root_dir. If True, a new working directory will be created in root_dir. If None or False, working directory is set to the last one that was created in root_dir that starts with the given prefix. Note that providing an integer instead of a bool will not work here. If no directories are found, one will be created.
- prefix (str or None) – String containing a prefix that is used for naming new working directories or scan for existing ones. If None, all directories in root_dir are considered working directories and ‘prism_’ will be used as a prefix for new ones.
Generates
The absolute paths to the root directory, working directory, emulator master HDF5-file and PRISM parameters file.
-
_get_uni_impl
(emul_i, par_set, adj_exp_val, adj_var_val)[source]¶ Calculates the univariate implausibility values at a given emulator iteration emul_i for specified expectation and variance values adj_exp_val and adj_var_val, corresponding to given par_set.
Parameters: - emul_i (int) – Number indicating the requested emulator iteration.
- par_set (1D
ndarray
object) – Model parameter value set to calculate the univariate implausibility values for. Only used to pass to theget_md_var()
method. - adj_exp_val, adj_var_val (1D array_like) – The adjusted expectation and variance values to calculate the univariate implausibility for.
Returns: uni_impl_val (1D
ndarray
object) – Univariate implausibility value for all requested emulator systems.
-
_listen_for_calls
()[source]¶ All worker ranks in the
comm
communicator start listening for calls from the corresponding controller rank and will attempt to execute the received message. Listening for calls continues until_worker_mode
is set to False.This method is automatically initialized and finalized when using the
worker_mode
context manager.
-
_load_data
()[source]¶ Loads in all the important pipeline data into memory for the controller rank. If it is detected that the last emulator iteration has not been analyzed yet, the implausibility analysis parameters are read in from the PRISM parameters file and temporarily stored in memory.
Generates
All relevant pipeline data up to the last emulator iteration is loaded into memory.
-
_make_call
(exec_fn, *args, **kwargs)[source]¶ Send the provided exec_fn to all worker ranks, if they are listening for calls, and tell them to execute it using the provided args and kwargs. All ranks that call this function will execute exec_fn as well.
If used within the
worker_mode
context manager, this function should only be called by the controller. If not, it should be called by all ranks that must execute exec_fn.Parameters: - exec_fn (str, callable or None) – If string, a callable attribute of this
Pipeline
instance or a callable object that the workers should execute if not. If None, the workers stop listening for calls instead (disables worker mode). - args (tuple) – Positional arguments that need to be provided to exec_fn.
- kwargs (dict) – Keyword arguments that need to be provided to exec_fn.
- exec_fn (str, callable or None) – If string, a callable attribute of this
-
_multi_call_model
(emul_i, sam_set, data_idx)[source]¶ Obtain the output set corresponding to the provided data_idx that is generated by the model for a given model parameter sample set sam_set. The current emulator iteration emul_i is also provided in case it is required by the
ModelLink
subclass.This is a multi-version of
_call_model()
.Parameters: - emul_i (int) – Number indicating the requested emulator iteration.
- sam_set (2D array_like) – Model parameter sample set to calculate the model output for.
- data_idx (list of tuples) – The list of data identifiers for which the model is requested to return the corresponding data values.
Returns: mod_set (2D
ndarray
object) – Model output set corresponding to given sam_set.
-
_read_parameters
(prism_par)[source]¶ Reads in all parameters in the provided prism_par and saves them as a dict in the current
Pipeline
instance.
-
_save_data
(data_dict)[source]¶ Saves a given data dict
{keyword: data}
at the last emulator iteration to the HDF5-file and as an data attribute to the currentPipeline
instance.Parameters: data_dict (dict) – Dict containing the data that needs to be saved to the HDF5-file.
Keyword Arguments: - keyword ({‘impl_par’, ‘impl_sam’, ‘n_eval_sam’}) – String specifying the type of data that needs to be saved.
- data ({int, float, str, array_like} or dict) – The actual data that needs to be saved at data keyword keyword. If dict, save every item individually.
Generates
The specified data is saved to the HDF5-file.
-
_save_statistics
(emul_i, stat_dict)[source]¶ Saves a given statistics dict
{keyword: [value, unit]}
at emulator iteration emul_i to the HDF5-file. The provided values are always saved as strings.Parameters: emul_i (int) – Number indicating the requested emulator iteration.
Keyword Arguments: - keyword (str) – String containing the name/keyword of the statistic that is being saved.
- value (int, float or str) – The value of the statistic.
- unit (str) – The unit of the statistic.
-
_set_impl_par
(impl_cut)[source]¶ Sets the
impl_cut
andcut_idx
properties for implausibility evaluations using the read-in PRISM parameters file and the provided impl_cut.Parameters: impl_cut (list of float or None) – Incomplete, shortened impl_cut-offs list to be used during the analysis of this emulator iteration. If None, use the read-in PRISM parameters file instead. Generates
- impl_cut : 1D
ndarray
object - Full list containing the impl_cut-offs for all data points provided to the emulator.
- cut_idx : int
- Index of the first impl_cut-off in the impl_cut list that is not 0.
- impl_cut : 1D
-
_set_parameters
()[source]¶ Sets the Pipeline parameters from the
prism_dict
property and saves them in the currentPipeline
instance.
-
analyze
(*, impl_cut=None)[source]¶ Analyzes the emulator at the last emulator iteration for a large number of emulator evaluation samples. All samples that survive the implausibility checks set by the provided impl_cut, are used in the construction of the next emulator iteration.
Other Parameters: impl_cut (list of float or None. Default: None) – Incomplete, shortened impl_cut-offs list to be used during the analysis of this emulator iteration. If None, the currently set implausibility cut-off values ( impl_cut
) will be used if this is the first analysis or the ‘impl_cut’ value inprism_dict
is used if this is a reanalysis.Generates
- impl_sam : 2D
ndarray
object - Array containing all emulator evaluation samples that survived the implausibility checks.
- impl_sam : 2D
-
construct
(emul_i=None, *, analyze=True, ext_real_set=None, force=False)[source]¶ Constructs the emulator at the specified emulator iteration emul_i, and performs an implausibility analysis on the emulator iteration right afterward if requested (
analyze()
).Other Parameters: - emul_i (int or None. Default: None) – If int, number indicating the requested emulator iteration. If None, the next iteration of the emulator will be constructed.
- analyze (bool. Default: True) – Bool indicating whether or not to perform an analysis after the specified emulator iteration has been successfully constructed, which is required for constructing the next iteration.
- ext_real_set (list, dict or None. Default: None) – List of arrays containing an externally calculated set of model
evaluation samples and its data values, a dict with keys
['sam_set', 'mod_set']
containing these arrays or None if no external set needs to be used. - force (bool. Default: False) – Controls what to do if the specified emulator iteration emul_i already (partly) exists. If False, finish construction of the specified iteration or skip it if already finished. If True, reconstruct the specified iteration entirely.
Generates
A new HDF5-group with the emulator iteration as its name, in the loaded emulator master file, containing emulator data required for this emulator iteration.
Notes
Using an emulator iteration that has been (partly) constructed before, will finish construction or skip it if already finished when force is False; or it will delete that and all following iterations, and reconstruct the specified iteration when force is True. Using emul_i = 1 and force is True is equivalent to reconstructing the entire emulator.
If no implausibility analysis is requested, then the implausibility parameters are read in from the PRISM parameters file and temporarily stored in memory in order to enable the usage of the
evaluate()
andproject()
methods.
-
details
(emul_i=None)[source]¶ Prints the details/properties of the currently loaded
Pipeline
instance at given emulator iteration emul_i. SeeProps
for detailed descriptions of all printed properties.Other Parameters: emul_i (int or None. Default: None) – If int, number indicating the requested emulator iteration. If None, the last iteration of the emulator will be used. Props
- Working directory
- The relative path to the working directory of the emulator starting at the current working directory.
- Emulator type
- The type of this emulator, corresponding to the
emul_type
of the provided emul_type duringPipeline
initialization. If no emulator type was provided during initialization, this is ‘default’. - ModelLink subclass
- Name of the
ModelLink
subclass used to construct this emulator. - Emulation method
- Indicates the combination of regression and Gaussian emulation methods that have been used for this emulator.
- Mock data used?
- Whether or not mock data has been used to construct this emulator. If so, the printed estimates for all model parameters are the parameter values used to create the mock data.
- Emulator iteration
- The iteration of the emulator this details overview is about. By default, this is the last (partly) constructed iteration.
- Construction completed?
- Whether or not the construction of this emulator iteration is completed. If not, the missing components for each emulator system are listed and the remaining information of this iteration is not printed.
- Plausible regions?
- Whether or not plausible regions have been found during the analysis of this emulator iteration. If no analysis has been done yet, “N/A” will be printed.
- Projections available?
- Whether or not projections have been created for this emulator iteration. If projections are available and analysis has been done, but with different implausibility cut-offs, a “desync” note is added. Also prints number of available projections versus maximum number of projections in parentheses.
- # of model evaluation samples
- The total number of model evaluation samples used to construct all emulator iterations up to this iteration, with the number for every individual iteration in parentheses.
- # of plausible/analyzed samples
- The number of emulator evaluation samples that passed the implausibility check out of the total number of analyzed samples in this emulator iteration. This is the number of model evaluation samples that was/will be used for the construction of the next emulator iteration. If no analysis has been done, the numbers show up as “-“.
- % of parameter space remaining
- The percentage of the total number of analyzed samples that passed the implausibility check in this emulator iteration. If no analysis has been done, the number shows up as “-“.
- # of active/total parameters
- The number of model parameters that was considered active during
the construction of this emulator iteration, compared to the total
number of model parameters defined in the used
ModelLink
subclass. - # of emulated data points
- The number of data points that have been emulated in this emulator iteration.
- # of emulator systems
- The total number of emulator systems that are required in this emulator. The number of active emulator systems is equal to the number of data points.
- Parameter space
- Lists the name, lower and upper value boundaries and estimate (if
provided) of all model parameters defined in the used
ModelLink
subclass. An asterisk is printed in front of the parameter name if this model parameter was considered active during the construction of this emulator iteration. A question mark is used instead if the construction of this emulator iteration is not finished.
-
evaluate
(sam_set, emul_i=None)[source]¶ Evaluates the given model parameter sample set sam_set up to given emulator iteration emul_i. The output of this function depends on the number of dimensions in sam_set. The output is always provided on the controller rank.
Parameters: sam_set (1D or 2D array_like or dict) – Array containing model parameter value sets to be evaluated in the emulator up to emulator iteration emul_i. Other Parameters: emul_i (int or None. Default: None) – If int, number indicating the requested emulator iteration. If None, the last iteration of the emulator will be used. Returns: - impl_check (list of bool) – List of bool indicating whether or not the given samples passed the implausibility check at the given emulator iteration emul_i.
- emul_i_stop (list of int) – List containing the last emulator iterations at which the given samples are still within the plausible region of the emulator.
- adj_exp_val (2D
ndarray
object) – Array containing the adjusted expectation values for all given samples. - adj_var_val (2D
ndarray
object) – Array containing the adjusted variance values for all given samples. - uni_impl_val (2D
ndarray
object) – Array containing the univariate implausibility values for all given samples.
Prints (if 1D sam_set)
- emul_i_stop : int
- Last emulator iteration at which the given sample is still within the plausible region of the emulator.
- adj_exp_val : 1D
ndarray
object - The adjusted expectation values for the given sample.
- adj_var_val : 1D
ndarray
object - The adjusted variance values for the given sample.
- sigma_val : 1D
ndarray
object - The corresponding sigma value for the given sample.
- uni_impl_val : 1D
ndarray
object - The univariate implausibility values for the given sample.
Notes
If given emulator iteration emul_i has been analyzed before, the implausibility parameters of the last analysis are used. If not, then the values are used that were read in when the emulator was loaded or that have been set by the user.
-
project
(emul_i=None, proj_par=None, **kwargs)¶ Analyzes the emulator iteration emul_i and constructs a series of projection figures detailing the behavior of the model parameters corresponding to the given proj_par. The input and output depend on the number of model parameters
n_par
.All optional keyword arguments (except force) control various aspects of drawing the projection figures and do not affect the projection data that is saved to HDF5. This is instead influenced by the
proj_res
andproj_depth
properties.Parameters: - emul_i (int or None. Default: None) – If int, number indicating the requested emulator iteration. If None, the last iteration of the emulator will be used.
- proj_par (1D array_like of {int, str} or None. Default: None) – For which model parameters to construct the projection figures.
If 1D array_like, construct projection figures for all combinations
of provided model parameters that are active, with a string
referring to the name of the model parameter and an integer
referring to the position in which the model parameter is shown in
the
details()
method. If None, projection figures are made for all active model parameters.
Keyword Arguments: - proj_type ({‘2D’, ‘3D’, ‘both’}. Default: ‘2D’ (2D), ‘both’ (nD)) – String indicating which projection type to create for all supplied
active parameters.
If
n_par
== 2, this is always ‘2D’ (and cannot be modified). - figure (bool. Default: True) – Whether or not to create the projection figures. If True, the figures are calculated, drawn and saved. If False, the figures are calculated and their data is returned in a dict.
- align ({‘row’/’horizontal’, ‘col’/’column’/’vertical’}. Default: ‘col’) – If figure is True, string indicating how to position the two subplots. If ‘row’/’horizontal’, the subplots are positioned on a single row. If ‘col’/’column’/’vertical’, the subplots are positioned on a single column.
- show_cuts (bool. Default: False) – If figure is True and proj_type is not ‘3D’, whether to show all implausibility cut-offs in the 2D projections (True) or only the first cut-off (False).
- smooth (bool. Default: False) – Controls what to do if a grid point contains no plausible samples, but does contain a minimum implausibility value below the first non-wildcard cut-off. If False, these values are kept, which can show up as artifact-like features in the projection figure. If True, these values are set to the first cut-off, removing them from the projection figure. Doing this may also remove interesting features. This does not affect the projection data saved to HDF5. Smoothed figures have an ‘_s’ string appended to their filenames.
- force (bool. Default: False) – Controls what to do if a projection hypercube has been calculated at the emulator iteration emul_i before. If False, it will use the previously acquired projection data to create the projection figure. If True, it will recalculate all the data required to create the projection figure. Note that this will also delete all associated projection figures.
- fig_kwargs (dict. Default: {‘figsize’: (6.4, 4.8), ‘dpi’: 100}) – Dict of keyword arguments to be used when creating the subplots
figure. It takes all arguments that can be provided to the
figure()
function. - impl_kwargs_2D (dict. Default: {}) – Dict of keyword arguments to be used for making the minimum
implausibility (top/left) plot in the 2D projection figures. It
takes all arguments that can be provided to the
plot()
function. - impl_kwargs_3D (dict. Default: {‘cmap’: ‘rainforest_r’}) – Dict of keyword arguments to be used for making the minimum
implausibility (top/left) plot in the 3D projection figures. It
takes all arguments that can be provided to the
hexbin()
function. - los_kwargs_2D (dict. Default: {}) – Dict of keyword arguments to be used for making the line-of-sight
(bottom/right) plot in the 2D projection figures. It takes all
arguments that can be provided to the
plot()
function. - los_kwargs_3D (dict. Default: {‘cmap’: ‘blaze’}) – Dict of keyword arguments to be used for making the line-of-sight
(bottom/right) plot in the 3D projection figures. It takes all
arguments that can be provided to the
hexbin()
function. - line_kwargs_est (dict. Default: {‘linestyle’: ‘–’, ‘color’: ‘grey’}) – Dict of keyword arguments to be used for drawing the parameter
estimate lines in both plots. It takes all arguments that can be
provided to the
draw()
function. - line_kwargs_cut (dict. Default: {‘color’: ‘r’}) – Dict of keyword arguments to be used for drawing the implausibility
cut-off line(s) in the top/left plot in the 2D projection figures.
It takes all arguments that can be provided to the
draw()
function.
Returns: fig_data (dict of dicts) – Dict containing the data for every requested projection figure, split up into the ‘impl_min’ and ‘impl_los’ dicts. For 2D projections, every dict contains a list with the x and y values. For 3D projections, it contains the x, y and z values. Note that due to the figures being interpolations, the y/z values can be below zero or the line-of-sight values being above unity.
Generates (if figure is True)
A series of projection figures detailing the behavior of the model. The lay-out and output of the projection figures depend on the type of figure:
2D projection figure: The output will feature a figure with two subplots for every active model parameter (
n_par
). Every figure gives details about the behavior of the corresponding model parameter, by showing the minimum implausibility value (top/left) and the line-of-sight depth (bottom/right) obtained at the specified parameter value, independent of the values of the other parameters.3D projection figure (only if
n_par
> 2): The output will feature a figure with two subplots for every combination of two active model parameters that can be made (n_par*(n_par-1)/2
). Every figure gives details about the behavior of the corresponding model parameters, as well as their dependency on each other. This is done by showing the minimum implausibility (top/left) and the line-of-sight depth (bottom/right) obtained at the specified parameter values, independent of the values of the remaining model parameters.Notes
If given emulator iteration emul_i has been analyzed before, the implausibility parameters of the last analysis are used. If not, then the values are used that were read in when the emulator was loaded or that have been set by the user.
-
run
(emul_i=None, *, force=False)[source]¶ Calls the
construct()
method to start the construction of the given iteration of the emulator and creates the projection figures right afterward if this construction was successful.Other Parameters: - emul_i (int or None. Default: None) – If int, number indicating the requested emulator iteration. If None, the next iteration of the emulator will be constructed.
- force (bool. Default: False) – Controls what to do if the specified emulator iteration emul_i already (partly) exists. If False, finish construction of the specified iteration or skip it if already finished. If True, reconstruct the specified iteration entirely.
-
File
¶ Custom
File
class that has added logging and automatically useshdf5_file
as the HDF5-file to open.Type: File
-
base_eval_sam
¶ Base number of emulator evaluations used to analyze the emulator systems. This number is scaled up by the number of model parameters and the current emulator iteration to generate the true number of emulator evaluations (
n_eval_sam
).Type: int
-
code_objects
¶ Collection of pre-compiled built-in code snippets that are used in the
_evaluate_sam_set()
method.Type: dict of code objects
-
comm
¶ The MPI intra-communicator that is used in this
Pipeline
instance. By default, this isMPI.COMM_WORLD
.Type: Intracomm
-
cut_idx
¶ The index of the first non-wildcard in a list of implausibility values. This is equivalent to the number of wildcards leading the cut-off values in
impl_cut
.Type: int
-
do_active_anal
¶ Whether or not to do an active parameters analysis during the construction of the emulator systems.
Type: bool
-
do_logging
¶ Whether or not to save all logging messages. If False, all logging messages of level
INFO
and below are ignored. It also enables/disables the progress bar for making projections.Type: bool
-
freeze_active_par
¶ Whether or not previously active parameters always stay active if possible.
Type: bool
-
impl_cut
¶ The non-wildcard univariate implausibility cut-off values for an emulator iteration. Setting it with the reduced implausibility cut-off list will change the values of
cut_idx
and this property at the last emulator iteration.Type: list of float
-
impl_sam
¶ The model evaluation samples that will be added to the next emulator iteration.
Type: ndarray
-
is_controller
¶ Whether or not this MPI process is a controller rank. If no MPI is used, this is always True.
Type: bool
-
is_worker
¶ Whether or not this MPI process is a worker rank. If no MPI is used, this is always False.
Type: bool
-
n_eval_sam
¶ The number of evaluation samples used to analyze an emulator iteration of the emulator systems. The number of plausible evaluation samples is stored in
n_impl_sam
. It is zero if the specified iteration has not been analyzed yet.Type: int
-
n_impl_sam
¶ Number of model evaluation samples that passed the implausibility checks during the analysis of an emulator iteration. It is zero if the specified iteration has not been analyzed yet or has no plausible samples.
Type: int
-
n_sam_init
¶ Number of evaluation samples used to construct the first iteration of the emulator systems.
Type: int
-
pot_active_par
¶ The potentially active parameters. Only parameters from this list can become active during the active parameters analysis. If
do_active_anal
is False, all parameters in this list will be active.Type: list of str
-
prism_dict
¶ Dictionary containing all PRISM parameters that were provided during
Pipeline
initialization.Type: dict
-
proj_depth
¶ Number of emulator evaluations that will be used to generate the samples in every grid point for the projection figures. Note that when making 2D projections of nD models, the used depth will be this number multiplied by
proj_res
.Type: int
-
proj_res
¶ Number of emulator evaluations that will be used to generate the grid for the projection figures.
Type: int
-
worker_mode
¶ Special context manager within which all code is executed in worker mode. In worker mode, all worker ranks are continuously listening for calls from the controller rank made with
_make_call()
.Note that all code within the context manager is executed by all ranks, with the worker ranks executing it after the controller rank exited. It is therefore advised to use an if-statement inside to make sure only the controller rank executes the code.
Using this context manager allows for easier use of PRISM in combination with serial/OpenMP codes (like MCMC methods).
Type: _GeneratorContextManager
-