Convergence, robustness and validation

Convergence of sensitivity indices

class gsa_framework.convergence_robustness_validation.convergence.Convergence(filepath_Y, num_params, gsa_func, gsa_label, write_dir, num_steps=10, **kwargs)

Class that computes iteration steps and sensitivity indices to monitor their convergence.

generate_converging_gsa_indices(gsa_func)

Computes sensitivity indices at each convergence step.

Parameters

gsa_func (function or method) – Corresponds to generate_gsa_indices_based_on_method from the class SensitivityAnalysisMethod. Needs to accept an argument selected_iterations.

Returns

sa_convergence_dict – Dictionary with sensitivity indices at each convergence step.

Return type

dict

generate_iterations_for_convergence()

Computes iteration steps, accounts for block design of some sampling methods.

make_dirs()

Create subdirectories where intermediate results will be stored.

plot_convergence(sa_convergence_dict, parameter_inds=None, fig_format=[])

Plots sensitivity indices for model inputs parameter_inds for all convergence steps.

run_convergence(parameter_inds=None, fig_format=())

Calls function that computes converging indices, saves them and plots for model inputs parameter_inds.

Robustness of GSA

class gsa_framework.convergence_robustness_validation.robustness.Robustness(stability_dicts, write_dir, **kwargs)

Class that computes statistics to monitor robustness and convergence of sensitivity indices and rankings.

get_bootstrap_rankings(bootstrap_data, sa_mean_results, tag, num_ranks=10, rho_choice='spearmanr')

Get clustered rankings from bootstrap sensitivity indices.

Parameters

  • bootstrap_data (dict) – Dictionary where keys are sensitivity methods names and values are arrays with sensitivity indices from bootstrapping in rows for each model input in columns.

  • sa_mean_results (dict) – Dictionary where keys are sensitivity methods names and values are mean results for each model input over all bootstrap samples.

  • tag (str) – Tag to save clustered rankings.

  • num_ranks (int) – Number of clusters.

Returns

bootstrap_rankings – Dictionary where keys are sensitivity methods names and values are clustered ranks for all model inputs.

Return type

dict

get_bootstrap_rankings_2(bootstrap_data, sa_mean_results, tag, num_ranks=10, rho_choice='spearmanr')

Get clustered rankings from bootstrap sensitivity indices.

Parameters

  • bootstrap_data (dict) – Dictionary where keys are sensitivity methods names and values are arrays with sensitivity indices from bootstrapping in rows for each model input in columns.

  • sa_mean_results (dict) – Dictionary where keys are sensitivity methods names and values are mean results for each model input over all bootstrap samples.

  • tag (str) – Tag to save clustered rankings.

  • num_ranks (int) – Number of clusters.

Returns

bootstrap_rankings – Dictionary where keys are sensitivity methods names and values are clustered ranks for all model inputs.

Return type

dict

get_bootstrap_rankings_width_percentiles(bootstrap_rankings, q_min=5, q_max=95)

Get percentiles of $Stat_{ranking}$ with respect to ranking on the last convergence step for robustness results.

get_confidence_intervals(bootstrap_data, ci_type='student')

Compute confidence intervals of bootstrap sensitivity indices.

get_confidence_intervals_max(confidence_intervals)

Compute statistic to monitor convergence of sensitivity indices.

The statistic is denoted as $Stat_{indices}$ in [KMF21], and is computed as a maximum confidence interval of sensitivity indices among all model inputs.

get_data_from_robustness_dicts(robustness_dicts)

Extract sensitivity methods, iterations, bootstrap data and mean sensitivity results.

get_one_clustered_ranking(array, num_ranks, breaks=None)

Compute clustered ranking of array given number of ranks (clusters) and, optionally, clusters themselves.

get_rankings_convergence_to_last(sa_mean_results, num_ranks=10)

Get convergence of ranking statistic $Stat_{ranking}$ with respect to ranking on the last convergence step.

get_stat_medians(bootstrap_data)

Get medians of statistics / metrics within sensitivity methods, such as $r^2$ in regression.

make_dirs()

Create subdirectories where intermediate results will be stored.

Validation of GSA results

class gsa_framework.convergence_robustness_validation.validation.Validation(model, write_dir, iterations=500, seed=None, default_x_rescaled=None, model_output_name='Model output')

Class to validate sensitivity analysis results with histograms and correlation plots.

References

Paper:

Kim, Mutel, and Froemelt [KMF21]

generate_X_rescaled_all_inputs_vary()

Rescale unitcube samples when all model inputs vary.

generate_Y_all_inputs_vary()

Run model when all inputs vary.

get_influential_Y_from_gsa(gsa_indices, num_influential, tag=None)

Run model when only influential inputs vary based on sensitivity indices values.

Parameters

  • gsa_indices (array) – Array with sensitivity indices values for all model inputs.

  • num_influential (int) – Number of first most influential inputs to vary.

  • tag (str) – Tag to save results.

Returns

influential_Y – Model outputs when only influential inputs vary.

Return type

array

get_influential_Y_from_parameter_choice(influential_inputs, tag=None)

Run model when only influential inputs vary based on chosen influential inputs.

make_dirs()

Create subdirectories where intermediate results will be stored.