Identify the best models in grid search

Identify the best models from training or test metrics calculated for all the models in a grid search. identify_best_models() duplicates Yangkang's model selection code and so needs to be updated as he changes the selection criteria.

Usage

identify_best_models(metrics, cor_threshold = 0.9, cor_metric = "min_dist_cor")

Arguments

metrics

A table of evaluation metrics for all the models in the grid search. This is saved to an Rds during Yangkang's grid search workflow and can be calculated on either the training models (often train) or the reserved testing models (test).

cor_threshold

A threshold in correlation between model distributions and training (S&T) distributions below which models will be excluded for selection by some of the criteria.

cor_metric

The particular correlation metric to which the threshold is applied - a column in metrics.

Value

A data.frame with:

type

the selection criteria used to select the model in its short form

model

the model id as it apears in metrics$model

Details

Note the appropriate value for cor_threshold depends on model resolution. To get an equivalent amount of filtering a lower threshold is needed when the cell size is smaller.

Last update 2025-12-12