Evaluate a set of BirdFlow HDF5 models (S3 method)

Submits evaluation jobs (one per HDF5) via batchtools using the provided evaluation_function (defaults to evaluate_model()). Retries failed/expired jobs up to two times. Returns a list of results gathered with batchtools::reduceResultsList().

Usage

# S3 method for class 'BatchBirdFlowEvaluator'
evaluate(
  object,
  data,
  evaluation_function = evaluate_model,
  test_one_evaluate = FALSE,
  ...
)

Arguments

object

A BatchBirdFlowEvaluator().

data

A list or object containing evaluation data. Typically split_data$training_data or split_data$test_data, where index [[1]] is interval data and [[2]] is 1-week data used for PIT calibration.

evaluation_function

Function of signature function(bf_path, data, params) returning a result per model (defaults to evaluate_model()).

test_one_evaluate

Logical; if TRUE, just do some test on this local session instead of submitting it to slurm

...

not used

Value

A list as returned by batchtools::reduceResultsList() (often a list of per-model lists/data frames produced by evaluation_function). Side effects: writes PIT objects to disk (if done by evaluation_function), prints job status.

Details

HDF5 models are discovered in evaluator$batch_trainer$params$hdf_dir matching the current species and resolution. A batchtools registry is created under output_path with a timestamped suffix.

See also

evaluate_model(), BatchBirdFlowEvaluator(), batchtools::reduceResultsList()

Examples

if (FALSE) { # \dontrun{
ev <- BatchBirdFlowEvaluator(trainer)
res <- evaluate(ev, split_data$test_data)
} # }