Evaluate the model performance of a model

performance() fits the model(s) in x and calculates a performance table, i.e. the same type of output as cv_performance(). The reported training error refers to the full model data, while test error is the error for the data provided in the newdata argument. Test error is NA if no newdata is specified.

Usage

performance(x, ...)

# S3 method for model
performance(
  x,
  newdata = NULL,
  metric = NULL,
  eval_weights = x$weights,
  newweights = NULL,
  na.rm = FALSE,
  ...
)

# S3 method for model_fm_xgb
performance(
  x,
  newdata = NULL,
  metric = NULL,
  eval_weights = x$weights,
  newweights = NULL,
  na.rm = FALSE,
  ...
)

# S3 method for default
performance(
  x,
  newdata = NULL,
  metric = NULL,
  eval_weights = x$weights,
  newweights = NULL,
  na.rm = FALSE,
  ...
)

# S3 method for multimodel
performance(
  x,
  newdata = NULL,
  metric = NULL,
  eval_weights = "default",
  newweights = NULL,
  na.rm = FALSE,
  param = TRUE,
  ...
)

# S3 method for cv
performance(x, newdata = NULL, metric = x$metric[1], ...)

Arguments

x

A model, or other object.

...

Passed to the metric function.

newdata

Additional data for out-of-sample evaluation

metric

A metric (see metrics), specified either as a character string (name of the metric function), or as a named list of length 1, as in list(rmse = rmse). metric=NULL selects the default metric, see default_metric.

eval_weights, newweights

Evaluation weights for training and test data, respectively.

na.rm

Logical: Whether NA values should be excluded from computations.

param

Logical: Keep parameters from the parameter table in the output?

Value

performance() returns a performance table. This is a param_table with additional class “performance”

having additional information in its attributes.

Methods

• performance.model() is the core method.

• performance.default(x, ...) executes x %>% model %>% performance(...), where x is a fitted model.

• performance.multimodel(x, ...) runs performance.model() for the models included in x and combines the results in performance table with n_model(x) rows. It uses metric = default_metric(x) by default.

• performance.cv() extracts the multimodel from the “cv” object and then applies performance.multimodel(), using the first cross-validated model's metric as default metric.

See also

cv_performance

Examples

even_ind <- seq(150)%%2 == 0
mylm <- lm(Sepal.Length ~., iris[even_ind, ])  # using half of the data to train
performance(mylm)
#> --- Performance table ---
#> Metric: rmse
#>       train_rmse test_rmse
#> model    0.26229        NA
performance(mylm, newdata = iris[!even_ind, ]) # test set is complement of model data
#> --- Performance table ---
#> Metric: rmse
#>       train_rmse test_rmse
#> model    0.26229   0.34786