calc_stats provides an interface for calculating statistics/metrics on
model predictions and/or observed data. Supported statistics include
Conditional Accuracy Functions (CAFs), Quantiles, Delta Functions, and Fit
Statistics. Results can be aggregated across individuals.
Usage
calc_stats(object, type, ...)
# S3 method for class 'data.frame'
calc_stats(
object,
type,
...,
conds = NULL,
verbose = 0,
average = FALSE,
split_by_ID = TRUE,
b_coding = NULL
)
# S3 method for class 'drift_dm'
calc_stats(object, type, ..., conds = NULL)
# S3 method for class 'fits_ids_dm'
calc_stats(object, type, ..., verbose = 1, average = FALSE)Arguments
- object
an object for which statistics are calculated. This can be a data.frame of observed data, a drift_dm object, or a
fits_ids_dmobject (see estimate_model_ids).- type
a character vector, specifying the statistics to calculate. Supported values include
"cafs","quantiles","delta_funs", and"fit_stats".- ...
additional arguments passed to the respective method and the underlying calculation functions (see Details for mandatory arguments).
- conds
optional character vector specifying conditions to include. Conditions must match those found in the
object.- verbose
integer, indicating if information about the progress should be displayed. 0 -> no information, 1 -> a progress bar. Default is 0.
- average
logical. If
TRUE, averages the statistics across individuals where applicable. Default isFALSE.- split_by_ID
logical. If
TRUE, statistics are calculated separately for each individual ID inobject(whenobjectis a data.frame). Default isTRUE.- b_coding
a list for boundary coding (see b_coding). Only relevant when
objectis a data.frame. For otherobjecttypes, theb_codingof theObjectis used.
Value
If type is a single character string, then a data.frame is returned.
If type contains multiple character strings (i.e., is a character vector)
a list with the calculated statistics (with entries being data.frames) is
returned.
Each returned data.frame has a certain class label and may store additional
attributes required for the custom plot() functions. If a list is returned,
then that list will have the class label list_stats_dm (to easily create
multiple panels using the respective plot() method).
Details
calc_stats is a generic function to handle the calculation of different
statistics/metrics for the supported object types. Per default, it returns
the requested statistics/metrics.
Conditional Accuracy Function (CAFs)
CAFs are a way to quantify response accuracy against speed. To calculate CAFs, RTs (whether correct or incorrect) are first binned and then the percent correct responses per bin is calculated.
When calculating model-based CAFs, a joint CDF combining both the pdf of correct and incorrect responses is calculated. Afterwards, this CDF is separated into even-spaced segments and the contribution of the pdf associated with a correct response relative to the joint CDF is calculated.
The number of bins can be controlled by passing the argument n_bins.
The default is 5.
Quantiles
For observed response times, the function stats::quantile is used with default settings.
Which quantiles are calcuated can be controlled by providing the
probabilites, probs, with values in \([0, 1]\). Default is
seq(0.1, 0.9, 0.1).
Delta Functions
Delta functions calculate the difference between quantiles of two conditions against their mean:
\(Delta_i = Q_{i,j} - Q_{i,k}\)
\(Avg_i = 0.5 \cdot Q_{i,j} + 0.5 \cdot Q_{i,k}\)
With i indicating a quantile, and j and k two conditions.
To calculate delta functions, users have to specify:
minuends: character vector, specifying condition(s) j. Must be inconds(drift_dm_obj).subtrahends: character vector, specifying condition(s) k. Must be inconds(drift_dm_obj)dvs: character, indicating which quantile columns to use. Default is "Quant_<u_label>". If multiple dvs are provided, then minuends and subtrahends must have the same length, and matching occurs pairwise. In this case, if only one minuend/subtrahend is specified, minuend and subtrahend are recycled to the necessary length.
Fit Statistics
Calculates the Akaike and Bayesian Information Criteria (AIC and BIC). Users
can provide a k argument to penalize the AIC statistic (see stats::AIC
and AIC.fits_ids_dm)
Examples
# Example 1: Calculate CAFs and Quantiles from a model ---------------------
# get a model for demonstration purpose
a_model <- ssp_dm(dx = .0025, dt = .0025, t_max = 2)
# and then calculate cafs and quantiles
some_stats <- calc_stats(a_model, type = c("cafs", "quantiles"))
head(some_stats$cafs)
#> Source Cond Bin P_corr
#> 1 pred comp 1 0.9812941
#> 2 pred comp 2 0.9812941
#> 3 pred comp 3 0.9812941
#> 4 pred comp 4 0.9812941
#> 5 pred comp 5 0.9812941
#> 6 pred incomp 1 0.6718931
head(some_stats$quantiles)
#> Source Cond Prob Quant_corr Quant_err
#> 1 pred comp 0.1 0.3626594 0.3626593
#> 2 pred comp 0.2 0.3824975 0.3824974
#> 3 pred comp 0.3 0.4011229 0.4011229
#> 4 pred comp 0.4 0.4206904 0.4206904
#> 5 pred comp 0.5 0.4426163 0.4426162
#> 6 pred comp 0.6 0.4686185 0.4686185
# Example 2: Calculate a Delta Function from a data.frame ------------------
# get a data set for demonstration purpose
some_data <- ulrich_simon_data
conds(some_data) # relevant for minuends and subtrahends
#> [1] "incomp" "comp"
some_stats <- calc_stats(
a_model,
type = "delta_funs",
minuends = "incomp",
subtrahends = "comp"
)
head(some_stats)
#> Source Prob Quant_corr_comp Quant_corr_incomp Delta_incomp_comp
#> 1 pred 0.1 0.3626594 0.4079033 0.04524393
#> 2 pred 0.2 0.3824975 0.4361191 0.05362164
#> 3 pred 0.3 0.4011229 0.4608977 0.05977485
#> 4 pred 0.4 0.4206904 0.4856208 0.06493038
#> 5 pred 0.5 0.4426163 0.5120811 0.06946478
#> 6 pred 0.6 0.4686185 0.5421471 0.07352853
#> Avg_incomp_comp
#> 1 0.3852813
#> 2 0.4093083
#> 3 0.4310103
#> 4 0.4531556
#> 5 0.4773487
#> 6 0.5053828
# Example 3: Calculate Quantiles from a fits_ids_dm object -----------------
# get an auxiliary fits_ids_dm object
all_fits <- get_example_fits_ids()
some_stats <- calc_stats(all_fits, type = "quantiles")
head(some_stats) # note the ID column
#> ID Source Cond Prob Quant_corr Quant_err
#> 1 1 obs comp 0.1 0.3346540 0.3612974
#> 2 1 obs comp 0.2 0.3680154 0.3879847
#> 3 1 obs comp 0.3 0.3846198 0.4146720
#> 4 1 obs comp 0.4 0.3847419 0.4413593
#> 5 1 obs comp 0.5 0.4013780 0.4680466
#> 6 1 obs comp 0.6 0.4180689 0.4680640
# one can also request that the statistics are averaged across individuals
head(
calc_stats(all_fits, type = "quantiles", average = TRUE)
)
#> Source Cond Prob Quant_corr Quant_err
#> 1 obs comp 0.1 0.3235133 0.3112677
#> 2 obs comp 0.2 0.3457705 0.3296183
#> 3 obs comp 0.3 0.3624091 0.3479690
#> 4 obs comp 0.4 0.3680182 0.3646525
#> 5 obs comp 0.5 0.3846893 0.3805026
#> 6 obs comp 0.6 0.3958161 0.3830177