GitHub

API reference

ComputerAdaptiveTesting.Aggregators — Module

This module takes care of integrating and optimizing over the ability/difficulty space. It includes TrackedResponses, which can store cumulative results during a test.

source
ComputerAdaptiveTesting.Sim.CatLoop — Type
struct CatLoop
CatLoop(; rules=..., get_response=..., new_response_callback=...)
  • get_response::Any: The function (index, label) -> Int8` which obtains the testee's response for a given question, e.g. by prompting or simulation from data.
  • new_response_callback::Any: A callback called each time there is a new responses. If provided, it is passed (responses::TrackedResponses, terminating).

Configuration for a simulatable CAT.

source
ComputerAdaptiveTesting.Sim.auto_responder — Method
auto_responder(
    responses
) -> ComputerAdaptiveTesting.Sim.var"#auto_responder##0#auto_responder##1"

This function constructs a next item function which automatically responds according to responses.

source
ComputerAdaptiveTesting.Sim.run_cat — Method
run_cat(cat_config::CatLoop, item_bank::AbstractItemBank; ib_labels=nothing)

Run a given CatLoop cat_config on the given item_bank. If ib_labels is not given, default labels of the form <<item #$index>> are passed to the callback.

source
ComputerAdaptiveTesting.NextItemRules — Module

This module implements the next item selection rules, which form the main part of CAT.

Bibliography

[1] Linden, W. J., & Pashley, P. J. (2009). Item selection and ability estimation in adaptive testing. In Elements of adaptive testing (pp. 3-30). Springer, New York, NY.

source
ComputerAdaptiveTesting.NextItemRules.AbilityVariance — Type
struct AbilityVariance{DistEst<:ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator, IntegratorT<:ComputerAdaptiveTesting.Aggregators.AbilityIntegrator} <: ComputerAdaptiveTesting.NextItemRules.StateCriterion

  • dist_est::ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator

  • integrator::ComputerAdaptiveTesting.Aggregators.AbilityIntegrator

  • skip_zero::Bool

This StateCriterion returns the variance of the ability estimate given a set of responses.

source
ComputerAdaptiveTesting.NextItemRules.EmpiricalInformationPointwiseItemCategoryCriterion — Type

In equation 10 of [1] we see that we can compute information using 2nd derivatives of log likelihood or 1st derivative squared. For single categories, we need to an extra term which disappears when we calculate the total see [2]. For this reason RawEmpiricalInformationPointwiseItemCategoryCriterion computes without this factor, while EmpiricalInformationPointwiseItemCategoryCriterion computes with it.

So in general, only use the former with TotalItemInformation

[1] ``Information Functions of the Generalized Partial Credit Model'' Eiji Muraki https://doi.org/10.1177/014662169301700403

[2] https://mark.reid.name/blog/fisher-information-and-log-likelihood.html

source
ComputerAdaptiveTesting.NextItemRules.ExpectationBasedItemCriterion — Type
struct ExpectationBasedItemCriterion{ResponseExpectationT<:ComputerAdaptiveTesting.NextItemRules.ResponseExpectation, CriterionT<:Union{ComputerAdaptiveTesting.NextItemRules.ItemCategoryCriterion, ComputerAdaptiveTesting.NextItemRules.ItemCriterion, ComputerAdaptiveTesting.NextItemRules.StateCriterion}} <: ComputerAdaptiveTesting.NextItemRules.ItemCriterion

  • response_expectation::ComputerAdaptiveTesting.NextItemRules.ResponseExpectation

  • criterion::Union{ComputerAdaptiveTesting.NextItemRules.ItemCategoryCriterion, ComputerAdaptiveTesting.NextItemRules.ItemCriterion, ComputerAdaptiveTesting.NextItemRules.StateCriterion}

This ItemCriterion wraps a a ResponseExpectation and a StateCriterion or ItemCriterion to look at the criterion's expected value for a particular item 1-ply ahead.

source
ComputerAdaptiveTesting.NextItemRules.GreedyForcedContentBalancer — Type
struct GreedyForcedContentBalancer{InnerRuleT<:ComputerAdaptiveTesting.NextItemRules.NextItemRule} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule

  • targets::Vector{Float64}

  • groups::Vector{Int64}

  • inner_rule::ComputerAdaptiveTesting.NextItemRules.NextItemRule

This content balancing procedure takes target proportions for each group of items. At each step the group with the lowest ratio of seen items to target is selected.

http://dx.doi.org/10.1207/s15324818ame0403_4

source
ComputerAdaptiveTesting.NextItemRules.ItemCriterionRule — Type
struct ItemCriterionRule{NextItemStrategyT<:ComputerAdaptiveTesting.NextItemRules.NextItemStrategy, ItemCriterionT<:ComputerAdaptiveTesting.NextItemRules.ItemCriterion} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule

  • strategy::ComputerAdaptiveTesting.NextItemRules.NextItemStrategy

  • criterion::ComputerAdaptiveTesting.NextItemRules.ItemCriterion

ItemCriterionRule which together with a NextItemStrategy acts as an adapter by which an ItemCriterion can serve as a NextItemRule.

ItemCriterionRule(bits...; ability_estimator=nothing

Implicit constructor for ItemCriterionRule. Will default to ExhaustiveSearch when no NextItemStrategy is given.

source
ComputerAdaptiveTesting.NextItemRules.NextItemRule — Type
abstract type NextItemRule <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase

Abstract base type for all item selection rules. All descendants of this type are expected to implement the interface (::NextItemRule)(responses::TrackedResponses, items::AbstractItemBank)::Int.

In practice, all adaptive rules in this package use ItemCriterionRule.

NextItemRule(bits...; ability_estimator=nothing, parallel=true)

Implicit constructor for NextItemRule. Uses any given NextItemRule or delegates to ItemCriterionRule the default instance.

source
ComputerAdaptiveTesting.NextItemRules.NextItemStrategy — Type
abstract type NextItemStrategy <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase

Abstract type for next item strategies, tightly coupled with ItemCriterionRule. All descendants of this type are expected to implement the interface (rule::ItemCriterionRule{::NextItemStrategy, ::ItemCriterion})(responses::TrackedResponses, items) where {ItemCriterionT <: }(strategy::NextItemStrategy)(; parallel=true)::NextItemStrategy`

source
ComputerAdaptiveTesting.NextItemRules.PointResponseExpectation — Type
struct PointResponseExpectation{PointAbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator} <: ComputerAdaptiveTesting.NextItemRules.ResponseExpectation
  • ability_estimator::ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator

This ResponseExpectation gets expected outcomes based on a point ability estimates.

source
ComputerAdaptiveTesting.NextItemRules.RandomNextItemRule — Type
struct RandomNextItemRule{RandomT<:Random.AbstractRNG} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule
  • rng::Random.AbstractRNG: Default: Xoshiro()

This is the most basic rule for choosing the next item in a CAT. It simply picks a random item from the set of items that have not yet been administered.

source
ComputerAdaptiveTesting.NextItemRules.UrryItemCriterion — Type
struct UrryItemCriterion{AbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator} <: ComputerAdaptiveTesting.NextItemRules.ItemCriterion
  • ability_estimator::ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator

This item criterion just picks the item with the raw difficulty closest to the current ability estimate.

source
ComputerAdaptiveTesting.Rules.CatRules — Type
struct CatRules{NextItemRuleT<:ComputerAdaptiveTesting.NextItemRules.NextItemRule, TerminationConditionT<:ComputerAdaptiveTesting.TerminationConditions.TerminationCondition, AbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.AbilityEstimator, AbilityTrackerT<:ComputerAdaptiveTesting.Aggregators.AbilityTracker} <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase
  • next_item::ComputerAdaptiveTesting.NextItemRules.NextItemRule: The rule to choose the next item in the CAT given the current state.
  • termination_condition::ComputerAdaptiveTesting.TerminationConditions.TerminationCondition: The rule to choose when to terminate the CAT.
  • ability_estimator::ComputerAdaptiveTesting.Aggregators.AbilityEstimator: The ability estimator, which estimates the testee's current ability.
  • ability_tracker::ComputerAdaptiveTesting.Aggregators.AbilityTracker: The ability tracker, which tracks the testee's current ability level. Default: NullAbilityTracker()

Configuration of the rules for a CAT. This all includes all the basic rules for the CAT's operation, but not the item bank, nor any of the interactivity hooks needed to actually run the CAT.

This may be more a more convenient layer to integrate than CatLoop if you want to write your own CAT loop rather than using hooks.

CatRules(; next_item=..., termination_condition=..., ability_estimator=..., ability_tracker=...)

Explicit constructor for CatRules.

CatRules(bits...)

Implicit constructor for CatRules.

source