Evaluation of Bayesian confidence markers in reinforcement learning

Confidence in decision-making is a metacognitive process estimating the probability of correct choices and regulating the exploration-exploitation balance. The Bayesian Confidence Hypothesis (BCH) posits this follows probabilistic rules with three signatures: (1) calibration between certainty and accuracy, (2) differential confidence modulation based on discriminability (higher in correct responses, lower in errors), and (3) intermediate confidence under neutral evidence. Its validity in reinforcement learning remains scarcely explored. This pre-registered study evaluated these signatures in a two-armed bandit task (30 adults, 600 trials/subject). Participants chose options with Beta reward distributions, reported confidence (1–4 scale) before feedback, and earned 0–100 points. Analyses utilized mixed logistic regression, ANOVAs, and t-tests. Signature 1 was confirmed: confidence predicted performance (Odds Ratio=3.87, t(29)=9.98, p<.001). Signature 2 was partially confirmed: confidence increased in correct trials (η²G=0.311) but didn't decrease in errors (η²G=0.034), revealing overconfidence and valence biases. Signature 3 was unconfirmed: participants showed underconfidence lacking information (M=0.39 vs. 0.5, p=.045), suggesting adaptive early exploration. Confidence acts as a dynamic metacognitive marker in reinforcement learning, with systematic deviations from normative models.