Abstract
This article revisits the concepts of reliability and measurement precision across classical test theory (CTT), item response theory (IRT) and cognitive diagnosis models (CDMs), emphasizing their conceptual differences and proposing a unified framework. We show how continuous score estimators in CTT and IRT relate to mastery probabilities in CDMs through integration over proficiency estimator distributions relative to defined thresholds. Building on this link, we introduce reliability indices grounded in the coefficient of determination (R2$$ {R}^2 $$) as a common measure of association between true scores and estimated true scores, applicable to both continuous proficiencies and discrete classifications. This unified perspective clarifies how reliability can be consistently estimated and reported across frameworks, promoting coherence in psychometric practice and supporting more transparent interpretation of test results. Ultimately, this study aims to clarify reliability concepts, facilitating a unified perspective and aiming to promote consistent reliability reporting across diverse psychometric applications.