Effect Sizes and Statistical Methods for Meta-Analysis in Higher Education

Quantitative meta-analysis is a very useful, yet underutilized, technique for synthesizing research findings in higher education.
Meta-analytic inquiry can be more challenging in higher education than in other fields of study as a result of (a) concerns
about the use of regression coefficients as a metric for comparing the magnitude of effects across studies, and (b) the non-independence
of observations that occurs when a single study contains multiple effect sizes. This methodological note discusses these two
important issues and provides concrete suggestions for addressing them. First, meta-analysis scholars have concluded that
standardized regression coefficients, which are often provided in higher education manuscripts, constitute an appropriate
metric of effect size. Second, hierarchical linear modeling (HLM) analyses provide an effective method for conducting meta-analytic
research while accounting for the non-independence of observations, and HLM is generally superior to other proposed methods
that attempt to remedy this same problem. A discussion of how to implement these techniques appropriately is provided.