Analyzing randomized controlled interventions: Three notes for applied linguists
Loading...
Date
2015-03
Authors
Advisor
Editor
Journal Title
Journal ISSN
Volume Title
Publisher
Zakład Filologii Angielskiej Wydział Pedagogiczno-Artystyczny Uniwersytetu im. Adama Mickiewicza w Kaliszu
Title alternative
Abstract
I discuss three common practices that obfuscate or invalidate the statistical analysis of randomized controlled interventions in applied linguistics. These are (a) checking whether randomization produced groups that are balanced on a number of possibly relevant covariates, (b) using repeated measures ANOVA to analyze pretest-posttest designs, and (c) using traditional significance tests to analyze interventions in which whole groups were assigned to the conditions (cluster randomization). The first practice is labeled superfluous, and taking full advantage of important covariates regardless of balance is recommended. The second is needlessly complicated, and
analysis of covariance is recommended as a more powerful alternative. The third
produces dramatic inferential errors, which are largely, though not entirely, avoided
when mixed-effects modeling is used. This discussion is geared towards applied linguists who need to design, analyze, or assess intervention studies or other randomized controlled trials. Statistical formalism is kept to a minimum throughout.
Description
Sponsor
Keywords
randomized experiments, cluster randomization, pretest-posttest de- signs, covariates, mixed-effects modeling
Citation
Studies in Second Language Learning and Teaching, 2015, vol. 5, no. 1, pp.135-152