This book shows how to conduct statistical power analysis when the underlying research data are unreliable and explores how sample design effects, measurement error, and equating error impact power analysis. As such, it covers unreliability in test scores and in the measurement scale, and incorporates both the sources of sampling error, such as design effects in survey sampling, and measurement error, such as unreliability and equating error in psychometrics, in power calculations. Five widely used statistics are treated, including the mean for one group, mean difference for two independent groups, mean difference for two dependent groups, proportion, and correlation. Each chapter develops procedures for a simple and complex random sample of true scores or units, and a complex random sample of observed scores or units without and with equating error. Analogously to design effects due to complex random sampling, the reader is introduced to indices of measurement effects, which reflect the impact of measurement error and equating error on the standard error of the statistic. The book includes numerous worked out examples and is aimed at practitioners, researchers and students in psychometrics and other social and behavioral sciences.
