Chapter 3 – How precise are our estimates? Confidence intervals

To assess the precision of an estimate, compute its confidence interval. Use confidence intervals around all point estimates to understand the plausible range of the unknown population mean or proportion. Computing a confidence interval requires four things: an estimate of the mean, an estimate of the variability (derived from the sample standard deviation), the desired confidence level (typically 95%), and the sample size. Use the adjusted-Wald binomial confidence interval for binomial metrics such as completion rates. For satisfaction data using rating scales use the confidence intervals based on the t-distribution (which takes the sample size into account). The geometric mean is the best estimate of the middle task time from small sample sizes (<25). Because task-time data is positively skewed, use a log transformation before computing confidence intervals based on the t-distribution. For larger samples of task-time data (≥25), the median is the best point estimate of the middle task time, so compute the confidence interval around the median using the binomial distribution method.