Biases in Bias Elicitation

We consider the biases that can arise in bias elicitation when expert assessors make random errors. We illustrate the phenomenon for two sources of bias: that due to omitting important variables in a least squares regression and that which arises in adjusting relative risks for treatment effects using an elicitation scale. Results show that, even when assessors' elicitations of bias have desirable properties (such as unbiasedness and independence), the nonlinear nature of biases can lead to elicitations of bias that are, themselves, biased. We show the corrections which can be made to remove this bias and discuss the implications for the applied literature which employs these methods.