Using Randomization to Learn About Framing Effects on LGBTQ Rights Questions

In a survey that included the same question about refusing service to gay and lesbian people due to religious beliefs in two different batteries, some respondents offered different answers. We probe the framing issues inherent in each question battery and find that support for refusing service to gay and lesbian people declines when the question is in the context of other group service refusals that could be considered civil rights issues. Additionally, inconsistent responses are also related to question order. We posit that randomization can lead to what appear to be fluctuations in attitudes but may, in fact, also be capturing framing effects. This has implications for both the survey design process and substantive findings.