Mannheim Research Colloquium on Survey Methods

Acquiescent response style (ARS) is the tendency to disproportionately agree with survey items and can impair survey measurement. ARS has been linked to Agree/Disagree (A/D) rating scales, which are broadly used in the social sciences. This has made ARS a vexing problem in the measurement of social constructs. Despite extensive research on the topic, there is little consensus on how to mitigate the effects of ARS. This study proposed that the order of A/D responses may be used to deter ARS. The extant literature shows how primacy effects in non-aural surveys increase the rate at which respondents select the first listed response category. Following this logic, listing agreement categories first may exacerbate ARS, particularly for Web or paper-and-pencil surveys. The aim of this study was to evaluate whether the response order of A/D rating scales impacts use of ARS.

Participants from three groups were recruited into a Web survey: non-Hispanic whites (n=1,200), Hispanics in the U.S. (n=1,200) and Hispanics in Mexico (n=1,200). An experiment was conducted using three measurement scales assessing emotional expressivity, affective orientation, and purpose in life. Respondents were randomly assigned to one of two response orders: (1) disagreement to agreement, placing agreement categories last or (2) agreement to disagreement scale order, placing agreement categories first. Respondents chose agreement responses significantly less often when the disagreement response options were presented first, than when the agreement responses were offered first. This response order similarly reduced the number of simultaneous agreeable responses to opposite scale items. When the disagreement options were placed first, reliability and convergent validity of the scales also improved. This study shows that putting the agreement categories last in the response scale may be a useful design option for addressing ARS in Web surveys.

Please use this link to attend the colloquium via Zoom: https://us02web.zoom.us/j/86829349377?pwd=OGdwMnV1d0hzMGRmNnhWaEdsc3VyQT09 .