MZES Social Science Data Lab

Abstract:

Large language models (LLMs) have the potential to make survey research more efficient, including the classification of open-ended survey responses. However, as most existing research on this topic has focused on English-language text or single LLMs, it is unclear whether their applicability generalizes and how the quality of classifications compares to established methods. In this talk, I will demonstrate how LLMs can be used for coding open-ended responses using different access options and prompting and fine-tuning techniques. I will present a study testing these approaches on a dataset of German open-ended survey responses, comparing several LLMs to human coders and other automated methods. Finally, I will discuss the implications of the study findings for practitioners, including the many trade-offs researchers need to consider.

Register for the live stream on Zoom: https://us02web.zoom.us/meeting/register/gPP40TSfTCWR-fut_KgH-Q#/registration