AI-Assisted Qualitative Coding: Comparing Human to Machine Outputs
Abstract
This study explores the application of generative AI in the qualitative coding of survey responses, comparing its performance to that of human coders. By using ChatGPT-4o, we aimed to automate the coding process traditionally performed manually, assessing the AI's ability to identify themes and patterns within textual data. Our findings reveal that while AI demonstrates remarkable efficiency and speed, it struggles with the nuanced understanding required for complex coding tasks. The AI frequently misinterpreted coding definitions and over-relied on certain codes, indicating a need for more balanced training data and iterative refinement. Despite these challenges, AI proved valuable in providing summative feedback and identifying overall trends, suggesting its potential as a complementary tool in qualitative research. The study underscores the importance of developing codebooks collaboratively with AI and highlights the necessity of human oversight to ensure accuracy and depth.
Department(s)
English
Document Type
Conference Proceeding
DOI
10.1109/ProComm64814.2025.00051
Keywords
Generative AI, human vs. AI, qualitative data coding, research methodology, survey responses
Publication Date
1-1-2025
Recommended Citation
Seawright, Leslie E.; Hodges, Amy; and Ponce, Timothy, "AI-Assisted Qualitative Coding: Comparing Human to Machine Outputs" (2025). Faculty Scholarship. 255.
https://bearworks.missouristate.edu/articles00/255
Journal Title
IEEE International Professional Communication Conference
