Parameter-Efficient Hate Speech Detection: A Comprehensive Evaluation of LoRA-Adapted LLMs Across 18 Architectures

Abstract

The proliferation of hate speech on social media platforms necessitates automated detection systems that balance accuracy with computational efficiency. This study evaluates 18 LoRA-adapted Large Language Models (LLMs) across three model families (Llama, Phi, Qwen) at scales ranging from 0.5B to 14B parameters for hate speech detection. Using a dataset of 48,049 samples, we investigate the relationship between model architecture, size, instruction tuning, and detection performance while focusing on parameter-efficient fine-tuning approaches. Our results demonstrate that Phi-4 achieves state-of-the-art performance of 91.84% accuracy and 92.79% F1-score despite using only 0.15% of its parameters for task-specific adaptation. Smaller models like Llama-3.2-1B show competitive performance (91.53% accuracy, 92.69% F1-score) with significantly faster training times. We find that architectural design often outweighs parameter count, with smaller models frequently outperforming larger counterparts within the same family. The inconsistent benefits of instruction tuning across architectures highlight the importance of task-specific adaptation for hate speech detection. These findings offer valuable insights for developing efficient and effective hate speech detection systems across diverse deployment scenarios, from high-performance servers to resource-constrained environments.

Department(s)

Computer Science

Document Type

Conference Proceeding

DOI

10.18293/SEKE2025-036

Keywords

hate speech detection, Llama, LLMs, low-rank adaptation, natural language processing, parameter-efficient fine-tuning, Phi, Qwen, text classification, transformer models

Publication Date

1-1-2025

Journal Title

Proceedings of the International Conference on Software Engineering and Knowledge Engineering Seke

Link to Full Text

Share

COinS