BERT-OTA: Enhancing Hate Speech Detection With Ontology-Guided Transformer Attention

Abstract

The proliferation of hate speech on social media platforms presents a significant challenge for content moderation, requiring sophisticated detection methods that can understand both explicit and implicit forms of harmful content. Traditional approaches struggle with contextual nuances and evolving patterns of online hate speech. We propose a novel architecture BERT-OTA (Ontology Transformer with Attention) that integrates ontological knowledge with transformer attention mechanisms to address these limitations. Our approach employs a dual-stream architecture: processing text through BERT with scaled dot-product attention while simultaneously learning ontological features through a two-layer Graph Convolutional Network. This design enables the model to leverage both contextual understanding and structured domain knowledge about hate speech categories and their relationships. Through experiments on a combined dataset of 48,049 samples from three established collections (Davidson, Impermium, and Waseem & Hovy), we evaluate BERT-OTA against seven distinct architectures, including BERT-based hybrid models and graph neural networks (GNN). Our results demonstrate that BERT-OTA achieves state-of-the-art performance with 91.30% accuracy and 91.32% F1-score, outperforming all other models. The performance improvements of our ontology-guided approach highlight the potential of structured knowledge integration in hate speech detection, offering valuable insights for developing robust online content moderation systems.

Department(s)

Computer Science

Document Type

Article

DOI

10.1109/ACCESS.2026.3650874

Keywords

BERT, graph neural networks, hate speech detection, ontology integration, transformer models

Publication Date

1-1-2026

Journal Title

IEEE Access

Link to Full Text

Share

COinS