Attention Mapping for Hallucination Reduction in Arabic Financial Using Retrieval-Augmented Generation Systems

Hasan Abdulameer Hasan, Department of Cybersecurity, College of Information Technology, University of Babylon, Hillah, Babylon, 51002, IraqFollow
Khaldoun H. Al-Hussayni, Department of Cybersecurity, College of Information Technology, University of Babylon, Hillah, Babylon, 51002, IraqFollow
Ali Z. K. Matloob, Department of Cybersecurity, College of Information Technology, University of Babylon, Hillah, Babylon, 51002, IraqFollow

ORCID

Hasan Abdulameer Hasan: https://orcid.org/0009-0007-6581-1970

Khaldoun H. Al-Hussayni: https://orcid.org/0000-0002-6585-4301

Ali Z. K. Matloob: https://orcid.org/0000-0002-6158-4439

Article Type

Original Study

Abstract

This research introduces a customized attention visualization framework for mitigating hallucinations in Arabic Retrieval-Augmented Generation (RAG) systems tailored for financial document analysis. The proposed architecture extends MarBERT with a dual-stage attention supervision mechanism and a hallucination-aware loss formulation, trained on a newly constructed dataset of 7,000 annotated Arabic financial query-context pairs. A grounding alignment score is computed over attended tokens, and generated responses are rejected when falling below a dynamically adjusted precision-aware threshold,serving as the core decision-making approach for hallucination detection. The system achieves 95.04% classification accuracy, 95.84% precision, 94.12% recall, and an F1 score of 94.97%, outperforming AraELECTRA and mBERT-RAG baselines. For hallucination detection, it reaches an 87.63% detection rate with only 8.91% false alarms and a detection latency of 3.2 tokens, showcasing reliability in identifying ungrounded financial claims. Visualizations using cross-attention matrices and token contribution scores reveal consistent alignment with critical Arabic financial terms, temporal expressions, and entity references. An interactive attention interface further empowers non-technical users to identify hallucinations with 83.4% success in usability tests. These results validate the effectiveness of integrating interpretable attention modeling with domain-specific retrieval, offering a scalable approach for risk-sensitive natural language processing in low-resource languages.

Keywords

Arabic NLP, Retrieval-augmented generation, Attention visualization, Hallucination mitigation, Financial

How to Cite This Article

Hasan, Hasan Abdulameer; Al-Hussayni, Khaldoun H.; and Matloob, Ali Z. K. (2025) "Attention Mapping for Hallucination Reduction in Arabic Financial Using Retrieval-Augmented Generation Systems," Journal of Intelligent Informatics, Networking, and Cybersecurity: Vol. 1: Iss. 2, Article 2.
Available at: https://jiinc.uobabylon.edu.iq/journal/vol1/iss2/2