Optimized Fully Sharded Data Parallel for Efficient Distributed Training of BiLSTM With Multi-Head Attention for Network Intrusion Detection Systems

Noor Al-Huda H. Oleiwi, Department Information Networks, College of Information Technology, University of Babylon, Babil, IraqFollow
Ameer K. Hadi, Department Information Networks, College of Information Technology, University of Babylon, Babil, IraqFollow

ORCID

Noor Al-Huda H. Oleiwi: https://orcid.org/0009-0005-5478-7122

Article Type

Original Study

Abstract

The expanding volume of network traffic, the growing size of cybersecurity datasets, and the increasing complexity of deep learning models present major computational challenges to training deep learning-based Network Intrusion Detection Systems (NIDSs). Fully Sharded Data Parallel (FSDP) enables distributed training by sharding model parameters across multiple GPUs; however, the synchronization and communication overhead can limit its training efficiency. To overcome these limitations, this paper proposes an Optimized Fully Sharded Data Parallel (Optimized FSDP) framework that improves distributed training efficiency by dynamic gradient accumulation and synchronization scheduling. The proposed framework is integrated with a Bidirectional Long Short-Term Memory (BiLSTM) network with Multi-Head Attention (MHA) for multiclass network intrusion detection. The proposed approach is evaluated on the CICIDS2017 and TRUSTLab2026 datasets for two training configurations: single-GPU and distributed training. The distributed training configurations include Standard FSDP and the proposed Optimized FSDP with two and four GPUs. The results show that the proposed optimized FSDP can reach high intrusion detection accuracy with a significant reduction in the training time. For CICIDS2017, the training time was decreased from 75.95 minutes using the Single-GPU configuration to 33.79 minutes using Optimized FSDP with two GPUs and 16.76 minutes using four GPUs, achieving a maximum speedup of 4.53× . For TRUSTLab2026, training time dropped from 108.80 minutes to 40.86 and 28.97 minutes with two and four GPUs, respectively, yielding a maximum speedup of 3.76× . The intrusion detection accuracies achieved were 99.43% for CICIDS2017 and 96.19% for TRUSTLab2026. The results show that the proposed training strategy can improve the scalability and efficiency of deep learning-based NIDSs with multi-GPU training while achieving high classification performance.

Keywords

Deep learning, Training time, FSDP, BiLSTM, Distributed training, IDS

How to Cite This Article

Oleiwi, Noor Al-Huda H. and Hadi, Ameer K. (2026) "Optimized Fully Sharded Data Parallel for Efficient Distributed Training of BiLSTM With Multi-Head Attention for Network Intrusion Detection Systems," Journal of Intelligent Informatics, Networking, and Cybersecurity: Vol. 2 : Iss. 2 , Article 12.
Available at: https://doi.org/10.65445/3106-1192.1023