Malicious Behaviour Analysis Using Vanilla Transformers in Deep Learning

Abstract: Malicious behaviour analysis is a critical aspect of cybersecurity aimed at identifying harmful activities such as data exfiltration, privilege escalation, and system exploitation. Traditional methods often rely on predefined signatures or shallow heuristics, which limit their ability to detect evolving or previously unseen threats. To address these limitations, this study employs a deep learning-based approach utilising Vanilla Transformers, a model architecture renowned for its powerful attention mechanisms and ability to capture complex dependencies in sequential data. Unlike recurrent architectures, Vanilla Transformers process entire sequences in parallel, enabling faster computation and more effective learning of behavioural patterns. The model demonstrated strong performance, achieving 99.89% accuracy, 100% recall, 100% precision, and an F1-score of 100%, indicating its effectiveness in identifying malicious behaviours with minimal false positives. This research highlights the potential of attention-based architectures in cybersecurity, providing a scalable and adaptive solution for real-time threat detection and behavioural analysis in complex digital environments.

Keywords: Malicious Behaviour Detection, Network Intrusion Detection System (NIDS), Vanilla Transformers, Deep Learning, Cybersecurity, Wireshark, CiscoFlow Meter, Real-Time Network Monitoring