Deep Learning for Classification of Malware System Call Sequences

Bojan Kolosnjaji, Apostolis Zarras, George Webster, Claudia Eckert

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review


The increase in number and variety of malware samples amplifies the need for improvement in automatic detection and classification of the malware variants. Machine learning is a natural choice to cope with this increase, because it addresses the need of discovering underlying patterns in large-scale datasets. Nowadays, neural network methodology has been grown to the state that can surpass limitations of previous machine learning methods, such as hidden markov models and support vector machines. As a consequence, neural networks can now offer superior classification accuracy in many domains, such as computer vision or natural language processing. This improvement comes from the possibility of constructing neural networks with a higher number of potentially diverse layers and is known as deep this paper, we attempt to transfer these performance improvements to model the malware system call sequences for the purpose of malware classification. We construct a neural network based on convolutional and recurrent network layers in order to obtain the best features for classification. This way we get a hierarchical feature extraction architecture that combines convolution of n-grams with full sequential modeling. Our evaluation results demonstrate that our approach outperforms previously used methods in malware classification, being able to achieve an average of 85.6% on precision and 89.4% on recall using this combined neural network architecture.keywordssystem call sequencesmalware samplesmalware classificationmalware detectionrecurrent layerthese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Original languageEnglish
Title of host publicationProceedings of the 29th Australasian Joint Conference on Artificial Intelligence (AI)
Publication statusPublished - 2016
Externally publishedYes

Cite this