Machine Learning for Computer Security

Malicious software poses a severe threat to security of computer systems. Whether you download a file, plug in the USB stick of a colleague or simply surf the Web, your computer is always at risk to be compromised by malicious software, such as computer worms, backdoors or Trojan horses. Once the evil has infiltrated your system, it usually initiates a process referred to as "phoning home": The malicious software contacts its author and hands over control of your computer to him, for example for sending spam messages or conducting a distributed flooding attack. Unfortunately, regular security tools, such as anti-virus scanners, increasingly fail to protect the many infection vectors of malicious software and thus users are often left alone with systems "phoning home" to bad people.

In our latest research (to be published at the 25th ACM Symposium on Applied Computing) we address this problem and introduce Botzilla, a method for automatically detecting the "phoning home" of malicious software. Botzilla operates by first collecting malicious software in the wild using honeypots. The malicious software is then repetitively executed in a controlled environment and its communication is recorded— similar to a rat in a lab. Invariants communication patterns, such as byte strings used for handshaking and remote control, are extracted and assembled to detection signatures using a naive-Bayes classification scheme. As a result, Botzilla is able to automatically generate signatures for malicious software within minutes and allows to counteract the propagation of evil in the first round. An abstract for this work is here:

Hosts infected with malicious software, so called malware, are ubiquitous in today's computer networks. The means whereby malware can infiltrate a network are manifold and range from exploiting of software vulnerabilities to tricking a user into executing malicious code. Monitoring and detection of all possible infection vectors is intractable in practice. Hence, we approach the problem of detecting malicious software at a later point when it initiates contact with its maintainer; a process referred to as "phoning home". In particular, we introduce Botzilla, a method for detection of malware communication, which proceeds by repetitively recording network traffic of malware in a controlled environment and generating network signatures from invariant content patterns. Experiments conducted at a large university network demonstrate the ability of Botzilla to accurately identify malware communication in network traffic with very low false-positive rates.

Botzilla: Detecting the "Phoning Home" of Malicious Software. Konrad Rieck, Guido Schwenk, Tobias Limmer, Thorsten Holz and Pavel Laskov. Proceedings of 25th ACM Symposium on Applied Computing (SAC).

The work on Botzilla is a small yet successful effort of Berlin Institute of Technology, Fraunhofer FIRST, University of Erlangen, Technical University Vienna and University of Tübingen.