TUTCRIS - Tampereen teknillinen yliopisto


Diversification of system calls in linux binaries



OtsikkoTrusted Systems - 6th International Conference, INTRUST 2014, Revised Selected Papers
KustantajaSpringer Verlag
ISBN (painettu)9783319279978
DOI - pysyväislinkit
TilaJulkaistu - 2015
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
Tapahtuma6th International Conference on Trusted Systems, INTRUST 2014 - Beijing, Kiina
Kesto: 16 joulukuuta 201417 joulukuuta 2014


NimiLecture Notes in Computer Science
ISSN (painettu)0302-9743
ISSN (elektroninen)1611-3349


Conference6th International Conference on Trusted Systems, INTRUST 2014


This paper studies the idea of using large-scale diversification to protect operating systems and make malware ineffective. The idea is to first diversify the system call interface on a specific computer so that it becomes very challenging for a piece of malware to access resources, and to combine this with the recursive diversification of system library routines indirectly invoking system calls. Because of this unique diversification (i.e. a unique mapping of system call numbers), a large group of computers would have the same functionality but differently diversified software layers and user applications. A malicious program now becomes incompatible with its environment. The basic flaw of operating system monoculture - the vulnerability of all software to the same attacks - would be fixed this way. Specifically, we analyze the presence of system calls in the ELF binaries. We study the locations of system calls in the software layers of Linux and examine how many binaries in the whole system use system calls. Additionally, we discuss the different ways system calls are coded in ELF binaries and the challenges this causes for the diversification process. Also, we present a diversification tool and suggest several solutions to overcome the difficulties faced in system call diversification. The amount of problematic system calls is small, and our diversification tool manages to diversify the clear majority of system calls present in standard-like Linux configurations. For diversifying all the remaining system calls, we consider several possible approaches.