The malware detection mechanisms, based on tracing the user space activities, can be easily bypassed by the kernel level rootkits. This thesis proposes a solution to detect attack techniques used by this type of rootkits using the tracing of linux kernel activities with the LTTng tool. This solution is based on the analysis of static and dynamic data of the linux kernel. It is designed along two axes: the first one is the integration of rootkit detection techniques in the kernel modules of the LTTng tracer and the second one is the use of the generated traces of LTTng containing the kernel data in the detection approach based on machine learning technique. The second axis has undergone optimization of input vectors corresponding to the different values of attacks and still underwent a paired t-test for selecting the best machine learning classifier. The validation of this solution was made on a test environment specifically designed for this type of rootkits.
Slaimia, T. (Author),
Gherbi (Supervisor) &
Talhi (Co-supervisor),
6 Aug 2015Student thesis: Master's thesis › Master in Engineering: Engineering