Log-Based Anomaly Detection through Correlation & Behavior Analysis for Cybersecurity 6a.010.DU

Project Start Date: Jul 1, 2017
Research Areas: Analytics, Analytics - Probabilistic Modeling, Data Management, Data Management -Security, Visualization, Visualization - Visual Analytics
Funding: Member Funded
Project Tags: ,

Project Summary

Anomaly correlation analysis for cybersecurity aims to discover correlated cyber activity patterns that exhibit notable departure from common patterns. For example, botnet is a distributed software that runs coordinated programs over target websites to perform malicious tasks like skewing website statistics, price scraping, spam distribution, DOS attack, etc. Discovering bot activities can help prevent significant economic losses for many enterprises that rely their business on websites. Although log data have been commonly leveraged for cyber anomaly detection, current methods are typically batch-based and not able to perform real-time detection on large-volume streaming data. Moreover, low-level or hardware-related information is usually part of the analysis, where such information is sometimes sensitive and needs additional effort to collect and pre-process. To overcome these limitations, our project aims to formulate general methods that discover anomalies based on application-level logs, such like Apache logs for website servers. The application-level logs are provided by applications that host certain network services, usually nearly structured and readily available for data analysis. The objectives of this project mainly include: 1) design novel methods to discover correlated anomalies from large-volume streaming log data; 2) design methods to understand the purpose of those anomalies, providing rich information for better management decision.


Principal Investigator(s)