Docstoc

Intrusion Detection Alert Correlation

Document Sample
Intrusion Detection Alert Correlation Powered By Docstoc
					Intrusion Detection
 Alert Correlation


       Mark Shaneck
        2/11/2005

                  1
                Outline
 Problem Statement
 Different Correlation Approaches
 A Comprehensive Approach
 Good News and Bad News
 A Better Approach?



                                     2
       What’s The Problem?
 Large organizations get tons of alerts
 Possibly up to 20,000 per day!
 Many false alarms




                                           3
                      Also…
 Alerts can come from many different sources
   – Signature based IDS (Snort)
   – File System Integrity Checkers
   – System Call Traces
 Alerts may represent multiple stages in one attack
 Hard to make sense out of a large pile of alerts!



                                                       4
 So What Is Alert Correlation?
 3 general categories
  – Alert Clustering
  – Matching Predefined Attack Scenarios
  – Prerequisites/Consequences




                                           5
           Alert Clustering
 Main Sources:
  – A. Valdes, K. Skinner, “Probabilistic Alert
    Correlation”, RAID 2001
  – O. Dain, R. Cunningham, “Building Scenarios
    from a Heterogeneous Alert Stream”, IEEE
    Workshop on Information Assurance and
    Security, 2001


                                                  6
              General Idea
 Join alerts together in some meaningful
  groups
 Group alerts into attack threads - one
  thread contains all alerts related to one
  attack
 For a new alert, compare to all alert threads
  – Join to the closest match
  – Or start new thread if none match

                                                  7
            Similarity Measure
 Feature Overlap - only consider features present
  in both (source, target, ports, attack class,
  timestamps, etc.)
 Each feature has a similarity measure
   – How much do port lists overlap?
   – Is one port contained within another’s list? (target port
     was previously scanned)
   – Are the IPs from the same subnet?
   – Attack classes have a similarity matrix

                                                                 8
        Similarity Expectation
 Different levels of similarity are expected
  for different features in different situations
   – SYN FLOOD with source spoofed
      • Expectation of similarity for source IP is 0
   – Scanning port(s)
      • Expectation of target IP is low (but not 0 - since it
        usually scans the subnet)



                                                                9
          Minimum Similarity
 Threshold for similarity measure
 Similarity is 0 if not above the minimum

 Adjusting thresholds
   – Synthetic Threads
      • high for sensor id, IPs
   – Security Incidents
      • low for sensor id, high for attack class
      • fuse alerts from multiple sources
   – Multistep attack detection
      • low for attack class
                                                   10
 So What Is Alert Correlation?
 3 general categories
  – Alert Clustering
  – Matching Predefined Attack Scenarios
  – Prerequisites/Consequences




                                           11
  Matching Predefined Attack
          Scenarios
 Main sources
  – H. Debar, A. Wespi, “Aggregation and
    Correlation of Intrusion-Detection Alerts”,
    RAID 2001
  – B. Morin, H. Debar, “Correlation of Intrusion
    Symptoms : an Application of Chronicles”,
    RAID 2003


                                                    12
   Aggregation and Correlation
 Correlation
   – Group alerts that are part of the same attack trend
   – Duplicates
   – Consequences (chain of related alerts)
 Aggregation
   – Group alerts based on certain criteria to aggregate
     severity level, reveal trends, clarify attacker’s
     intentions
   – Situations

                                                           13
                 Duplicates
 Duplicates Definition
  – Initial Alert Class
  – Duplicate Alert Class
  – List of Attributes (that must be equal)
  – Severity Level (new severity level for new
    merged alert)
 Specified by analyst

                                                 14
                Consequences
 Consequences Definition
   – Initial Alert Class
   – Initial Probe Token
   – Consequence Alert Class
   – Consequence Probe Token
   – Severity Level
   – Wait Period
 Links together alerts that are sequential in nature

                                                        15
              Aggregation
 Aggregate based on three axes
  – Alert Class
  – Source
  – Target
 Putting wildcards for different cases gives
  different views
 Aggregate into scenarios

                                                16
                    Scenarios
 Same source/target/attack class
   – A single attacker launching attacks against a single
     victim
 Same source/destination
   – Single attacker running many attacks on a single
     victim
 Same target/attack class
   – Distributed attack against a single victim
 Same source/attack class
   – A single attacker running the same attack against
     multiple victims                                       17
                Chronicles
 “Set of events, linked together by time
  constraints, whose occurrence may depend
  on the context”
 Similar to plan recognition
 Used to model known attack “chunks”
  – Long attack scenarios may have many paths
  – Certain small sequences of events almost
    certainly occur together

                                                18
 So What Is Alert Correlation?
 3 general categories
  – Alert Clustering
  – Matching Predefined Attack Scenarios
  – Prerequisites/Consequences




                                           19
   Prerequisites/Consequences
 F. Cuppens, A. Miège, “Alert Correlation
  in a Cooperative Intrusion Detection
  Framework”, In IEEE Symposium on
  Security and Privacy, 2002
 P. Ning, D. Reeves, et al. (many papers)
  – Check my website for the list
  – Or the very last slide…..

                                             20
   Prerequisites/Consequences
 Prerequisite: the necessary condition for
  the attack to be successful
 Consequence: the possible outcome of the
  attack
 Represented as a logical formula
  – Using only AND and OR connectives



                                              21
          Hyper Alert Type
 (fact, prerequisite, consequence)
 SadmindBufferOverflow =
  ({VictimIP, VictimPort},
  ExistHost(VictimIP) AND
    VulnerableSadmind(VictimIP)
  {GainRootAccess(VictimIP)})



                                      22
    Prepare-For Relationships
 An alert “prepares for” another alert if it
  contributes to the second alert’s
  prerequisite set
 Also must occur earlier in time




                                                23
         Correlation Graph
 Directed acyclic graph, with the nodes
  being alerts and the edges being the
  prepares-for relations
 Could be huge!




                                           24
       Adjustable Reduction
 Aggregation of alerts of the same type
 Can result in overly simple graphs
 Adjustable
  – Analyst can specify a time interval
  – Only alerts with time gap less than the interval
    are merged


                                                       25
Adjustable Reduction




                       26
            Focused Analysis
 Logical combination of comparisons
  between attribute names and constants
 SrcIP = 129.174.142.2 OR DestIP = 129.174.142.2
 Useful for focusing on a critical server




                                                    27
        Graph Decomposition
 Cluster alerts based on “common” features
 Use clusters to separate large graph into
  smaller ones
 (A1.SrcIP = A2.SrcIP) AND (A1.DestIP = A2.DestIP)
 Clustering constraints are specified by the
  analyst


                                                      28
Reduced and Decomposed
    Graph Example




                         29
   Matching Attack Strategies
 Attack Strategy Graph
  – Set of events linked together by certain
    constraints
     • Time Order
     • IP Addresses
 Events can be generalized to deal with
  variations
        SadmindBufferOverflow
                                  RPCBufferOverflow
         TooltalkBufferOverflow                       30
 Measuring Similarity Between
      Attack Strategies
 Error Tolerant Graph Isomorphism
 Use edit distance to derive a similarity
  measure
 Can be used to find similar attacks or to
  match against predefined strategies



                                              31
  Hypothesizing About Missed
           Attacks
 Missed attacks can break up the graphs
  – One attack graph becomes two disconnected,
    seemingly unrelated, attack graphs
 Indirect Prepares-for
 Similarity based merging of attack graphs
 Prune hypotheses with network traffic
  – E.g. one hypothesized attack is ICMP ping,
    but no ICMP traffic occurred during that time

                                                    32
                Outline
 Problem Statement
 Different Correlation Approaches
 A Comprehensive Approach
 Good News and Bad News




                                     33
   A Comprehensive Approach
 F. Valeur, G. Vigna, C. Kruegel, R. Kemmerer, "A
  Comprehensive Approach to Intrusion Detection Alert
  Correlation", In IEEE Transactions on Dependable and
  Secure Computing, 2004




                                                         34
               Alert Fusion
 Combine alerts that are independent
  detection of the same attack instance
  – Must be temporally close
  – From different sensors
  – Identical overlapping attributes




                                          35
          Alert Verification
 Idea: False positives can negatively impact
  alert correlation
 Filter out false positives and irrelevant
  positives (alerts that correspond to failed
  attacks)



                                                36
              Alert Verification
 Passive: use network knowledge to see if attack
  could succeed (low overhead, low confidence)
   – Listing of existence of/services running on IPs
   – Firewall configurations
 Active: check for evidence (high overhead, high
  confidence)
   –   See if service is still running and available
   –   See if extra ports are open
   –   Use vulnerability scanner to test target machine
   –   Remote login and run scripts
                                                          37
       Thread Reconstruction
 Group alerts that refer to attacks launched
  by one attacker against a single target
 Merge alerts with same source and
  destination and within a time interval




                                                38
 Attack Session Reconstruction
 Link network based alerts to host based
  alerts
 Manually specify links between network
  events and process events
  – Alert on web server process (or one of its
    children) can be correlated to a (temporally)
    nearby network alert targeted to that machine
    on port 80
                                                    39
          Focus Recognition
 Identify hosts that are the source or target
  of lots of attacks
 Merge these alerts together into one
 Source: Scanning
 Target: DDoS



                                                 40
       Multi-Step Correlation
 Identify attack patterns that are made up of
  multiple individual attacks
 Create attack patterns by means of expert
  knowledge
 Simply match the merged alerts to the
  attack strategies


                                                 41
         Experimental Results
 Defcon9
  – Input: 6,378,096 alerts
  – Output: 203,303 alerts
  – Reduction: 96.81%
 TreasureHunt
  – Input: 2,811,169 alerts
  – Output: 1,080 alerts
  – Reduction: 99.96%
 MIT/LL 2000
  – Input: 36,635 alerts
  – Output: 17,220
  – Reduction: 53.00%
                                42
  Benefits of Alert Correlation
 Higher level representation of alerts
  reduces clutter and can show attack
  structure
 Reduce false positives
  – False positives are unlikely to correlate with
    other alerts
 May find many attacks and respective
  scenarios

                                                     43
     Limitations of Correlation
 Relies on IDS to alarm each step of the attack
   – Exploit mutations
   – Novel attacks
   – Bad sensor placement
   – Sensor overload - packet loss
   – Restricted ruleset for better performance
 Relies heavily on a priori expert knowledge


                                                   44
Limitations of Correlation (cont)
 Cannot provide a comprehensive view on
  network attacks




                                           45
           MINDS Level 2
 Level 1 IDS alerts
 Anchor Point Identification
 Context Extraction
 Attack Characterization
 Behavior/Host Profiling



                                46
               Questions?
 Paper links located at:
  http://www.cs.umn.edu/~shaneck/wormlist.html
  – At the bottom of the page
 Slides available:
  http://www.cs.umn.edu/~shaneck/Correlation.ppt




                                                   47
A Budding Hacker




                   48
        Peng Ning Reference List
1.   P. Ning, D. Reeves, Y. Cui, "Correlating Alerts Using Prerequisites of Intrusions",
     Technical Report, TR-2001-13, North Carolina State University, Department of
     Computer Science, December 2001
2.   P. Ning, Y. Cui, D. Reeves, "Analyzing Intensive Intrusion Alerts via Correlation",
     In Recent Advances in Intrusion Detection, 2002
3.   P. Ning, Y. Cui, D. Reeves, "Constructing Attack Scenarios through Correlation of
     Intrusion Alerts", In CCS 2002
4.   P. Ning, D. Xu, "Learning Attack Strategies from Intrusion Alerts", In CCS 2003
5.   P. Ning, D. Xu, C. Healey, R. St. Amant, "Building Attack Scenarios through
     Integration of Complementary Alert Correlation Methods", NDSS, February 2004
6.   Y. Zhai, P. Ning, P. Iyer, D. Reeves, "Reasoning about Complementary Intrusion
     Evidence", 20th Annual Computer Security Applications Conference, December
     2004
7.   D. Xu, P. Ning, "Alert Correlation Through Triggering Events and Common
     Resources", 20th Annual Computer Security Applications Conference, December
     2004
8.   P. Ning, D. Xu, "Hypothesizing and Reasoning about Attacks Missed by Intrusion
     Detection Systems", ACM Transactions on Information and System Security, 2004
                                                                                           49

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:12
posted:2/1/2012
language:German
pages:49