AI by yaofenji


									      Intrusion Detection Systems in the
              Methods and Issues

                    Leigh Heyman - Artificial Intelligence Lab
                     Massachusetts Institute of Technology

           (with assistance from Curt Freeland, Univ. of Notre Dame and Sally
                               Goldberg, St. Mary's College )

Leigh Heyman,GCIA                                                   Artificial Intelligence L
                       – Mix of general and technical
                       – "sureveillance cameras"
                       – A quick (and surprising) statistic
                    What is Intrusion Detection?
                       – Host Based
                       – Network Based (Focus is on Network Systems)
                       – Implementation
                       – IDSes-- discussion of free software systems
                             "   SHADOW
                             "   Snort
                       – Politics
                             " Privacy concerns
                       –   Access to data
                       –   Evidence: Chain of Custody
                       –   Network Security BIG PICTURE
                            " Policy is key

                       –   Why run an IDS?
                       – an intrusion
Leigh Heyman,GCIA                                                      Artificial Intelligence L
                    What's this???
                             "   Video cameras for
                             "   Established method of
                             "   Can we do the same with
                                 our network?

Leigh Heyman,GCIA                             Artificial Intelligence L
       Do you know what's on your net?
                            Seen this in your syslogs?:
       Aug 13 07:01:58 rpc.statd[315]: gethostbyname error

           Statistics from our IDS indicate on average one "network
                         event" EVERY THREE HOURS!

         ("Network Events" are any data on the network which, according to
                 your policy, may indicate malicious activity, including:
            port/host scanning, vulnerability scanning, "out-of-spec" packets--
             SYNFIN, tiny fragaments,etc., active exploits, DNS zone transfer
                                   attempts and more...)
Leigh Heyman,GCIA                                                   Artificial Intelligence L
            What is Intrusion Detection?

            Intrustion detection systems monitor and
            analyze key system "vital signs" to detect
             and track certain types of security policy

            (ID, and IDS refer to Intrusion Detection, and Intrusion Detection
                                   System(s) respectively)

Leigh Heyman,GCIA                                                      Artificial Intelligence L
                                  Host Based ID
      "   Watches individual systems
          –   System logs
               "    syslog
               "    sulog, etc.
          –   Personal firewalls
               • Norton, McAfee, NetworkICE, etc.
          –   Tripwire
          –   Tcp-wrappers
          –   Portsentry

Leigh Heyman,GCIA                                   Artificial Intelligence L
                       Network Based ID
      "   Network ID systems capture all the data from the
          network stream and process that data against a set of
          signatures, or patterns of known attacks.
      "   Centralized data about entire network of hosts
      "   essentially a network sniffer.
           – plugs into a shared bandwidth link on the network.
             s     all
           – “ ees” packets that go by on the wire.
           – can save copies of the entire packet, the packet headers, or just
             those packets which meet certain specifications, to the system
           – analysis determines what is happening on the network.

Leigh Heyman,GCIA                                                 Artificial Intelligence L

Leigh Heyman,GCIA             Artificial Intelligence L
                      Implementing a NIDS
      "   What makes a good IDS?
          – First and foremost is log fidelity- is all the info there?
          – "Tunability"-- the ability to write/modify the attack signature
          – Low "false-positive/negative" rate, and the abiity to tune to
            optimize this rate
          – A “riendly”user interface requires less skill and knowledge to
          – The capability to filter the input-- and control the input filter
              • Cannot truly watch and analyze every packet, so we want
                the ability to decide which ones to watch
          – archiving/database features
          – Cheap is always good.
Leigh Heyman,GCIA                                                      Artificial Intelligence L
                    Implementing a NIDS
                        "   Where does it go?
                            –   DMZ model: Inside or outside the DMZ
                                often a matter of religion, truly robust
                                IDSes will have sensors on both sides.
                            –   With most routers and some switches
                                you have the ability to "mirror" or "span"
                                a port, the sensor can be on that port,
                                this is better than putting in "in the
                            –   Be careful with switches, often
                                "mirroring" or "spanning" can have
                                adverse affects on switch performance.
                            –   Few IDSes can keep up with ethernet
                                speeds, anything beyond 300Mbps is
Leigh Heyman,GCIA               pushing it.                Artificial Intelligence L
                    Implementing a NIDS
     • If you haves a single Internet connection, you could get by with a
        single sensor to monitor that feed.
     • Sites with multiple external connections or a single high-speed
        (ethernet) uplink will require multiple sensors!
     • Residence Hall networks
         – Prime location of unsecured, unmaintained systems
         – Most common scenario: student wkstation compromised, then used to
            attack others
     • Business unit networks (bookstore, athletic ticket office, cafeteria)
         – Any place where money is involved....
     • Corporate Records (grades, registration, accounts due)
         – Any place where protection of sensitive information is a priority...
     • "mobile sensor" to plug in to hot-spots as needed.

Leigh Heyman,GCIA                                                       Artificial Intelligence L
                      Implementing a NIDS
     • care and feeding.
         – keep the IDS systems secure!
         – Run the system for ~1 month before really taking action
         – go through the output regularly.
             • analyst may spend as much as 20 minutes per hour on
               analyzed output from a single sensor.
             • Multiple sensor sites will require more time to examine and
         – monitor disk space consumption.
         – back up the collected data in a manner consistent with the
            accepted rules of criminal evidence collection.
Leigh Heyman,GCIA                                               Artificial Intelligence L
       "   Free
           –   SHADOW
                "   One of the first broadly implemented free IDSes. VERY high
                    fidelity output, but requires a lot of knowledge to operate.
           –   Snort
                "   Full featured, very robust... becoming the de-facto free IDS
       "   Commercial
           –   Network Flight Recorder
                "   Popular product, as simple or complex as you want it to be.
                    "Appliance" type implementation. Research/Academic licensed
           –   RealSecure
                "   Robust, modular implementation. NIDS is a component of larger
                    security suite including host-based ID, and firewalling

Leigh Heyman,GCIA                                                        Artificial Intelligence L
      "   By default, Shadow collects everything on the wire.
      "   The output can consume extraordinary amounts of disk.
      "   Comes with a built in web-based analysis tool.
      "   Analysis tool handles multiple sensors.
      "   Drawbacks:
          –   output requires strong knowledge to interpret.
          –   output is not flashy.
          –   not being (actively) developed.
          –   Analyst station needs a fast CPU, and a lot of memory!

Leigh Heyman,GCIA                                              Artificial Intelligence L
      "   Perl scripts with tcpdump
          –   (at least) two PCs:
               "    Sensor node(s)
                     –   runs tcpdump with rudimentary filter to hourly log file
               "    Analysis host
                     –   hourly cron job pulls tcpdump log from sensor, and
                         processes it against more complex series of filters. Perl
                         scripts parse through the output, finds scans, then generates
                         html output.

Leigh Heyman,GCIA                                                           Artificial Intelligence L
Leigh Heyman,GCIA   Artificial Intelligence L
      "   simple, efficient FREE IDS
      "   Very well-written and maintaned, robust application
      "            d
          Snort is “ riven”by a set of (community developed) rules.
      "   Actively (constantly) under development.
      "   Windows and UNIX versions available (source form).
      "   SnortSnarf (an add-on) provides a web-based analysis
      "   In general requires lower level of skill to operate and
          analyze output

Leigh Heyman,GCIA                                          Artificial Intelligence L
      "   Alerts generated and/or packets logged when a "rule"
          is triggered.
      "   Very simple rule language for writing your own rules
      "   Ability to log alerts to syslog, directories in ascii,
          tcpdump format raw data, even send "windowgrams"
      "   Different alert styles from one-line, to verbose
      "   Modular "plug-in" architechture for adding functionality
           – Many available plug-ins, including SQL and Oracle database
             logging, statistical analysis, TCP stream and telnet session
             reassembly, active response using "sniping"
      "   Resistant against some of the newer attacks directed at
          foiling IDSes

Leigh Heyman,GCIA                                                     Artificial Intelligence L
Leigh Heyman,GCIA   Artificial Intelligence L

Leigh Heyman,GCIA            Artificial Intelligence L
                          Political Issues
      "   Privacy Concerns
          –   No easy answers here
          –   Much of the responsibilty lies with the analyst
          –   Can only manipulate the data so much before it loses its
          –   Some talk about whether the network headers on certain
              networks constitute "student information" under FERPA.
              Still an open question.

Leigh Heyman,GCIA                                               Artificial Intelligence L
      Privacy concerns (cont'd)
      "   What Notre Dame did when the users found out about it,
          and threated to stampede because they thought their
          privacy was being invaded:
           – Newspaper learned of IDS, and launched Carnivore-like hype

           –   Showed them the IDS output!
                "   made the IDS output available to anyone on campus and invited
                    all to explore it for themselves.
                "   asked them to find the sensitive information (from their host) that
                    they are afraid was gathered.
                "                                           s
                    showed them that the IDS just looks for “ ignatures”that should
                    not appear on the network.
                "   assured them that an have not got the time, or the desire, to read
                    their mail, monitor their surfing, or peruse their files.
Leigh Heyman,GCIA                                                          Artificial Intelligence L
                              Access to Data
      "   So now that we are seeing results from our IDS, what
          do we do with it, who do we show it to?
          – "cease and decist" notices
          – C&D clearing houses:
               "    Internet Storm Watch
          – Availability of web output
          – Sanitization of data before publishing
          – Who needs to know
          – Staging the data (classification model)
               "    Only ID analysts have access "classified" raw data, and analysis
                    host itself
               "    "Declassified" data can then be propagated to interested party
                    via a more publicly available means
          –   Why do it this way......
Leigh Heyman,GCIA                                                       Artificial Intelligence L
                         Chain of Custody
      "   The reason we do it this way
           –   Must preserve "chain-of-custody" should the data ever be
               needed as evidence
           –   If everyone has login shells on the sensor and/or
               analysis station, the integrity of the data as evidence is
           –   Devise methods to have an accurate and reliable audit of
               access to the raw and post-processed data
      "   Lastly, the more loudly we announce the presence of
          our IDS, and its data, the more likely it will be viewed by
          the "bad guys"... how hard is it to turn a tool into a
Leigh Heyman,GCIA                                                Artificial Intelligence L
                    Network ID, the BIG PICTURE

      "   IDS is only a tool
           –   "Detection" is not "Prevention"
           –   The Intrusion Prevention System, by Brooklyn Bridge
               Technologies inc.
      "   Only useful as a component of a larger "defense-in-
          depth" framework.
           –   Safe rating system: 60TLTR
                " How long before the folks with the big guns show up?

      "   Policy is key
           –   Establish users' expectation of privacy
           –   Cannot detect "network events"-- anomalous traffic--
               without at least loosly defined baseline
Leigh Heyman,GCIA                                            Artificial Intelligence L
            Why Should we run an IDS?
      "   Will an IDS help us block Napster/Gnutella/Cow Porn?
           – No, but it will point out the depth of your problems!
           – Yes, because now you know what is eating up all of your bandwidth.

      "   Will an IDS make things simpler for the security staff?
           – No, because they will now be aware of all of the nasty things on the
           – Yes, because they can have specific data on where to focus the
             needs and skills of the group.
      "   Will an IDS create problems for the users?
           – No, because the IDS process is invisible to the users.
           – Yes, because the users may find certain IP/port combinations
             blocked due to the information gleaned from the IDS.

Leigh Heyman,GCIA                                                     Artificial Intelligence L
     • Intrusion Detection systems can help you secure your
            • give a clear picture of the security landscape
            • help tune the security infrastructure in accordance
            • significantly improve most elements of the incident
              handling capability
                    – Preparation, detection, containment, and eradication

     • IDSes require several investments:
          • time required to actively monitor and analyze IDS output
          • hardware for the sensor and analysis stations.
          • staff training to make the most of the IDS system’ potential.
          • You need to prepare for a surprise! .....
Leigh Heyman,GCIA                                                      Artificial Intelligence L
                    An Incident....

Leigh Heyman,GCIA                     Artificial Intelligence L
                            August              13th       2000
      "   52 hosts comprosmised in 20 seconds using scripted
          rpc.statd attack
      "   What the syslogs showed:
      Aug13 07:01:58 txakoli syslogd: Cannot glue message parts together
      Aug 13 07:01:59 txakoli 173>
      Aug 13 07:01:58 rpc.statd[315]: gethostbyname error for
         > ....... etc.

      "   So.... whodunit? What evidence do we have?
           – Where did the attack come from?
           – What was the attack method?
           – Do we have any hope of catching the bad guy?

      "   What if the attack script erased the logs?
Leigh Heyman,GCIA                                                          Artificial Intelligence L
                           August             13th       2000
      "   What the network saw:
      (names and addresses have been left unchanged to implicate the guilty)
      07:01:51.141438 > udp 56
      07:01:51.141918 > udp 28
      07:01:55.217308 > udp 1076
      07:02:02.230468 > : S 843433918:843433918(0)
         win 32120 (DF)
      07:02:02.230706 > S 4178397570:4178397570(0)
         ack 843433919 win
      07:02:02.288170 > . ack 1 win 32120 (DF)
      07:02:02.288183 > P 1:72(71) ack 1 win 32120
         (DF) **
      07:02:02.288496 > . ack 72 win 32120 (DF)
      07:02:03.290337 > P 72:100(28) ack 1 win 32120
         (DF) ***
      07:02:03.290496 > F 100:100(0) ack 1 win 32120
      07:02:03.290698 > . ack 101 win 32120 (DF)

      "   A clear audit trail, AND circumstantial evidence we can
          forward to Nenette.
Leigh Heyman,GCIA                                                              Artificial Intelligence L
                       August 13th 2000
      "   Result
          –   All 50+ hosts contained within a few hours, eradicated
              and back online in less than 24
          –   Captured subsequent incriminating data of attacker trying
              to come back to visit compromised host
          –   Disapointed Nenette because we recovered so fast we
              never broke the $8000 mark.

Leigh Heyman,GCIA                                            Artificial Intelligence L
      "   URLs
           – (internet storm watch)
      "   Books
           – Intrustion Detection, An Analyst's Handbook (2nd Edition), by
             Stephen Northcutt et al.
      "   Mailing Lists
           – Many lists at / :
               "    bugtraq
               "    focus-ids
               "    focus-incidents
                      – digests and searcheable archives
Leigh Heyman,GCIA                                                    Artificial Intelligence L
                    Thank You


Leigh Heyman,GCIA                      Artificial Intelligence L

To top