Auditing: Measuring something against a standard
How do you know you…?
The student shall be able to:
Define audit, vulnerability, threat, policy, policy objective, policy
control, procedure, baseline, auditor, audit exception, and audit
exception root cause.
Describe the purpose of a baseline, and the contents of a Network
Traffic Baseline and System Baseline.
Define the terms protection, detection, response, and exposure, given
an example time-based security situation.
Describe the purpose of the audit plan’s scope, purpose, checklist,
policy resource guideline, audit strategy.
Write an audit plan.
Describe the purpose of each stage of an audit.
Describe important points of staying out of jail while doing an audit.
Conduct a complete audit, procedurally.
There is no reading this week. Work on your audit plan/report
Types of Audits
Security Audit: Measures how well our security
policies/procedures are relative to best-in-class
Assessment or Verification: Analysis of security
improvements. Are our procedures effective?
Conformance Audit: Measures how well a system or process
conforms to policies/procedures
Validation: How well are we following our guidelines?
Validation: Is it really protecting us?
Assessment: Is our plan effective?
Vulnerability: An unlocked door in infrastructure or
Vulnerability Assessment: An evaluation of
potential vulnerabilities related to the described scope
Threat: An action that exposes a vulnerability
Examples: File deletion, information exposure, improper
use of assets, malware attack
Intentional versus Accidental Threat: Both have same
Exposure = Vulnerability + Threat
Policy: Requirements Rule: Example 1:
Describes ‘what’ needs to be Policy Objective: Reduce
accomplished highway deaths
“Only students currently Policy Control: Set speed limit
enrolled in computer science to 55
courses shall have access to the
computer science lab”
Policy Objective: Differentiate
Policy Objective: Describes why between different users on a
the policy is required system
Policy Control: Technique to Policy Control: Logon
meet objectives restrictions, smart card,
May include a procedure biometric authentication
Discussion: Are these effective
controls by themselves?
Procedure: Outlines ‘how’ the Policy will be
1. “The CS System Administrator shall provide a list of
student IDs to the lab entrance system by running
the XXX program using the YYY file one week before
2. “Students must slide their student ID card through
the card reader and enter the last four digits of their
SSN to gain entry at the CS lab door”
Baseline: Snapshot of a system in a Known Good State
Is a static measure of a system
Enables recognition of changes in system via activity
Enables description of how a system has changed
Most useful when generated automatically
Network Traffic Baseline: Shows traffic volume per
hour of day (Wireshark, Shadow/NFR IDS, etc.)
System Baseline: Shows OS version, available disk
space, description of system files, size of different
Start-> Run-> winver: Prints the version of OS
Start-> Run-> winmsdo; File-> Export: Saves system
Preparing a Baseline
Take a copy of a new system or
To achieve Known Good State:
Update virus signatures
Ensure system fully patched
Do comprehensive virus scan
Check all files (not just system files)
Turn on heuristic virus scanning, which recognizes
suspicious patterns in addition to signatures
Save baselines to CD for offline storage
Time Based Security
Can we react to an attack quickly enough to control it?
Defense in Depth requires multiple layers
Exposure = Detection + Response
Protection > Detection + Response
Estimate Best and Worst Detection and Response
Time to get Exposure
Example 1: Defending a Castle Example 2: Home Alarm
On a hill or mountain An apt. alarm beeps for 15 seconds
waiting for a passcode to be entered
Has a moat The alarm takes 15 seconds to dial the
Has an outer wall The security company takes 30
Trees cut down around the wall seconds to inform the police
It takes the police 2-5 minutes to
Protection: How long will it take arrive at the site
to get through the multiple
layers of defense? Protection:
It takes one minute to empty a jewel
Detection: How long will it take box in the bedroom and walk out
for us to recognize an attack? It takes n minutes to steal all
expensive appliances in a home with
Response: How long will it take to one person
react to an attack?
Example 5: Network Traffic
Example 3: USS Cole Baseline
USS Cole Attack Response: Shadow IDS measures traffic
Move all US military vessels and reports hourly of traffic
out of foreign ports and onto against a baseline. What is best
the open sea and worst Detection times?
Example 6: Sluggish Web
Example 4: Edge router, IDS, service
What is best and worst
Implementation: Measure D + R
Results and Recommendations
Audit Exceptions: Items that fail to meet the audit
Mitigation: Recommendation to reduce loss/harm
Remediation: How to fix an Audit Exception, by policy,
procedure, best practice
Root Cause: Why is there an audit exception? Treat
the illness, not the symptom
Responsibilities include: Raising Awareness:
Measure and report on It’s not ‘if’ we’ll be
risk hacked but ‘when’.
Raise awareness of You can never be too
security issues in order secure…
to reduce risk
Often provide input to
policies and procedures
Audit Plan Outline
What part of the organization are we auditing?
Can audit a process, a technology, a department/division
Example: “Enumerate vulnerabilities for a web server”
What do we hope to accomplish or measure through the audit?
Validation: Are rules implemented correctly?
Measure conformance to policy
Measure if system has been compromised
Audit Plan Outline Cont’d
Policy Resource Guidelines:
Documentation for existing and recommended security guidelines
A definition of how the audit will occur. What tools and techniques will best
meet the objectives?
Each policy has a number of checklist line items
Each checklist line item describes a procedure of what and how to measure a
On cover page request signature of the audit team, the instructor, and the team
from the audited company.
Make sure that both you and company have signed copy of Audit Plan
Policy Resource Guidelines
Company policies: Statement in full or summary
Best Practice references:
Center for Internet Security: www.cisecurity.org
Provides documents that can easily be used as part of an audit checklist, including procedures,
standards, tools, benchmarks
COBIT: Control OBjectives for Information and related Technology: www.isaca.org/cobit.htm
IT-oriented framework for control and mgmt of an organization
FISCAM: Federal Information Systems Control Audit Manual www.ignet.gov/pande/faec/fiscam.pdf
Used by US GAO and many Inspector General Offices
Focuses on Financial Applications of IT
ISO/IEC 27002:2005 Information technology -- Security techniques -- Code of practice for information
Audit Process Outline
Audit Process includes:
Audit Planning: Create Audit Plan
Entrance Conference: Inform people of process
Fieldwork: Measurement of the system
Report Preparation: Complete report
Exit Conference: Discuss report with affected
Report to Management: Provide revised report to
Step 1: Audit Planning
Auditor works with contracting individual to
determine scope/purpose of audit
Research corporate policies, industry best standards
Prepare audit strategy, checklist, and audit procedures
Step 2: Entrance Conference
Auditor meets with all people involved in the audit
Mgmt schedules the meeting, including mgmt, security, system
administrators, users being audited (e.g., if random workstations are
being audited, those users shall attend)
Manager introduces you and explains purpose of audit and discusses
his/her support for audit
Auditor then takes control over meeting to discuss:
Role of others
Timeframes: Make appointments with all parties you need to during
Take team approach: Do not offend anyone or play power games.
People should be excited, not intimidated by you.
Step 3: Fieldwork
Auditor performs audit (often with worker)
Report facts as you find them – as a detective would
Even if the security breach is fixed when found, still
report the breach and the fix
Step 4: Report Preparation
Include Scope/Purpose of audit
Develop technical write-up of report first
What organization does well
What organization needs to do better
If system administrator patched a hole, mention that
Organize findings in a logical way.
Write Executive Summary last
Put Executive Summary as first section in Report
Executive Summary should be understandable to non-technical
Describe good and bad points in bullets (Make people look
Your report must be written professionally, if it is to be credible.
Have another writer/auditor proofread
Step 5: Exit Conference
Auditor communicates findings to entire team
Exit Conference Team = Entrance Conference Team
Go over Executive Summary first
Then give a copy of Audit Report to the team
Team may defend themselves in meeting. Discussion
(not argumentation) is healthy
Amend report after meeting if new information arises
Be careful in wording: “Best Practices include …” NOT
“Most administrators know better than …”
Stay out of arguments if you can
Step 6: Report to Management
Presentation (Plan for 60
Power Point should
Audit purpose, scope,
Positive and Negative
Schedule 2-hour meeting
Have highest executive schedule the meeting
Highest executive kicks off the meeting. Auditor then takes over
Give out copies of power point slides – executives love them
Present for ½ hour
Give full report out and take 15 minute break. This break gives
mgmt a chance to talk to technical staff and ask questions
After 15 minutes, start promptly again (or try to)
Put a list of names of people who did exceptionally well – and
should be encouraged and retained
Answer additional questions when report is complete
Clear up scope/purpose in one meeting (You will look unprofessional if you
keep returning for clarification)
Do not test/venture beyond what is agreed will be done. Extraneous
information is not always welcome
Do not go beyond scope – do not demonstrate vulnerabilities for legal reasons
Always maintain a professional demeanor – not too chummy or informal
Always have company representative present who is most knowledgeable about
the matter being validated
Company retains control: No surprises in tests, results
Work together: Two heads are better than one
Work with in-house expert. Involve them. Be humble
Teach each other: Teach someone to fish is better than giving them a fish
Discuss your findings with the in-house experts as you find them. There
should be no surprises in the exit conference
Oh yeah – dress well!
Audit Report Outline
Audit Report Example
The company is interested in learning about their internet
traffic at headquarters, including what applications are
running, who is using which applications, and when. The
company is also very interested in which web pages are
being accessed both internally and externally. The time
frame for measurement is one week.
Determine amount of traffic not related to business goals.
Identify potential risks and additional controls.
Audit Report Example (2)
“At least M% of bandwidth is used for chat, external email,
SSL, streaming media. N% of web references are for non-
business use. External email is prone to viruses not
protected by company email screeners. Most illegal web
use comes from Building 205, 206 in particular, the Sales
Block chat IP/port addresses in firewall.
Train management on handling inappropriate use of time.
Audit Report Example (3)
Results - Validation:
This section shows line charts demonstrating usage for
each protocol type per hour of working day (on average). It
also shows pie charts showing usage of different categories
of web page accesses. Actual results are provided in
Best-in-class standards (i.e., COBIT) define that policies
should be written and communicated to employees
relating to what they can and cannot do …
Changes for University Environment
SANS recommends providing a technical summary
of the results of the checklist tests.
However, the professor needs to see more detail
Each checklist item must describe:
1) the procedure of how to measure the policy
2) the outcome of the test
3) any recommendations arising from the audit step.
This technique allows the instructor and the
organization to learn how the auditor arrived at his or
her conclusions, and determine the validity of the
Audit Report Example (4)
IT Control Objectives for Sarbanes-Oxley, 2nd Ed.,
Exposure Draft, IT Governance Institute, April 30,
How to Stay Out of Jail!
Audits often require scanning a network to determine
open ports, open applications.
Results can include:
Aborted production systems
VERY upset administrators and managers
The difference between a hacker and a security analyst is
Your written permission is your GET OUT OF JAIL card.
To stay out of jail and keep your job
Plan to scan one subnet at a time! Pick an off-peak time in case something does go
Get permission in Writing!
Publicize the scan! The managers and system administrators must know the exact date
and time of the scan.
Eventually something will go seriously wrong, so always take precautionary steps.
System administrators who go into panic mode for hours over your audit will not
Be present! Be available for the entire duration of the scan, in case something does go
wrong or you do get questions. Also, expect to answer questions up to a few days later.
Be persistent! Be careful to check all devices within the scope. False positives and false
negatives occur, so be extra careful.
Provide Feedback! When the audit is complete, report to the system administrator or
network manager and help them fix vulnerabilities. Complete the cycle within schedule,
then begin scanning the next subnet.
Note: If a host reboots due to an audit scan, it would have happened with a hacker – just
a matter of time.
Example Written Notice
Subject: Security Audit Tuesday Oct 10
Next Tuesday, Oct 10, from 4-6 PM we will be conducting an audit of
the firewall. We plan to validate the services that the firewall allows to
pass through, both inbound and outbound. As part of this audit’s
scanning process, a significant number of TCP and UDP packets will be
generated, and some ICMP packets. Specifically, we will be scanning
ports 1-NNNN with a UDP scan, a SYN half-open scan and a full tcp-
connect scan. In order to try to minimize any significant impact to the
firewall operations, we will generate packets slowly, at the rate of 1
packet every X seconds.
During the scan period, I will be available in room XXXX. I will also be
reachable via phone: 255-5466; via pager 262-445-9933; or email:
firstname.lastname@example.org. I will be happy to reply to any questions or
concerns, and provide more detail about our audit if necessary.
Summary & Conclusion
Stay out of Jail:
Get signature on audit plan
Broadcast what you plan to do when
Only do what is in the audit plan
For this class:
Be very specific about what tests you did and what results you got
Be sure you have a member of the organization with when you do
audit – allow them to see all problems at time of audit
Double-check with me before submitting proposal or report to
your customer – submit most professional document