Managing the business risk of fraud.ppt

Document Sample
Managing the business risk of fraud.ppt Powered By Docstoc
					                 Managing the Business
                  Risk of Fraud using
                Sampling and Data Mining


                                   Mike Blakley
  Presented to:

                                                         Fall 2009
Managing the business risk of fraud using sampling and
data mining                                                          EZ-R Stats, LLC
        PWC Global Survey – Nov, 2009
                “Economic crime in a downturn”


   Sharp rise in accounting fraud
    over the past 12 months
   Accounting fraud had grown to 38
    percent of the economic crimes in
    2009
   Employees face increased
    pressures to :
    –   meet performance targets
    –   keep their jobs
    –   keep access to funding



        Managing the business risk of fraud   EZ-R Stats, LLC
                      Survey findings

   Greater risk of fraud due to increased
    incentives or pressures
   More opportunities to commit fraud, partially
    due to reductions in internal finance staff
   While companies are expecting more fraud,
    they have not done much
   People who look for fraud are more likely to
    find it

      Managing the business risk of fraud   EZ-R Stats, LLC
     Session objectives
       Understand the framework for managing the business
        risk of fraud
       Plan, perform and explain statistical sampling in audits
       Reduce audit costs using data mining, sequential
        sampling and other sampling techniques
       Apply SAS 56, the new SAS suite and the revised
        (2007) Yellow Book.
       Run, hands-on, the most productive analytic technique
        (regression analysis).
       Use data mining to introduce greater efficiency into the
        audit process, without losing effectiveness.



    Managing the business risk of fraud                EZ-R Stats, LLC
          Session agenda - 1

   Introduction and the Process for Managing the Business Risk of
    Fraud
   Introductions All Around
   Course Objectives
   Framework of risk management for fraud
   Fundamentals of data mining
   Data mining: The Engine That Drives analysis
     –   Analytics and Regression
              Sources of Analytics Data
              Basic and Intermediate ARTs
   SAS 56
   IIA Practice Advisory 2320
   The Yellow Book (2007 revision)
   The Guide – “Managing the Business Risk of Fraud”


         Managing the business risk of fraud            EZ-R Stats, LLC
                Session Agenda (cont’d) –
                   Sampling refresher

   Sampling
   The sampling process
   Sampling methods
   RAT-STATS
     –   Random Numbers
     –   Determining Sample Size
     –   Case Study
     –   Attribute sampling
     –   Variable Sampling
     –   Case study
     –   Stratified Sampling
     –   Obtaining and Interpreting the results
   Other Sampling Approaches
   DCAA Audit Package
   Sequential Sampling
          Overview of the process
          Attribute Sampling
          Variable Sampling



         Managing the business risk of fraud      EZ-R Stats, LLC
               Session Agenda (cont’d) –
                Linear regression as an
                       audit tool

   Regression Analysis
   Overview
   Terms
   Statistical basis
   Charting Regression … Seeing Is Believing
   Plotting Data
     –   Inserting a “Trend line”
   Statistical Intervals
     –   Confidence Intervals
     –   Prediction Intervals
     –   Calculation of Statistical “Confidence Bounds”
   Case Study - Wake County Schools Bus Maintenance


         Managing the business risk of fraud              EZ-R Stats, LLC
              Session Agenda (cont’d) –
                   Data mining, or
                  How to test 100%

   Overview
   Statistical Basis
   Data Conversion and Extraction
   Data mining objectives
    –   Classification
    –   Trends
    –   Identification of extremes
    –   Major types of data analysis
             Numeric
             Date
             Text



        Managing the business risk of fraud   EZ-R Stats, LLC
              Session Agenda (cont’d) –
           Excel as an Analytics tool

   Macros
     Tools – Data Analysis
     The Macro facility
    –   Adding a little “class” to your audit
    –   VBA – “friend” or “foe”




        Managing the business risk of fraud     EZ-R Stats, LLC
Handout (CD)

   CD with articles and software
   PowerPoint presentation
   More info at www.ezrstats.com




      Managing the business risk of fraud   EZ-R Stats, LLC
“Cockroach” theory of auditing

   If you spot one roach….




      Managing the business risk of fraud   EZ-R Stats, LLC
“Cockroach” theory of auditing

   There are probably 30
    more that you don’t
    see…




      Managing the business risk of fraud   EZ-R Stats, LLC
Statistics based “roach” hunting




  Many frauds coulda/woulda/shoulda been detected with analytics

    Managing the business risk of fraud           EZ-R Stats, LLC
                Overview
   Fraud patterns detectable with
    digital analysis
   Basis for digital analysis
    approach
   Usage examples
   Continuous monitoring
   Business analytics

Managing the business risk of fraud   EZ-R Stats, LLC
                                       Objective 1

    The Why and How

               Three brief examples
               ACFE/IIA/AICPA Guidance Paper
               Practice Advisory 2320-1
               Auditors “Top 10”
               Process Overview
               Who, What, Why, When & Where


Managing the business risk of fraud   EZ-R Stats, LLC
                                         Objective 1a
          Example 1
Wake County Transportation Fraud

                  Supplier Kickback – School Bus
                   parts
                  $5 million
                  Jail sentences
                  Period of years




  Managing the business risk of fraud   EZ-R Stats, LLC
                                           Objective 1a

Too little too late

            Understaffed internal audit
            Software not used
            Data on multiple platforms
            Transaction volumes large




   Managing the business risk of fraud   EZ-R Stats, LLC
                                          Objective 1a

                           Preventable

        Need structured, objective
         approach
        Let the data “talk to you”
        Need efficient and effective
         approach




Managing the business risk of fraud      EZ-R Stats, LLC
                                              Objective 1
           Regression Analysis

   Stepwise to find
    relationships
    –   Forwards
    –   Backwards
   Intervals
    –   Confidence
    –   Prediction




        Managing the business risk of fraud     EZ-R Stats, LLC
                                          Objective 1

                         Data outliers

   Sometimes an “out
    and out Liar”
   But how do you
    detect it?




    Managing the business risk of fraud     EZ-R Stats, LLC
                                              Objective 1

         Data Outliers

   Plot transportation costs vs.
    number of buses
   “Drill down” on costs
    –   Preventive maintenance
    –   Fuel
    –   Inspection




        Managing the business risk of fraud     EZ-R Stats, LLC
 Scatter plot with prediction and
 confidence intervals




Managing the business risk of fraud   EZ-R Stats, LLC
                                                                   Objective 1a
                                     Example 2
Cost of six types of AIDS drugs

                                    Total Cost of AIDS Drugs

                       200
       Dollar Amount




                       150                                                     NDC1
                                                                               NDC2
                       100
                                                                               NDC3
                        50                                                     NDC4

                         0                                                     NDC5

                             NDC1    NDC2   NDC3    NDC4   NDC5     NDC6       NDC6

                                              Drug Type




   Managing the business risk of fraud                            EZ-R Stats, LLC
                                      Objective 1

 Medicare HIV Infusion Costs
   CMS Report for 2005
   South Florida - $2.2 Billion
   Rest of the country combined -
    $.1 Billion




Managing the business risk of fraud     EZ-R Stats, LLC
                                                                                 Objective 1
                                      Pareto Chart
                                     Medicare HIV Infusion Costs - 2005 ($Billions)
                                               data source: HHS CMS

                                 120.0%


                                 100.0%
         Annual Medicare Costs




                                 80.0%

                                                                                      Pct
                                 60.0%
                                                                                      Cum Pct

                                 40.0%

                                 20.0%


                                  0.0%
                                      1

                                           3

                                               5

                                                    7

                                                         9

                                                             11

                                                                  13

                                                                       15
                                                        County




Managing the business risk of fraud                                                         EZ-R Stats, LLC
                                                                           Objective 1a
                                      Example 2
  Typical Prescription Patterns

                                  AIDS Drugs Prescription Patterns

                      60.0
                      50.0                                                             NDC1
       Dollar Value




                                                                                       NDC2
                      40.0
                                                                                       NDC3
                      30.0
                                                                                       NDC4
                      20.0
                                                                                       NDC5
                      10.0                                                             NDC6
                       0.0
                             Prov 1   Prov 2   Prov 3   Prov 4   Prov 5     Prov 6
                                                 Prescriber



Managing the business risk of fraud                                       EZ-R Stats, LLC
                                                                   Objective 1a
                                      Example 2
           Prescriptions by Dr. X

                               Dr. X compared with Total Population

                         350
                         300
         Dollar Amount




                         250
                         200
                         150                                               Population
                         100                                               Dr. X
                          50
                           0
                               NDC1   NDC2   NDC3   NDC4   NDC5   NDC6
                                              Drug Type



Managing the business risk of fraud                               EZ-R Stats, LLC
                                           Objective 1a
                             Example 2
                           Off-label use

    Serostim
      –   Treat wasting syndrome, side effect of
          AIDS, OR
      –   Used by body builders for recreational
          purposes
      –   One physician prescribed $11.5 million
          worth (12% of the entire state)



    Managing the business risk of fraud    EZ-R Stats, LLC
                                                                     Objective 1a
                                     Example 3
                                  Revenue trends

                                           Overall Revenue Trend

                            1.2
         Annual Billings




                           1.15
                            1.1
                                                                           Overall
                           1.05
                                                                           Linear (Overall)
                             1
                           0.95
                            0.9
                                    2001          2002        2003
                                              Calendar Year




Managing the business risk of fraud                                  EZ-R Stats, LLC
                                Example 3                     Objective 1a

                            Dental Billings

                                 Rapid Increase in Revenues

                            5
          Annual Billings




                            4
           ($millions)




                                                                 Billings A
                            3
                                                                 Billings B
                            2
                                                                 Linear (Billings A)
                            1
                            0
                                2001       2002        2003
                                       Calendar Year




Managing the business risk of fraud                           EZ-R Stats, LLC
                                       Objective 1b
         Guidance Paper

   A proposed implementation approach
   “Managing the Business Risk of Fraud: A
    Practical Guide” http://tinyurl.com/3ldfza
   Five Principles
   Fraud Detection
   Coordinated Investigation Approach




Managing the business risk of fraud    EZ-R Stats, LLC
                                            Objective 1b

Managing the Business Risk of
Fraud: A Practical Guide

                ACFE, IIA and AICPA
                 Exposure draft issued
                 11/2007, final 5/2008
                Section 4 – Fraud
                 Detection




   Managing the business risk of fraud   EZ-R Stats, LLC
Guidance Paper

     Five Sections
       –   Fraud Risk Governance
       –   Fraud Risk Assessment
       –   Fraud Prevention
       –   Fraud Detection
       –   Fraud Investigation and
           corrective action




   Managing the business risk of fraud   EZ-R Stats, LLC
Risk Governance

   Fraud risk management program
   Written policy – management’s expectations
    regarding managing fraud risk




      Managing the business risk of fraud   EZ-R Stats, LLC
Risk Assessment

   Periodic review and assessment of potential
    schemes and events
   Need to mitigate risk




      Managing the business risk of fraud   EZ-R Stats, LLC
Fraud Prevention

   Establish prevention techniques
   Mitigate possible impact on the organization




      Managing the business risk of fraud   EZ-R Stats, LLC
Fraud Detection

   Establish detection techniques for fraud
   “Back stop” where preventive measures fail,
    or
   Unmitigated risks are realized




      Managing the business risk of fraud   EZ-R Stats, LLC
Fraud Investigation and Corrective
Action

   Reporting process to solicit input on fraud
   Coordinated approach to investigation
   Use of corrective action




      Managing the business risk of fraud   EZ-R Stats, LLC
“60 Minutes” – “World of Trouble”

     2/15/09 – Scott Pelley
       –   Fraud Risk Governance – “one grand wink-wink,
           nod-nod “
       –   Fraud Risk Assessment - categorically false
       –   Fraud Prevention – “my husband passed away”
       –   Fraud Detection - We didn't know? Never saw one.
       –   Fraud Investigation and corrective action - Pick-A-
           Payment losses $36 billion




   Managing the business risk of fraud           EZ-R Stats, LLC
                                            Objective 1b


Section 4 – Fraud Detection
         Detective Controls
         Process Controls
         Anonymous Reporting
         Internal Auditing
         Proactive Fraud Detection




   Managing the business risk of fraud   EZ-R Stats, LLC
                                              Objective 1b


Proactive Fraud Detection

                Data Analysis to identify:
                 – Anomalies
                 – Trends
                 – Risk indicators




   Managing the business risk of fraud   EZ-R Stats, LLC
Fraud Detective Controls

   Operate in the background
   Not evident in everyday business
    environment
   These techniques usually –
    –   Occur in ordinary course of business
    –   Corroboration using external information
    –   Automatically communicate deficiencies
    –   Use results to enhance other controls

        Managing the business risk of fraud   EZ-R Stats, LLC
Examples of detective controls

   Whistleblower hot-lines (DHHS and OSA
    have them)
   Process controls (Medicaid audits and edits)
   Proactive fraud detection procedures
    –   Data analysis
    –   Continuous monitoring
    –   Benford’s Law




        Managing the business risk of fraud   EZ-R Stats, LLC
                                            Objective 1b


Specific Examples Cited

            Journal entries – suspicious
             transactions
            Identification of relationships
            Benford’s Law
            Continuous monitoring




   Managing the business risk of fraud   EZ-R Stats, LLC
                                                Objective 1b
Data Analysis enhances ability to
detect fraud

               Identify hidden relationships
               Identify suspicious transactions
               Assess effectiveness of internal
                controls
               Monitor fraud threats
               Analyze millions of transactions




    Managing the business risk of fraud      EZ-R Stats, LLC
Continuous Monitoring of Fraud
Detection

   Organization should develop ongoing
    monitoring and measurements
   Establish measurement criteria (and
    communicate to Board)
   Measurable criteria include:




      Managing the business risk of fraud   EZ-R Stats, LLC
Measurable Criteria – number of

   fraud allegations
   fraud investigations resolved
   Employees attending annual ethics course
   Whistle blower allegations
   Messages supporting ethical behavior
    delivered by executives
   Vendors signing ethical behavior standards

      Managing the business risk of fraud   EZ-R Stats, LLC
Management ownership of each
technique implemented

   Each process owner should:
    –   Evaluate effectiveness of technique regularly
    –   Adjust technique as required
    –   Document adjustments
    –   Report modifications needed for techniques which
        become less effective




        Managing the business risk of fraud   EZ-R Stats, LLC
Practice Advisory 2320-1
Analysis and Evaluation

   International standards for the professional
    practice of Internal Auditing
   Analytical audit procedures
    –   Efficient and effective
    –   Useful in detecting
             Differences that are not expected
             Potential errors
             Potential irregularities


        Managing the business risk of fraud       EZ-R Stats, LLC
Analytical Audit Procedures

      May include
       – Study of relationships
       – Comparison of amounts with
         similar information in the
         organization
       – Comparison of amounts with
         similar information in the
         industry
   Managing the business risk of fraud   EZ-R Stats, LLC
Analytical audit procedures

   Performed using monetary amounts, physical
    quantities, ratios or percentages
   Ratio, trend and regression analysis
   Period to period comparisons
   Auditors should use analytical audit
    procedures in planning the engagement



      Managing the business risk of fraud   EZ-R Stats, LLC
Factors to consider

   Significance of the area being audited
   Assessment of risk
   Adequacy of system of internal control
   Availability and reliability of information
   Extent to which procedures provide support
    for engagement results


      Managing the business risk of fraud   EZ-R Stats, LLC
                                                 Objective 1c


     Peeling the Onion

                                      Fraud Items
                                      Possible Error Conditions
                                      Population as Whole




Managing the business risk of fraud          EZ-R Stats, LLC
                                                            Objective 1d


     Fraud Pattern Detection

                             Round Numbers
                 Market Basket           Benford’s Law



          Stratification                           Gaps

                                Target Group

            Trend Line                          Univariate



                      Holiday             Duplicates
                                Day of Week


Managing the business risk of fraud                      EZ-R Stats, LLC
                                            Objective 1e



Digital Analysis (5W)
 A little about the basics of digital analysis….
        Who
        What
        Why
        Where
        When


   Managing the business risk of fraud   EZ-R Stats, LLC
                                            Objective 1e


Who Uses Digital Analysis

              Traditionally, IT specialists
              With appropriate tools, audit
               generalists (CAATs)
              Growing trend of business
               analytics
              Essential component of
               continuous monitoring


   Managing the business risk of fraud   EZ-R Stats, LLC
                                         Objective 1e


      What - Digital Analysis

        Using software to:
           –   Classify
           –   Quantify
           –   Compare
        Both numeric and non-numeric
         data



Managing the business risk of fraud   EZ-R Stats, LLC
                                         Objective 1e


How - Assessing fraud risk

   Basis is quantification
   Software can do the “leg work”
   Statistical measures of difference
    – Chi square
    – Kolmogorov-Smirnov
    – D-statistic
   Specific approaches

Managing the business risk of fraud   EZ-R Stats, LLC
                                             Objective 1e


     Why - Advantages

    Automated process
    Handle large data populations
    Objective, quantifiable metrics
    Can be part of continuous monitoring
    Can produce useful business analytics
    100% testing is possible
    Quantify risk
    Repeatable process


    Managing the business risk of fraud   EZ-R Stats, LLC
                                         Objective 1e


    Why - Disadvantages


     Costly (time and software costs)
     Learning curve
     Requires specialized knowledge




Managing the business risk of fraud   EZ-R Stats, LLC
                                         Objective 1e


When to Use Digital Analysis

         Traditional – intermittent (one off)
         Trend is to use it as often as possible
         Continuous monitoring
         Scheduled processing




Managing the business risk of fraud   EZ-R Stats, LLC
                                            Objective 1e


  Where Is It Applicable?

       Any organization with data in digital
        format, and especially if:
          –   Volumes are large
          –   Data structures are complex
          –   Potential for fraud exists




Managing the business risk of fraud     EZ-R Stats, LLC
Disadvantages of digital analysis

   Cost
    –   Software
    –   Training
    –   Skills not widely available
   Time consuming
    –   Development costs
    –   Testing resources


        Managing the business risk of fraud   EZ-R Stats, LLC
                                          Objective 1

Objective 1 Summarized

       Three brief examples
       CFE Guidance Paper
       “Top 10” Metrics
       Process Overview
       Who, What, Why, When & Where



   Managing the business risk of fraud   EZ-R Stats, LLC
        Objective 1 - Summarized

     Understand the framework for managing the business
      risk of fraud
     Plan, perform and explain statistical sampling in audits
     Reduce audit costs using data mining, sequential
      sampling and other sampling techniques
     Apply SAS 56, the new SAS suite and the revised (2007)
      Yellow Book.
     Run, hands-on, the most productive analytic technique
      (regression analysis).
     Use data mining to introduce greater efficiency into the
      audit process, without losing effectiveness.


         Next is plan, perform …

Managing the business risk of fraud             EZ-R Stats, LLC
Statistical Sampling

   Brief History / Timeline
   Overview
   Attribute Sampling – Compliance
   Variable Sampling – Numeric Estimates




      Managing the business risk of fraud   EZ-R Stats, LLC
History of Sampling

   Basis is two laws/theorems of probability
   Law of Large Numbers
   Central Limit Theorem




      Managing the business risk of fraud   EZ-R Stats, LLC
Law of large numbers

                                Simulated rolling of dice

              7


              6


              5


              4                                                  Result
      Value




                                                                 Average
              3                                                  Linear (Result)


              2


              1


              0
                  1   7 13 19 25 31 37 43 49 55 61 67 73 79 85
                                  Observation




   Managing the business risk of fraud                                    EZ-R Stats, LLC
Time Line - LLN

   Indian mathematician Bramagupta 600 AD
   Italian mathematician Cardon 1500’s

   Statement without proof that empirical
    statistics improve with more trials




      Managing the business risk of fraud   EZ-R Stats, LLC
Time line LLN (continued)

   Jacob Bernoulli first to prove in 1713
   Foundation for central limit theorem




      Managing the business risk of fraud   EZ-R Stats, LLC
Central limit theorem

 Classic measure


  Mean of a sufficiently large
  number of random samples
  will be approximately
  normally distributed.




     Managing the business risk of fraud   EZ-R Stats, LLC
The traditional explanation




   Managing the business risk of fraud   EZ-R Stats, LLC
Central Limit Theorem

   See it in action today
   Any population
   Large number of samples
   Average is “normally” distributed




      Managing the business risk of fraud   EZ-R Stats, LLC
History of Central Limit Theorem

   French mathematician
    Abraham de Moivre
   1733 – approximate
    distribution from tossing
    coin (heads/tails)
   Ho hum reaction
   French Mathematician
    LaPlace – expanded it
   Ho hum reaction

       Managing the business risk of fraud   EZ-R Stats, LLC
History of CLT (cont’d)

   Russian mathematician
    Lyapunov
   Proof in 1901
   Same reaction




      Managing the business risk of fraud   EZ-R Stats, LLC
Industrial revolution

 Manufacturing

 Engineering




 Excitement!



     Managing the business risk of fraud   EZ-R Stats, LLC
Student’s T

William Gosset - 1908


 Guinness Brewery




     Managing the business risk of fraud   EZ-R Stats, LLC
SAS 39

   Effective June, 1983
   Exposure draft for
    revision in 2009




      Managing the business risk of fraud   EZ-R Stats, LLC
Attribute sampling

   Buonaccorsi (1987)
   Refined calculations
   Few software packages use it




      Managing the business risk of fraud   EZ-R Stats, LLC
Overview

   Sample size calculations
   Attribute sampling
   Variable sampling
   Random number generators




      Managing the business risk of fraud   EZ-R Stats, LLC
Sample size calculation

   It’s a guess…
   Every package – different
    answer
   Need to know the
    population
   But that’s why you’re
    taking a sample!



      Managing the business risk of fraud   EZ-R Stats, LLC
                           Attribute Sampling
                                    Using RAT-STATS



                                                         Unrestricted populations




Managing the business risk of fraud using sampling and
data mining                                                              EZ-R Stats, LLC
                    Session Objectives


1.   Understand what is attribute sampling and
     when to use it
2.   Understand unrestricted populations
3.   Overview of the process using RAT-STATS
4.   Understand the formula behind the
     computations



      Managing the business risk of fraud   EZ-R Stats, LLC
Attribute sampling

 “Attribute”
 Compliance  testing
 Signatures on approval
  documents, attachment of
  supporting documentation, etc.

    Managing the business risk of fraud   EZ-R Stats, LLC
Statistical approach

 Recommended
 Economical
 Efficient
 Requiresdetermination of a
 sample size

    Managing the business risk of fraud   EZ-R Stats, LLC
Overview of process

   Determine the sampling objective
    –   Confidence
    –   Precision
   Determine required sample size
   Identify samples to be selected based upon random
    numbers
   Pull the sample and test
   Compute the sampling results (i.e. estimate of
    range)

        Managing the business risk of fraud   EZ-R Stats, LLC
How this is done in RAT-STATS

   The sampling parameters are first developed
    by the auditor
   RAT-STATS is used to compute sample size
   RAT-STATS used to generate random
    numbers
   Pull the sample and test
   Enter results in RAT-STATS to compute
    estimates

      Managing the business risk of fraud   EZ-R Stats, LLC
Step 1 – Develop sampling parameters

1.   Size of population
2.   Expected error rate
3.   Required confidence
4.   Required precision


     Managing the business risk of fraud   EZ-R Stats, LLC
Step 2 – Obtain the random numbers

   Done by entering info into RAT-STATS
   Output can be a variety of sources:
    –   Text File
    –   Excel
    –   Microsoft Access
    –   Print File




        Managing the business risk of fraud   EZ-R Stats, LLC
Step 3 – Pull the sample

   Each random number selected corresponds
    with an item
   Put the selected item on a separate schedule




      Managing the business risk of fraud   EZ-R Stats, LLC
Step 4 - Test each selected item

   Generally requires reviewing documents




      Managing the business risk of fraud   EZ-R Stats, LLC
Step 5 – Compute the results

   Enter summary information into RAT-STATS
   Output can be in a variety of formats:
    –   Excel
    –   Microsoft Access
    –   Text File
    –   Print File
    –   Printer


        Managing the business risk of fraud   EZ-R Stats, LLC
That’s It!

   Now we’ll see an actual demo using the
    RAT-STATS software
   Excel population of 5,000 invoices
   Results of test of attributes stored in the
    worksheet




      Managing the business risk of fraud   EZ-R Stats, LLC
                            Variable Sampling
                                    Using RAT-STATS



                                                         Unrestricted populations




Managing the business risk of fraud using sampling and
data mining                                                              EZ-R Stats, LLC
                    Session Objectives


1.   Understand what variable sampling is
     and when to use it
2.   Understand unrestricted populations
3.   Overview of the process using RAT-
     STATS
4.   Understand the formula behind the
     computations
      Managing the business risk of fraud   EZ-R Stats, LLC
Variable sampling

 “Variable”
 Estimating account balances
 Estimating transaction totals




   Managing the business risk of fraud   EZ-R Stats, LLC
Statistical approach

 Recommended
 Economical
 Efficient
 Requiresdetermination of a
 sample size

    Managing the business risk of fraud   EZ-R Stats, LLC
Overview of process

   Determine the sampling objective
    –   Confidence
    –   Precision
   Determine required sample size
   Identify samples to be selected based upon random
    numbers
   Pull the sample and test
   Compute the sampling results (i.e. estimate of
    range)

        Managing the business risk of fraud   EZ-R Stats, LLC
How this is done in RAT-STATS

   The sampling parameters are first developed
    by the auditor
   RAT-STATS is used to compute sample size
   RAT-STATS used to generate random
    numbers
   Pull the sample and test
   Enter results in RAT-STATS to compute
    estimates

      Managing the business risk of fraud   EZ-R Stats, LLC
Step 1 – Develop sampling parameters


1.   Probe sample
2.   Statistical measure
3.   Excel formula


     Managing the business risk of fraud   EZ-R Stats, LLC
Step 1 – Develop sampling parameters


1.   Size of population
2.   Average value
3.   Standard deviation


     Managing the business risk of fraud   EZ-R Stats, LLC
Step 2 – Obtain the random numbers

 Done by entering info into RAT-STATS
 Output can be a variety of sources:
  – Text File
  – Excel
  – Microsoft Access
  – Print File

    Managing the business risk of fraud   EZ-R Stats, LLC
Step 3 – Pull the sample

 Each  random number selected
  corresponds with an item
 Put the selected item on a
  separate schedule



   Managing the business risk of fraud   EZ-R Stats, LLC
Step 4 - Test each selected item

 Generallyrequires reviewing
  documents
 Example data contains both
  “examined” and “audited” value.



   Managing the business risk of fraud   EZ-R Stats, LLC
Step 5 – Compute the results

   Enter summary information into RAT-STATS
   Output can be in a variety of formats:
    –   Excel
    –   Microsoft Access
    –   Text File
    –   Print File
    –   Printer


        Managing the business risk of fraud   EZ-R Stats, LLC
                             That’s It!

 Now  we’ll see an actual demo
  using the RAT-STATS software
 Excel population of 5,000 invoices
 Audited values stored in the
  worksheet

   Managing the business risk of fraud    EZ-R Stats, LLC
                           Attribute Sampling
                                    Using RAT-STATS



                                                         Stratified populations




Managing the business risk of fraud using sampling and
data mining                                                               EZ-R Stats, LLC
                    Session Objectives


1.   Understand what is stratification and when
     to use it
2.   Overview of the process using RAT-STATS




      Managing the business risk of fraud   EZ-R Stats, LLC
Stratified sampling

 “Strata”
 Homogenous
 More       efficient in some instances




    Managing the business risk of fraud   EZ-R Stats, LLC
Overview of process

   Separation into strata
   Determine the sampling objective
    –   Confidence
    –   Precision
   Determine required sample size
   Identify samples to be selected based upon random
    numbers
   Pull the sample and test
   Compute the sampling results (i.e. estimate of
    range)


        Managing the business risk of fraud   EZ-R Stats, LLC
How this is done in RAT-STATS

   The sampling parameters are first developed
    by the auditor
   RAT-STATS is used to compute sample size
   RAT-STATS used to generate random
    numbers
   Pull the sample and test
   Enter results in RAT-STATS to compute
    estimates

      Managing the business risk of fraud   EZ-R Stats, LLC
Step 1 – Develop sampling parameters

1.   Size of population
2.   Expected error rate
3.   Required confidence
4.   Required precision


     Managing the business risk of fraud   EZ-R Stats, LLC
Step 2 – Obtain the random numbers

   Done by entering info into RAT-STATS
   Output can be a variety of sources:
    –   Text File
    –   Excel
    –   Microsoft Access
    –   Print File




        Managing the business risk of fraud   EZ-R Stats, LLC
Step 3 – Pull the sample

   Each random number selected corresponds
    with an item
   Put the selected item on a separate schedule




      Managing the business risk of fraud   EZ-R Stats, LLC
Step 4 - Test each selected item

   Generally requires reviewing documents




      Managing the business risk of fraud   EZ-R Stats, LLC
Step 5 – Compute the results

   Enter summary information into RAT-STATS
   Output can be in a variety of formats:
    –   Excel
    –   Microsoft Access
    –   Text File
    –   Print File
    –   Printer


        Managing the business risk of fraud   EZ-R Stats, LLC
That’s It!

   Now we’ll see an actual demo using the
    RAT-STATS software
   Excel population of 5,000 invoices
   Results of test of attributes stored in the
    worksheet




      Managing the business risk of fraud   EZ-R Stats, LLC
                            Variable Sampling
                                    Using RAT-STATS



                                                         Stratified populations




Managing the business risk of fraud using sampling and
data mining                                                               EZ-R Stats, LLC
             Session Objectives


1.   Understand what stratified sampling is
     and when to use it
2.   Populations benefiting from stratified
     sampling
3.   Overview of the process using RAT-
     STATS
4.   Understand the formula behind the
     computationsof fraud
      Managing the business risk EZ-R Stats, LLC
Stratified variable sampling

 “Stratified”
 “Variable”
 Estimatingamounts
 Narrower standard deviation



   Managing the business risk of fraud   EZ-R Stats, LLC
Overview of process

   Determine the sampling objective
    –   Confidence
    –   Precision
   Determine required sample size
   Identify samples to be selected based upon random
    numbers
   Pull the sample and test
   Compute the sampling results (i.e. estimate of
    range)

        Managing the business risk of fraud   EZ-R Stats, LLC
How this is done in RAT-STATS

   The sampling parameters are first developed
    by the auditor
   RAT-STATS is used to compute sample size
   RAT-STATS used to generate random
    numbers
   Pull the sample and test
   Enter results in RAT-STATS to compute
    estimates

      Managing the business risk of fraud   EZ-R Stats, LLC
Step 1 – Develop sampling parameters


1.   Probe sample
2.   Statistical measure
3.   Excel formula


     Managing the business risk of fraud   EZ-R Stats, LLC
Step 1 – Develop sampling parameters


1.   Number of strata
2.   Size of population
3.   Average value
4.   Standard deviation

     Managing the business risk of fraud   EZ-R Stats, LLC
Step 2 – Obtain the random numbers

   Done by entering info into RAT-STATS
   Multi-stage random numbers
   Output can be a variety of sources:
    – Text File
    – Excel
    – Microsoft Access
    – Print File


      Managing the business risk of fraud   EZ-R Stats, LLC
Step 3 – Pull the sample

 Each   random number selected
  corresponds with an item in a
  strata
 Put the selected item on a
  separate schedule


   Managing the business risk of fraud   EZ-R Stats, LLC
Step 4 - Test each selected item

 Generallyrequires reviewing
  documents
 Example data contains both
  “examined” and “audited” value.



   Managing the business risk of fraud   EZ-R Stats, LLC
Step 5 – Compute the results

   Enter summary information into RAT-STATS
   Output can be in a variety of formats:
    –   Excel
    –   Microsoft Access
    –   Text File
    –   Print File
    –   Printer


        Managing the business risk of fraud   EZ-R Stats, LLC
                             That’s It!

 Now  we’ll see an actual demo
  using the RAT-STATS software
 Excel population of 5,000 invoices
 Divided into three strata
 Audited values stored in the
  worksheet
   Managing the business risk of fraud    EZ-R Stats, LLC
        Objective 2 - Summarized

     Understand the framework for managing the business risk of
      fraud
     Plan, perform and explain statistical sampling in audits
     Reduce audit costs using data mining, sequential sampling and
      other sampling techniques
     Apply SAS 56, the new SAS suite and the revised (2007)
      Yellow Book.
     Run, hands-on, the most productive analytic technique
      (regression analysis).
     Use data mining to introduce greater efficiency into the audit
      process, without losing effectiveness.

        Next is cost reduction …

Managing the business risk of fraud                  EZ-R Stats, LLC
Techniques for cost reduction

   Optimize sample size
    (most “bang” for the
    buck)
   Skip sampling – review
    100% of transactions
    using computer
    assisted audit
    techniques (CAATs)



      Managing the business risk of fraud   EZ-R Stats, LLC
Sample optimization

   Sequential sampling




      Managing the business risk of fraud   EZ-R Stats, LLC
University of Hawaii

   Banana aphids




      Managing the business risk of fraud   EZ-R Stats, LLC
Sequential sampling

   Banana aphids




      Managing the business risk of fraud   EZ-R Stats, LLC
100% test using CAATs

   Provides complete coverage
   Best practice
   Basis for continuous monitoring
   Repeatable process




      Managing the business risk of fraud   EZ-R Stats, LLC
        Objective 3 - Summarized

     Understand the framework for managing the business risk of
      fraud
     Plan, perform and explain statistical sampling in audits
     Reduce audit costs using data mining, sequential sampling and
      other sampling techniques
     Apply SAS 56, the new SAS suite and the revised (2007)
      Yellow Book.
     Run, hands-on, the most productive analytic technique
      (regression analysis).
     Use data mining to introduce greater efficiency into the audit
      process, without losing effectiveness.




        Next is Yellow Book and
                SAS 56 …
Managing the business risk of fraud                  EZ-R Stats, LLC
Yellow book standards

 Standards regarding statistical sampling and IT




    Managing the business risk of fraud            EZ-R Stats, LLC
General standards

   3.43 Technical Knowledge and competence

“The staff assigned to conduct an audit or attestation
  engagement under GAGAS must collectively
  possess the technical knowledge, skills, and
  experience necessary to be competent for the type
  of work being performed before beginning work on
  that assignment.
The staff assigned to a GAGAS audit or attestation
engagement should collectively possess: “

      Managing the business risk of fraud   EZ-R Stats, LLC
Stat sampling and IT

Skills appropriate for the work being performed.
  For example, staff or specialist skills in

   (1) statistical sampling if the work involves
    use of statistical sampling;

   (2) information technology

      Managing the business risk of fraud   EZ-R Stats, LLC
SAS 56 – Analytical procedures

   Requires use of analytic review procedures
    for:
   Audit planning
   Overall review stages




      Managing the business risk of fraud   EZ-R Stats, LLC
SAS 56 – Analytical Review

   Encourages use of analytical review
   Provides guidance


              “A wide variety of analytical
              procedures may be useful for
              this purpose.”



      Managing the business risk of fraud     EZ-R Stats, LLC
        Objective 4 - Summarized

     Understand the framework for managing the business risk of
      fraud
     Plan, perform and explain statistical sampling in audits
     Reduce audit costs using data mining, sequential sampling and
      other sampling techniques
     Apply SAS 56, the new SAS suite and the revised (2007)
      Yellow Book.
     Run, hands-on, the most productive analytic technique
      (regression analysis).
     Use data mining to introduce greater efficiency into the audit
      process, without losing effectiveness.




                      Next is linear
                      regression …
Managing the business risk of fraud                  EZ-R Stats, LLC
                      Next Metric
        1.      Outliers
        2.      Stratification
        3.      Day of Week
        4.      Round Numbers
        5.      Made Up Numbers
        6.      Market basket
        7.      Trends
        8.      Gaps
        9.      Duplicates
        10.     Dates

Managing the business risk of fraud   EZ-R Stats, LLC
                                                        7 - Trends

                   Trend Busters
       Does the pattern make sense?

                                      ACME Technology

                  30,000
                  25,000
         Amount




                  20,000
                                                             Sales
                  15,000
                  10,000                                     Employee Count
                   5,000
                       0
                                        7




                                        8
                                        7




                                M 8
                                        7
                                Se 7
                         7




                                      08
                                      07
                                      -0




                                      -0
                                     -0




                                     -0
                                     -0
                                    l-0
                        0
                     n-




                                   n-
                                   p-
                                  ay




                                  ay
                             ar




                                  ar
                                  ov
                                 Ju
                    Ja




                                Ja
                            M




                                M
                                  M




                                N




                                        Date




Managing the business risk of fraud                     EZ-R Stats, LLC
                                      7 – Trends
             Trend Busters

      Linear regression
      Sales are up, but cost of goods sold is
       down
      “Spikes”




Managing the business risk of fraud     EZ-R Stats, LLC
                                      7 – Trends

 Purpose / Type of Errors


          Identify trend lines, slopes,
           etc.
          Correlate trends
          Identify anomalies
          Key punch errors where
           amount is order of
           magnitude

Managing the business risk of fraud     EZ-R Stats, LLC
                                      7 – Trends

 Linear Regression

 Test relationships (e.g.
  invoice amount and sales
  tax)
 Perform multi-variable
  analysis

Managing the business risk of fraud     EZ-R Stats, LLC
                                      7 – Trends

                 How is it done?

    Estimate linear trends using “best
     fit”
    Measure variability (standard
     errors)
    Measure slope
    Sort descending by slope,
     variability, etc.

Managing the business risk of fraud     EZ-R Stats, LLC
                                                   7 – Trends
    Trend Lines by Account - Example
                 Results


   Account                N           Slope         Std Err
          32451                  18       1.230               0.87
          43517                  17       1.070                4.3
          32451                  27       1.023               0.85
          43517                  32       1.010               0.36
          43870                  23       0.340               2.36
          54630                  56       -0.560              1.89


Generally the trend is gently sloping
up, but two accounts (43870 and
54630) are different.
Managing the business risk of fraud                 EZ-R Stats, LLC
Scatter plot with prediction and
confidence intervals




Managing the business risk of fraud   EZ-R Stats, LLC
        Objective 5 - Summarized

     Understand the framework for managing the business risk of
      fraud
     Plan, perform and explain statistical sampling in audits
     Reduce audit costs using data mining, sequential sampling and
      other sampling techniques
     Apply SAS 56, the new SAS suite and the revised (2007)
      Yellow Book.
     Run, hands-on, the most productive analytic technique
      (regression analysis).
     Use data mining to introduce greater efficiency into the audit
      process, without losing effectiveness.




           Next is data mining …

Managing the business risk of fraud                  EZ-R Stats, LLC
                                           Objective 6


Basis for Pattern Detection

   Analytical review
   Isolate the “significant few”
   Detection of errors
   Quantified approach




     Managing the business risk of fraud   EZ-R Stats, LLC
                                       Objective 2

    Understanding the Basis

         Quantified Approach
         Population vs. Groups
         Measuring the Difference
         Stat 101 – Counts, Totals, Chi
          Square and K-S
         The metrics used


Managing the business risk of fraud   EZ-R Stats, LLC
                                      Objective 2a


  Quantified Approach

  Based on measureable
   differences
  Population vs. Group
  “Shotgun” technique




Managing the business risk of fraud    EZ-R Stats, LLC
                                              Objective 2a


Detection of Fraud Characteristics

   Something is different than expected




      Managing the business risk of fraud   EZ-R Stats, LLC
                                        Objective 2b


Fraud patterns

   Common theme – “something is
    different”
   Groups
   Group pattern is different than
    overall population


  Managing the business risk of fraud   EZ-R Stats, LLC
                                      Objective 2c


Measurement Basis

      Transaction
       counts
      Transaction
       amounts


Managing the business risk of fraud   EZ-R Stats, LLC
                                      Objective 2d
  A few words about statistics
         (the “s” word)

     Detailed knowledge of statistics not
      necessary
     Software packages do the “number-
      crunching”
     Statistics used only to highlight
      potential errors/frauds
     Not used for quantification


Managing the business risk of fraud    EZ-R Stats, LLC
                                        Objective 2d

How is digital analysis done?

       Comparison of group with population as a
        whole
       Can be based on either counts or amounts
       Difference is measured
       Groups can then be ranked using a selected
        measure
       High difference = possible error/fraud




  Managing the business risk of fraud    EZ-R Stats, LLC
Demo in Excel of the process

   Based roughly on the Wake County
    Transportation fraud
   Illustrates how the process works, using
    Excel




      Managing the business risk of fraud   EZ-R Stats, LLC
                                          Objective 2d

                         Histograms

   Attributes tallied and categorized into “bins”
   Counts or sums of amounts




    Managing the business risk of fraud    EZ-R Stats, LLC
                                                                                                       Objective 2d


           Two histograms obtained

                                  Population and group
                            Population                                                            Group

700                                                                 80
600                                                                 70
500                                                                 60
400                                                                 50
                                                                    40
300
                                                                    30
200
                                                                    20
100                                                                 10
  0                                                                  0
      Jan- Feb- Mar- Apr- May- Jun- Jul- Aug- Sep- Oct- Nov- Dec-        Jan- Feb- Mar- Apr- May- Jun- Jul- Aug- Sep- Oct- Nov- Dec-
       07 07 07 07 07 07 07 07 07 07 07 07                                07 07 07 07 07 07 07 07 07 07 07 07




      Managing the business risk of fraud                                                                 EZ-R Stats, LLC
                                                                  Objective 2d


Compute Cumulative Amount for each


               Count by Month
                                                             Cum Pct
          80
                                       120.0%
          70

          60                           100.0%

          50
                                       80.0%
  Count




          40
                                       60.0%
          30

          20
                                       40.0%
          10
                                       20.0%
          0
          Au 7
          Ju 7
          Fe 7




           Ju 7
          M 07




          O 7


          De 7
                07
          Ap 7




          No 7
          Se 7
          M 7



              l-0
               -0




                0


                0
               -0
                0




                0




                0
                0
             r-0




             p-


             v-
             c-




                                        0.0%
             n-




             n-
             b-




            g-


            ct-
           ay
            ar
          Ja




                                                                       7
                                                             07
                                             07




                                                                               07



                                                                                         7
                                                     07




                                                                    l-0




                                                                                       -0
                                                                             p-
                                                              -
                                           n-


                                                      -
                      Month




                                                                                    ov
                                                           ay
                                                   ar




                                                                  Ju
                                         Ja




                                                                           Se
                                                  M




                                                                                    N
                                                          M



 Managing the business risk of fraud                               EZ-R Stats, LLC
                                         Objective 2d


Are the histograms different?

             Two statistical measures of
              difference
             Chi Squared (counts)
             K-S (distribution)
             Both yield a difference metric




   Managing the business risk of fraud    EZ-R Stats, LLC
                                      Objective 2d

                     Chi Squared


         Classic test on data in a table
         Answers the question – are the
          rows/columns different
         Some limitations on when it can be
          applied




Managing the business risk of fraud    EZ-R Stats, LLC
                                      Objective 2d


                 Chi Squared

             Table of Counts
             Degrees of Freedom
             Chi Squared Value
             P-statistic
             Computationally intensive




Managing the business risk of fraud    EZ-R Stats, LLC
                                        Objective 2d


Kolmogorov-Smirnov



            Two Russian
             mathematicians
            Comparison of distributions
            Metric is the “d-statistic”


  Managing the business risk of fraud    EZ-R Stats, LLC
                                      Objective 2d


    How is K-S test done?

      Four step process
     1.     For each cluster element
            determine percentage
     2.     Then calculate cumulative
            percentage
     3.     Compare the differences in
            cumulative percentages
     4.     Identify the largest difference
Managing the business risk of fraud    EZ-R Stats, LLC
                                      Objective 2d - KS


        Kolmogorov-Smirnov




Managing the business risk of fraud      EZ-R Stats, LLC
                                         Objective 2e


Classification by metrics

        Stratification
        Day of week
        Happens on holiday
        Round numbers
        Variability
        Benford’s Law
        Trend lines
        Relationships (market basket)
        Gaps
        Duplicates



Managing the business risk of fraud      EZ-R Stats, LLC
                                               Objective e


Auditor’s “Top 10” Metrics

              1.      Outliers / Variability
              2.      Stratification
              3.      Day of Week
              4.      Round Numbers
              5.      Made Up Numbers
              6.      Market basket
              7.      Trends
              8.      Gaps
              9.      Duplicates
              10.     Dates


Managing the business risk of fraud            EZ-R Stats, LLC
                                           Objective 2

Understanding the Basis

             Quantified Approach
             Population vs. Groups
             Measuring the Difference
             Stat 101 – Counts, Totals, Chi Square
              and K-S
             The metrics used




   Managing the business risk of fraud    EZ-R Stats, LLC
            Objective 2 - Summarized

1.     Understand why and how
2.     Understand statistical basis for quantifying
       differences
3.     Identify ten general tools and techniques
4.     Understand examples done using Excel
5.     How pattern detection fits in


           Next are the metrics …


 Managing the business risk of fraud           EZ-R Stats, LLC
                  It’s that time!



 Session Break!


Managing the business risk of fraud   EZ-R Stats, LLC
                                           Objective 3

        The “Top 10” Metrics

   Overview
   Explain Each Metric
   Examples of what it can detect
   How to assess results


    Managing the business risk of fraud   EZ-R Stats, LLC
                                          Objective 3


          Trapping anomalies




Managing the business risk of fraud   EZ-R Stats, LLC
                                                           Objective 3


     Fraud Pattern Detection
                              Round Numbers
                  Market Basket          Benford’s Law



           Stratification                          Gaps

                                Target Group

             Trend Line                         Univariate



                      Holiday             Duplicates
                                Day of Week




Managing the business risk of fraud                    EZ-R Stats, LLC
                                      1 - Outliers
    Outliers / Variability

       Outliers are
       amounts which
       are significantly
       different from
       the rest of the
       population
Managing the business risk of fraud   EZ-R Stats, LLC
                                      1 - Outliers
           Outliers / Variability

           Charting (visual)
           Software to analyze “z-scores”
           Top and Bottom 10, 20 etc.
           High and low variability (coefficient
            of variation)




Managing the business risk of fraud       EZ-R Stats, LLC
                                          1 - Outliers

Drill down to the group level

         Basic statistics
           – Minimum, maximum
             and average
           – Variability
         Sort by statistic of interest
           – Variability (coefficient
             of variation)
           – Maximum, etc.


Managing the business risk of fraud       EZ-R Stats, LLC
                                                   1 - Outliers
             Example Results

         Provider                     N           Coeff Var
              3478421                     3,243        342.23
              2356721                     4,536         87.23
              3546789                     3,421         23.25
              5463122                     2,311         18.54

   Two providers (3478421 and
   2356721) had significantly more
   variability in the amounts of their
   claims than all the rest.
Managing the business risk of fraud                      EZ-R Stats, LLC
                      Next Metric

            1.      Outliers
            2.      Stratification
            3.      Day of Week
            4.      Round Numbers
            5.      Made Up Numbers
            6.      Market basket
            7.      Trends
            8.      Gaps
            9.      Duplicates
            10.     Dates

Managing the business risk of fraud   EZ-R Stats, LLC
                                        2 - Stratification
  Unusual stratification
       patterns

                                       Do you
                                      know how
                                      your data
                                        looks?
Managing the business risk of fraud       EZ-R Stats, LLC
                                      2 - Stratification


           Stratification - How

  Charting (visual)
  Chi Squared
  Kolmogorov-Smirnov
  By groups




Managing the business risk of fraud     EZ-R Stats, LLC
                                      2 – Stratification
Purpose / types of errors

   Transactions out of the ordinary
   “Up-coding” insurance claims
   “Skewed” groupings
   Based on either count or amount




Managing the business risk of fraud         EZ-R Stats, LLC
                                      2 – Stratification
        The process?

1.     Stratify the entire population into
       “bins” specified by auditor
2.     Same stratification on each group
       (e.g. vendor)
3.     Compare the group tested to the
       population
4.     Obtain measure of difference for each
       group
5.     Sort descending on difference
       measure

Managing the business risk of fraud         EZ-R Stats, LLC
                                         2 – Stratification
    Units of Service Stratified -
         Example Results

   Provider                 N         Chi Sq      D-stat
     2735211                  6,011      7,453     0.8453
     4562134                  8,913      5,234     0.7453
     4321089                  3,410       342      0.5231
     4237869                  2,503       298      0.4632

   Two providers (2735211 and
   4562134) are shown to be much
   different from the overall population
   (as measured by Chi Square).
Managing the business risk of fraud              EZ-R Stats, LLC
                      Next Metric

               1.      Outliers
               2.      Stratification
               3.      Day of Week
               4.      Round Numbers
               5.      Made Up Numbers
               6.      Market basket
               7.      Trends
               8.      Gaps
               9.      Duplicates
               10.     Dates

Managing the business risk of fraud      EZ-R Stats, LLC
                                      3 – Day of Week

                   Day of Week


            Activity on weekdays
            Activity on weekends
            Peak activity mid to late week




Managing the business risk of fraud     EZ-R Stats, LLC
                                         3 – Day of Week

Purpose / Type of Errors

          Identify unusually high/low
           activity on one or more days of
           week
          Dentist who only handled
           Medicaid on Tuesday
          Office is empty on Friday




   Managing the business risk of fraud        EZ-R Stats, LLC
                 How it is done?

     Programmatically check entire population
     Obtain counts and sums by day of week
      (1-7)
     Prepare histogram
     For each group do the same procedure
     Compare the two histograms
     Sort descending by metric (chi square/d-
      stat)



Managing the business risk of fraud   EZ-R Stats, LLC
                                           3 – Day of Week
   Day of Week - Example Results


    Provider                N           Chi Sq         D-stat
        2735211                 5,404     12,435         0.9802
       4562134                  5,182       7,746        0.8472
       4321089                  5,162            87       0.321
       4237869                  7,905            56      0.2189

Provider 2735211 only provided
service for Medicaid on Tuesdays.
Provider 4562134 was closed on
Thursdays and Fridays.
Managing the business risk of fraud                   EZ-R Stats, LLC
                      Next Metric

           1.      Outliers
           2.      Stratification
           3.      Day of Week
           4.      Round Numbers
           5.      Made Up Numbers
           6.      Market basket
           7.      Trends
           8.      Gaps
           9.      Duplicates
           10.     Dates

Managing the business risk of fraud   EZ-R Stats, LLC
                                        4 – Round Numbers


          Round Numbers


                                      It’s about….
                                        Estimates!



Managing the business risk of fraud          EZ-R Stats, LLC
                                             4 – Round Numbers

        Purpose / Type of Errors

   Isolate estimates
   Highlight account numbers in
    journal entries with round
    numbers
   Split purchases (“under the radar”)
   Which groups have the most
    estimates



       Managing the business risk of fraud         EZ-R Stats, LLC
                                      4 – Round Numbers
            Round numbers

    Classify population amounts
      – $1,375.23 is not round
      – $5,000 is a round number – type 3 (3
        zeros)
      – $10,200 is a round number type 2 (2
        zeros)
    Quantify expected vs. actual (d-statistic)
    Generally represents an estimate
    Journal entries


Managing the business risk of fraud         EZ-R Stats, LLC
                                             4 – Round Numbers
Round Numbers in Journal
Entries - Example Results

   Account                  N           Chi Sq     D-stat
       2735211                  4,136     54,637     0.9802
       4562134                   833      35,324   0.97023
       4321089                  8,318        768      0.321
       4237869                  9,549        546     0.2189

  Two accounts, 2735211 and 4562134
  have significantly more round number
  postings than any other posting
  account in the journal entries.
 Managing the business risk of fraud               EZ-R Stats, LLC
                      Next Metric

            1.      Outliers
            2.      Stratification
            3.      Day of Week
            4.      Round Numbers
            5.      Made Up Numbers
            6.      Market basket
            7.      Trends
            8.      Gaps
            9.      Duplicates
            10.     Dates

Managing the business risk of fraud   EZ-R Stats, LLC
                                      5 – Made up numbers


        Made up Numbers


          Curb stoning
          Imaginary numbers
           Benford’s Law

Managing the business risk of fraud         EZ-R Stats, LLC
                                             5 – Made Up Numbers


            What can be detected
   Made up numbers –
    e.g. falsified inventory
    counts, tax return
    schedules




       Managing the business risk of fraud            EZ-R Stats, LLC
                                      5 – Made Up Numbers


  Benford’s Law using Excel

       Basic formula is “=log(1+(1/N))”
       Workbook with formulae available at
        http://tinyurl.com/4vmcfs
       Obtain leading digits using “Left”
        function, e.g. left(Cell,1)




Managing the business risk of fraud            EZ-R Stats, LLC
                                      5 – Made Up Numbers

            Made up numbers


        Benford’s Law
        Check Chi Square and d-statistic
        First 1,2,3 digits
        Last 1,2 digits
        Second digit
        Sources for more info



Managing the business risk of fraud            EZ-R Stats, LLC
                                      5 – Made Up Numbers
      How is it done?

     Decide type of test – (first 1-3 digits, last
      1-2 digit etc)
     For each group, count number of
      observations for each digit pattern
     Prepare histogram
     Based on total count, compute expected
      values
     For the group, compute Chi Square and
      d-stat
     Sort descending by metric (chi square/d-
      stat)

Managing the business risk of fraud            EZ-R Stats, LLC
                        5 – Made Up Numbers
      Invoice Amounts tested with
     Benford’s law - Example Results
   Store              Hi Digit        Chi Sq      D-stat
           324                   79       5,234      0.9802
           563                   89       4,735     0.97023
           432                   23        476        0.321
           217                   74        312       0.2189


     During tests of invoices by store, two
     stores, 324 and 563 have significantly
     more differences than any other store
     as measured by Benford’s Law.


Managing the business risk of fraud               EZ-R Stats, LLC
                      Next Metric

             1.      Outliers
             2.      Stratification
             3.      Day of Week
             4.      Round Numbers
             5.      Made Up Numbers
             6.      Market basket
             7.      Trends
             8.      Gaps
             9.      Duplicates
             10.     Dates

Managing the business risk of fraud    EZ-R Stats, LLC
                                      6 – Market Basket

                Market Basket


   Medical “Ping ponging”
   Pattern associations
   Apriori program
   References at end of slides
   Apriori – Latin a (from) priori
    (former)
   Deduction from the known

Managing the business risk of fraud        EZ-R Stats, LLC
                                      6 – Market basket

        Purpose / Type of Errors

   Unexpected patterns and
    associations
   Based on “market basket” concept
   Unusual combinations of diagnosis
    code on medical insurance claim




Managing the business risk of fraud         EZ-R Stats, LLC
                                      6 – Market basket
               Market Basket

          JE Accounts
          JE Approvals
          Credit card fraud in Japan –
           taxi and ATM




Managing the business risk of fraud         EZ-R Stats, LLC
                                      6 – Market basket

                 How is it done?

           First, identify groups, e.g. all
            medical providers for a patient
           Next, for each provider, assign a
            unique integer value
           Create a text file containing the
            values
           Run “apriori” analysis


Managing the business risk of fraud         EZ-R Stats, LLC
                                      6 – Market basket

  Apriori outputs

         For each unique value, probability of
          other values
         If you see Dr. Jones, you will also
          see Dr. Smith (80% probability)
         If you see a JE to account ABC, there
          will also an entry to account XYZ
          (30%)



Managing the business risk of fraud         EZ-R Stats, LLC
                      Next Metric

              1.     Outliers
              2.     Stratification
              3.     Day of Week
              4.     Round Numbers
              5.     Made Up Numbers
              6.     Market basket
              7.     Trends
              8.     Gaps
              9.     Duplicates
              10.    Dates

Managing the business risk of fraud    EZ-R Stats, LLC
                                         8 - Gaps

Numeric Sequence Gaps


 What’s there is
 interesting, what’s not
 there is critical …

 Managing the business risk of fraud   EZ-R Stats, LLC
                                          8 – Gaps

Purpose / Type of Errors

    Missing documents (sales, cash,
     etc.)
    Inventory losses (missing receiving
     reports)
    Items that “walked off”




    Managing the business risk of fraud    EZ-R Stats, LLC
                                      8 – Gaps

                 How is it done?

       Check any sequence of numbers
        supposed to be complete, e.g.
       Cash receipts
       Sales slips
       Purchase orders



Managing the business risk of fraud    EZ-R Stats, LLC
                                      8 – Gaps
      Gaps Using Excel

            Excel – sort and check
            Excel formula
            Sequential numbers and dates




Managing the business risk of fraud    EZ-R Stats, LLC
                                                    8 – Gaps

        Gap Testing - Example Results



      Start                           End            Missing

               10789                        10791                      1

               12523                        12526                      2

               17546                        17548                      1


  Four check numbers are missing.



Managing the business risk of fraud                  EZ-R Stats, LLC
                      Next Metric

               1.      Outliers
               2.      Stratification
               3.      Day of Week
               4.      Round Numbers
               5.      Made Up Numbers
               6.      Market basket
               7.      Trends
               8.      Gaps
               9.      Duplicates
               10.     Dates

Managing the business risk of fraud      EZ-R Stats, LLC
                                      9 - Duplicates

                   Duplicates


     Why is there more
        than one?
      Same, Same, Same, and
          Same, Same, Different

Managing the business risk of fraud    EZ-R Stats, LLC
                                            9 – Duplicates

Two types of (related) tests


      Same items – same vendor, same invoice
       number, same invoice date, same amount
      Different items – same employee name,
       same city, different social security number




      Managing the business risk of fraud       EZ-R Stats, LLC
                                      9 - Duplicates


  Duplicate Payments

      High payback area
    “Fuzzy”               logic
      Overriding software
       controls



Managing the business risk of fraud    EZ-R Stats, LLC
                                        9 - Duplicates
Fuzzy matching with
software
         Levenshtein distance
         Soundex
         “Like” clause in SQL                Russian
         Regular expression                  physicist
          testing in SQL
         Vendor/employee
          situations


  Managing the business risk of fraud    EZ-R Stats, LLC
                                      9 - Duplicates


                 How is it done?

           First, sort file in sequence for
            testing
           Compare items in consecutive
            rows
           Extract exceptions for follow-up




Managing the business risk of fraud    EZ-R Stats, LLC
                                                  9 - Duplicates

 Possible Duplicates - Example Results




                                      Invoice
     Vendor          Invoice Date     Amount        Count

          10245          6/15/2007     3,544.78              4

          10245          8/31/2007     2,010.37              2

          17546          2/12/2007     1,500.00              2


Five invoices may be duplicates.

Managing the business risk of fraud                EZ-R Stats, LLC
                      Next Metric

           1.      Outliers
           2.      Stratification
           3.      Day of Week
           4.      Round Numbers
           5.      Made Up Numbers
           6.      Market basket
           7.      Trends
           8.      Gaps
           9.      Duplicates
           10.     Dates

Managing the business risk of fraud   EZ-R Stats, LLC
                                      10 - Dates

            Date Checking


  If we’re closed, why
       is there …
       Adjusting journal entry?
               Receiving report?
                Payment issued?
Managing the business risk of fraud   EZ-R Stats, LLC
                                             10 – Dates

        Holiday Date Testing
   Red Flag indicator




       Managing the business risk of fraud     EZ-R Stats, LLC
                                          10 – Dates
Date Testing challenges

    Difficult to determine
    Floating holidays –
     Friday, Saturday,
     Sunday, Monday




    Managing the business risk of fraud     EZ-R Stats, LLC
                                      10 – Dates
      Typical audit areas

     Journal entries
     Employee expense
      reports
     Business telephone calls
     Invoices
     Receiving reports
     Purchase orders


Managing the business risk of fraud     EZ-R Stats, LLC
                                      10 – Dates

    Determination of Dates

    Transactions when business is
     closed
    Federal Office of Budget
     Management
    An excellent fraud indicator in
     some cases



Managing the business risk of fraud     EZ-R Stats, LLC
                                      10 – Dates
   Holiday Date Testing


      Identifying holiday
       dates:
        – Error prone
        – Tedious
      U.S. only




Managing the business risk of fraud     EZ-R Stats, LLC
                                      10 – Dates
           Federal Holidays

   Established by Law
   Ten dates
   Specific date (unless
    weekend), OR
   Floating holiday




Managing the business risk of fraud     EZ-R Stats, LLC
                                           10 – Dates
Federal Holiday Schedule

    Office of Personnel Management
    Example of specific date – Independence
     Day, July 4th (unless weekend)
    Example of floating date – Martin Luther
     King’s birthday (3rd Monday in January)
    Floating – Thanksgiving – 4th Thursday in
     November




     Managing the business risk of fraud     EZ-R Stats, LLC
                                      10 – Dates
          How it is done?

   Programmatically count holidays for
    entire population
   For each group, count holidays
   Compare the two histograms (group
    and population)
   Sort descending by metric (chi
    square/d-stat)

Managing the business risk of fraud     EZ-R Stats, LLC
                                                10 – Dates

Holiday Counts - Example Results


Employee
 Number                    N           Chi Sq             D-stat
        10245                     37       5,234             0.9802
        32325                     23       4,735            0.97023
        17546                     18            476            0.321
        24135                     34            312          0.2189

  Two employees (10245 and 32325)
  were “off the chart” in terms of
  expense amounts incurred on a
  Federal Holiday.
 Managing the business risk of fraud                  EZ-R Stats, LLC
                                              Objective 3

The “Top 10” Metrics

            Overview
            Explain Each Metric
            Examples of what it can detect
            How to assess results




   Managing the business risk of fraud    EZ-R Stats, LLC
         Objective 3 - Summarized


1.    Understand why and how
2.    Understand statistical basis for quantifying
      differences
3.    Identify ten general tools and techniques
4.    Understand examples done using Excel
5.    How pattern detection fits in


            Next – using Excel …

Managing the business risk of fraud    EZ-R Stats, LLC
                                       Objective 4

    Use of Excel

        Built-in functions
        Add-ins
        Macros
        Database access




Managing the business risk of fraud   EZ-R Stats, LLC
                                       Objective 4

           Excel templates

   Variety of tests
    –   Round numbers
    –   Benford’s Law
    –   Outliers
    –   Etc.




Managing the business risk of fraud   EZ-R Stats, LLC
                                          Objective 4
Excel – Univariate statistics

       Work with Ranges
       =sum, =average, =stdevp
       =largest(Range,1),
        =smallest(Range,1)
       =min, =max, =count
       Tools | Data Analysis | Descriptive
        Statistics

   Managing the business risk of fraud   EZ-R Stats, LLC
                                       Objective 4

              Excel Histograms


    Tools | Data Analysis | Histogram
    Bin Range
    Data Range




Managing the business risk of fraud   EZ-R Stats, LLC
                                       Objective 4

            Excel Gaps testing


        Sort by sequential value
        =if(thiscell-lastcell <>
         1,thiscell-lastcell,0)
        Copy/paste special
        Sort



Managing the business risk of fraud   EZ-R Stats, LLC
                                          Objective 4
Detecting duplicates with Excel

    Sort by sort values
    =if testing
    =if(=and(thiscell=lastcell, etc.))




  Managing the business risk of fraud   EZ-R Stats, LLC
                                           Objective 4
Performing audit tests with macros

          Repeatable process
          Audit standardization
          Learning curve
          Streamlining of tests
          More efficient and effective
          Examples -
           http://ezrstats.com/Macros/home.html



   Managing the business risk of fraud    EZ-R Stats, LLC
                                                  Objective 4
Using database audit software

     Many “built-in” functions right off the shelf
      with SQL
     Control totals
     Exception identification
     “Drill down”
     Quantification
     June 2008 article in the EDP Audit &
      Control Journal (EDPACS) “SQL as an
      audit tool”
     http://ezrstats.com/doc/SQL_As_An_Audit_Tool.pdf

   Managing the business risk of fraud          EZ-R Stats, LLC
                                       Objective 4

              Use of Excel


              Built-in functions
              Add-ins
              Macros
              Database access




Managing the business risk of fraud   EZ-R Stats, LLC
         Objective 4 - Summarized


1.     Understand why and how
2.     Understand statistical basis for quantifying
       differences
3.     Identify ten general tools and techniques
4.     Understand examples done using Excel
5.     How Pattern Detection fits in


                          Next – Fit …
Managing the business risk of fraud     EZ-R Stats, LLC
                                        Objective 5

How Pattern Detection Fits In


            Business Analytics
            Fraud Pattern Detection
            Continuous monitoring




 Managing the business risk of fraud   EZ-R Stats, LLC
                                              Objective 5


Where does Fraud Pattern Detection fit in?



              Right in the middle
             Business Analytics
             Fraud Pattern Detection
             Continuous fraud pattern
              detection
             Continuous Monitoring

    Managing the business risk of fraud   EZ-R Stats, LLC
                                             Objective 5



Business Analytics

        Fraud analytics -> business
         analytics
        Business analytics -> fraud
         analytics




   Managing the business risk of fraud   EZ-R Stats, LLC
                                              Objective 5


Role in Continuous Monitoring (CM)


         Fraud analytics can feed (CM)
         Continuous fraud pattern detection
         Use output from CM to tune fraud
          pattern detection




    Managing the business risk of fraud   EZ-R Stats, LLC
        Objective 6 - Summarized

     Understand the framework for managing the business risk of
      fraud
     Plan, perform and explain statistical sampling in audits
     Reduce audit costs using data mining, sequential sampling and
      other sampling techniques
     Apply SAS 56, the new SAS suite and the revised (2007)
      Yellow Book.
     Run, hands-on, the most productive analytic technique
      (regression analysis).
     Use data mining to introduce greater efficiency into the audit
      process, without losing effectiveness.




Managing the business risk of fraud                  EZ-R Stats, LLC
     Links for more information

   Kolmogorov-Smirnov
   http://tinyurl.com/y49sec
   Benford’s Law http://tinyurl.com/3qapzu
   Chi Square tests http://tinyurl.com/43nkdh
   Continuous monitoring
    http://tinyurl.com/3pltdl




Managing the business risk of fraud   EZ-R Stats, LLC
                       Market Basket


   Apriori testing for “ping ponging”
   Temple University
    http://tinyurl.com/5vax7r
   Apriori program (“open source”)
    http://tinyurl.com/5qehd5
   Article – “Medical ping ponging”
    http://tinyurl.com/5pzbh4


    Managing the business risk of fraud   EZ-R Stats, LLC
Excel macros used in auditing

    Excel as an audit software
     http://tinyurl.com/6h3ye7
    Selected macros -
     http://ezrstats.com/Macros/home.html
    Spreadsheets forever -
     http://tinyurl.com/5ppl7t



 Managing the business risk of fraud   EZ-R Stats, LLC
                 Questions?




Managing the business risk of fraud   EZ-R Stats, LLC
                    Contact info

 Phone:         (919)-219-1622
 E-mail:
  Mike.Blakley@ezrstats.com
 Blog: http://blog.ezrstats.com




   Managing the business risk of fraud   EZ-R Stats, LLC

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:17
posted:6/5/2012
language:
pages:256
shensengvf shensengvf http://
About