Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out

Discovering Knowledge in Data.ppt

VIEWS: 155 PAGES: 73

									Data Mining Course

         Chapter 1
Introduction to Data Mining



   Data Mining Course, Sharif University of Technology   1
Introduction to Data Mining

• Examples of Data Mining
   – Bank of America
   – 13 million contact Bank of America’s call center each month
   – In past, customers listened to same marketing message
   – Whether relevant to customer or not
   – Kelly, VP Database Marketing states, “...we want to be as
     relevant as possible to each customer”
   – Customer profiles available to service representatives
   – May suggest applicable products or services
   – Data mining helps identify marketing approach based on
     customer’s profile


                  Data Mining Course, Sharif University of Technology   2
Introduction to Data Mining                                            (cont’d)

 – Homeland Security
 – Shortly after 9/11/2001 events, FBI announced identification of
   five terrorists in consumer database records
 – One had 30 credit cards with $250,000 debt
 – Another had 12 different addresses
 – Former President Clinton concluded data should be proactively
   searched
 – Clinton said, “...they have 12 homes, they’re either really rich or
   up to no good...shouldn’t be hard to figure out which.”




                 Data Mining Course, Sharif University of Technology              3
Introduction to Data Mining                                           (cont’d)

 – Gene Expression Database
 – In children, brain tumors represent deadly form of cancer
 – 3,000 cases diagnosed per year
 – Children’s Memorial Hospital building gene expression database
 – Goal is developing more effective treatment
 – Bremer, Director of Brain Research, uses Clementine as initial
   step in tumor identification
 – Classification identifies one of 12 different tumor types




                Data Mining Course, Sharif University of Technology              4
 Trends leading to Data Flood
• More data is generated:
  – Bank, telecom, other
    business transactions ...
  – Scientific data:
    astronomy, biology, etc
  – Web, text, and e-
    commerce




                 Data Mining Course, Sharif University of Technology   5
Big data examples
• Europe's Very Long Baseline Interferometry
  (VLBI) has 16 telescopes, each of which
  produces 1 Gigabit/second of astronomical
  data over a 25-day observation session
  – storage and analysis a big problem
• AT&T handles billions of calls per day
  – so much data, it cannot be all stored -- analysis has
    to be done “on the fly”, on streaming data



               Data Mining Course, Sharif University of Technology   6
Largest databases in 2003
• Commercial databases:
  – Winter Corp. 2003 Survey: France Telecom has
    largest decision-support DB, ~30TB; AT&T ~ 26 TB
• Web
  –   Alexa internet archive: 7 years of data, 500 TB
  –   Google searches 4+ Billion pages, many hundreds TB
  –   IBM WebFountain, 160 TB (2003)
  –   Internet Archive (www.archive.org),~ 300 TB



                Data Mining Course, Sharif University of Technology   7
Data growth rate

• Twice as much information was created in
  2002 as in 1999 (~30% growth rate)
• Other growth rate estimates even higher
• Very little data will ever be looked at by a
  human
• Knowledge Discovery is NEEDED to make
  sense and use of data.

            Data Mining Course, Sharif University of Technology   8
What is Data Mining?
• “…the process of discovering meaningful new correlations, patterns, and
    trends by sifting through large amounts of data…” (Gartner Group)

• “…the analysis of observational data sets to find unsuspected relationships
    and to summarize data in novel ways…” (Hand et al.)

• “Extraction of interesting (non-trivial, implicit, previously unknown and
    potentially useful) patterns or knowledge from huge amount of data”, (Han
    et al.)

• “…is an interdisciplinary field bringing together techniques from machine
    learning, pattern recognition, statistics, databases, and visualization…”
    (Cabana et al.)
•   Alternative names:
      – Knowledge discovery (mining) in databases (KDD), knowledge
        extraction, data/pattern analysis, data archeology, data dredging,
        information harvesting, business intelligence, etc.

                       Data Mining Course, Sharif University of Technology      9
What is Data Mining?                                            (cont’d)


• Data mining chosen as one of top 10 emerging
  technologies   (MIT Technology Review)


• “Data mining expertise is most sought after...” (1999
  Information Week Survey)


• Brown, from BridgeGate LLC said, “Many companies
  have implemented a data warehouse...starting to look at
  what they can do with all that data”



                  Data Mining Course, Sharif University of Technology      10
What is Data Mining?                                             (cont’d)


• How widespread is data mining?
   –   Boston Celtics listed employment position in 12/2003
   –   Statistics Intern: Work with Basketball Operations
   –   “Responsibilities include: ...data mining, etc.”
   –   New York Nicks already using IBM’s Advanced Scout data mining
       software
   –   Software includes NBA’s game data in form of “events”
   –   Each game includes statistics such as shots, passes, points,
       rebounds, etc.
   –   Against Chicago Bulls, software discovered pattern coaching
       staff missed
   –   16 of 29 NBA teams have turned to Advanced Scout to mine
       play-by-play data
                   Data Mining Course, Sharif University of Technology      11
Why Data Mining?

• “...we are drowning in information but starved for
    knowledge.” (Naisbitt, author Megatrends)
•   Not enough trained analysts available to translate data
    into knowledge
•   Data mining fueled by several factors
    – Explosive growth in data collection
    – The storage of enterprise-wide data in data warehouses
    – Increased availability of Web clickstream data
    – The tremendous growth in computing power and storage
      capacity
    – Development of off-the-shelf commercial data mining software
      products
                  Data Mining Course, Sharif University of Technology   12
Potential applications
• Data analysis and decision support
  – Market analysis and management
     • Target marketing, customer relationship management (CRM),
       market basket analysis, cross selling, market segmentation
     • Forecasting, customer retention, improved underwriting,
       quality control, competitive analysis
  – Fraud detection and detection of unusual patterns
• Other applications
  – Text mining (news group, email, documents) and
    Web mining
  – Stream data mining
  – DNA and bio-data analysis

                Data Mining Course, Sharif University of Technology   13
    Market Analysis and Management
•   Where does the data come from?
     –   Credit card transactions, loyalty cards, discount coupons, customer complaint calls,
         plus (public) lifestyle studies
•   Target marketing
     –   Find clusters of “model” customers who share the same characteristics: interest,
         income level, spending habits, etc.
     –   Determine customer purchasing patterns over time
•   Cross-market analysis
     –   Associations/co-relations between product sales, & prediction based on such
         association
•   Customer profiling
     –   What types of customers buy what products (clustering or classification)
•   Customer requirement analysis
     –   identifying the best products for different customers
     –   predict what factors will attract new customers
•   Provision of summary information
     –   multidimensional summary reports
     –   statistical summary information (data central tendency and variation)
                            Data Mining Course, Sharif University of Technology             14
Corporate Analysis & Risk Management
• Finance planning and asset evaluation
   – cash flow analysis and prediction
   – contingent claim analysis to evaluate assets
   – cross-sectional and time series analysis (financial-ratio, trend
     analysis, etc.)
• Resource planning
   – summarize and compare the resources and spending
• Competition
   – monitor competitors and market directions
   – group customers into classes and a class-based pricing
     procedure
   – set pricing strategy in a highly competitive market



                   Data Mining Course, Sharif University of Technology   15
Fraud Detection & Mining Unusual Patterns
• Approaches: Clustering & model construction for frauds, outlier analysis
• Applications: Health care, retail, credit card service, telecomm.
    – Auto insurance: ring of collisions
    – Money laundering: suspicious monetary transactions
    – Medical insurance
       • Professional patients, ring of doctors, and ring of references
       • Unnecessary or correlated screening tests
    – Telecommunications: phone-call fraud
       • Phone call model: destination of the call, duration, time of day or week. Analyze
            patterns that deviate from an expected norm
    – Retail industry
       • Analysts estimate that 38% of retail shrink is due to dishonest employees
    – Anti-terrorism




                         Data Mining Course, Sharif University of Technology                 16
Other Applications
• Sports
    – IBM Advanced Scout analyzed NBA game statistics (shots blocked, assists, and
      fouls) to gain competitive advantage for New York Knicks and Miami Heat
• Astronomy
    – JPL and the Palomar Observatory discovered 22 quasars with the help of data
      mining
• Internet Web Surf-Aid
    – IBM Surf-Aid applies data mining algorithms to Web access logs for market-
      related pages to discover customer preference and behavior pages, analyzing
      effectiveness of Web marketing, improving Web site organization, etc.




                      Data Mining Course, Sharif University of Technology            17
 Data Mining: A KDD Process

   – Data mining—core of        Pattern Evaluation

     knowledge discovery
     process            Data Mining

                    Task-relevant Data


      Data Warehouse                 Selection


Data Cleaning

          Data Integration


        Databases    Data Mining Course, Sharif University of Technology   18
Steps of a KDD Process
• Learning the application domain
     – relevant prior knowledge and goals of application
•   Creating a target data set: data selection
•   Data cleaning and preprocessing: (may take 60% of effort!)
•   Data reduction and transformation
     – Find useful features, dimensionality/variable reduction, invariant
        representation.
•   Choosing functions of data mining
     – summarization, classification, regression, association, clustering.
•   Choosing the mining algorithm(s)
•   Data mining: search for patterns of interest
•   Pattern evaluation and knowledge presentation
     – visualization, transformation, removing redundant patterns, etc.
•   Use of discovered knowledge

                      Data Mining Course, Sharif University of Technology    19
The Need for Human Direction of
Data Mining
 – Some early data mining definitions described process as
   “automatic”
 – “…this has misled many people into believing data mining is
   product that can be bought rather than a discipline that must be
   mastered.” (Berry, Linoff)
 – Automation no substitute for human input
 – Data mining is easy to do badly
 – Understanding statistical and mathematical model structures of
   underlying software required
 – Humans need to be actively involved in every phase of data
   mining process
 – Task of data mining should be integrated into human process of
   problem solving
                Data Mining Course, Sharif University of Technology   20
Cross Industry Standard Process:
CRISP-DM
• Cross-Industry Standard Process for Data Mining (CRISP-
  DM) developed in 1996
   – Contributors include DaimlerChrysler, SPSS, and NCR
   – Developed to fit data mining into general business strategy
   – Process vendor and tool-neutral
   – Non-proprietary and freely available
   – Data mining projects follow iterative, adaptive life cycle
     consisting of 6 phases
   – Phase sequences are adaptive
   – Next, Figure 1.1 illustrates CRISP-DM lifecycle




                  Data Mining Course, Sharif University of Technology   21
  Cross Industry Standard Process:
  CRISP-DM (cont’d)
– Iterative CRIP-DM process
  shown in outer circle

– Most significant                              Business / Research         Data Understanding
  dependencies between                          Understanding Phase               Phase


  phases shown
                                   Deployment Phase                                    Data Preparation
                                                                                            Phase
– Next phase depends on
  results from preceding
  phase                                          Evaluation Phase           Modeling Phase




– Returning to earlier phase
  possible before moving
  forward
                      Data Mining Course, Sharif University of Technology                                 22
Cross Industry Standard Process:
CRISP-DM (cont’d)
• (1) Business Understanding Phase
   – Define business requirements and objectives
   – Translate objectives into data mining problem definition
   – Prepare initial strategy to meet objectives
• (2) Data Understanding Phase
   – Collect data
   – Assess data quality
   – Perform exploratory data analysis (EDA)
• (3) Data Preparation Phase
   – Cleanse, prepare, and transform data set
   – Prepares for modeling in subsequent phases
   – Select cases and variables appropriate for analysis
                  Data Mining Course, Sharif University of Technology   23
Cross Industry Standard Process:
CRISP-DM (cont’d)
• (4) Modeling Phase
   – Select and apply one or more modeling techniques
   – Calibrate model settings to optimize results
   – If necessary, additional data preparation may be required
• (5) Evaluation Phase
   – Evaluate one or more models for effectiveness
   – Determine whether defined objectives achieved
   – Make decision regarding data mining results before deploying to
     field




                  Data Mining Course, Sharif University of Technology   24
Cross Industry Standard Process:
CRISP-DM (cont’d)
• (6) Deployment Phase
   – Make use of models created
   – Simple deployment: generate report
   – Complex deployment: implement additional data mining effort in
     another department
   – In business, customer often carries out deployment based on
     model
• See http://www.crisp-dm.org for more information




                 Data Mining Course, Sharif University of Technology   25
Case Study 1

• Analyzing Automobile Warranty Claims
   – Business Understanding
   – Objectives include improving customer satisfaction and reducing
     costs
   – Manufacturing engineers consulted to formulate business
     problems
   – Data mining techniques used to uncover possible issues:
      • Warranty claim interdependencies?
      • Past claims associated with future claims?
      • Association between claim and repair facility?



                  Data Mining Course, Sharif University of Technology   26
Case Study 1                   (cont’d)

 – Data Understanding
 – 40GB QUIS database containing 7 million vehicle records used
 – Vehicle records include manufacturing location, warrant claims,
   and additional codes
 – Database unintelligible to non-domain experts
 – Costly effort to consult with domain experts from different
   departments
 – Data Preparation
 – QUIS discovered to have limited SQL access
 – Cases and variables manually extracted
 – Additional variables derived for modeling phase


                Data Mining Course, Sharif University of Technology   27
Case Study 1                   (cont’d)

 – Proprietary data mining software used
 – Data format requirements varied for different algorithms
 – Resulted in exhaustive pre-processing of data
 – Modeling Phase
 – Applied Bayesian networks and association rules to uncover
   dependencies between warranty claims
 – Discovered specific combination of construction specifications
   doubles probability of electrical cable claim
 – Investigated whether some garages had more claims than
   others
 – Remaining results confidential


                Data Mining Course, Sharif University of Technology   28
Case Study 1                   (cont’d)

 – Evaluation
 – Researchers disappointed in results
 – Association rules could not be generalized
 – Rules “not interesting” according to domain experts
 – Data models fell short of business objectives
 – Legacy databases not suited to data mining
 – Proposal suggested database redesign for future data mining
   efforts
 – Deployment
 – Foregoing effort identified as pilot project, models not deployed
 – Future data mining efforts planned to integrate more closely to
   database systems at DaimlerChrysler

                Data Mining Course, Sharif University of Technology   29
Case Study 1                   (cont’d)

 – Summary
 – Uncovering hidden nuggets very difficult
 – During each phase, researchers encountered roadblocks
 – Applying new data mining effort problematic
 – Data mining effort requires management support
 – Substantial human participation required at every stage
 – Installation, configuration, and data mining modeling not magic
 – Wrong analysis leads to possibly expensive policy
   recommendations
 – No guarantee data mining effort delivers actionable results
 – However, used properly, data mining may provide profitable
   results

                Data Mining Course, Sharif University of Technology   30
Fallacies of Data Mining

• Four Fallacies of Data Mining (Louie, Nautilus Systems, Inc.)
   –   Fallacy 1
   –   Set of tools can be turned loose on data repositories
   –   Finds answers to all business problems
   –   Reality 1
   –   No automatic data mining tools solve problems
   –   Rather, data mining is process (CRISP-DM)
   –   Integrates into overall business objectives

   – Fallacy 2
   – Data mining process is autonomous
   – Requires little oversight

                    Data Mining Course, Sharif University of Technology   31
Fallacies of Data Mining                                                (cont’d)

  –   Reality 2
  –   Requires significant intervention during every phase
  –   After model deployment, new models require updates
  –   Continuous evaluative measures monitored by analysts

  –   Fallacy 3
  –   Data mining quickly pays for itself
  –   Reality 3
  –   Return rates vary
  –   Depending on startup, personnel, data preparation costs, etc.

  – Fallacy 4
  – Data mining software easy to use
                  Data Mining Course, Sharif University of Technology              32
Fallacies of Data Mining                                                 (cont’d)

   – Reality 4
   – Ease of use varies across projects
   – Analysts must combine subject matter knowledge with specific
     problem domain

• Two Additional Fallacies (Larose)
   –   Fallacy 5
   –   Data mining identifies causes of business problems
   –   Reality 5
   –   Knowledge discovery process uncovers patterns of behavior
   –   Humans interpret results and identify causes



                   Data Mining Course, Sharif University of Technology              33
Fallacies of Data Mining                                                (cont’d)

  –   Fallacy 6
  –   Data mining automatically cleans data in databases
  –   Reality 6
  –   Data mining often uses data from legacy systems
  –   Data possibly not examined or used in years
  –   Organizations starting data mining efforts confronted with huge
      data preprocessing task




                  Data Mining Course, Sharif University of Technology              34
What Tasks Can Data Mining
Accomplish?
• Six common data mining tasks
   –   Description
   –   Estimation
   –   Prediction
   –   Classification
   –   Clustering
   –   Association

• (1) Description
   – Describes patterns or trends in data
   – For example, pollster may uncover patterns suggesting those
     laid-off less likely to support incumbent
   – Descriptions of patterns, often suggest possible explanations


                        Data Mining Course, Sharif University of Technology   35
What Tasks Can Data Mining
Accomplish? (cont’d)
 – For example, those laid-off now less financially secure;
   therefore, prefer alternate candidate
 – Data mining models should be transparent
 – That is, results should be interpretable by humans
 – Some data mining methods more transparent than others
 – For example, Decision Trees (transparent) <-> Neural Networks
   (opaque)
 – High-quality description accomplished using Exploratory Data
   Analysis (EDA)
 – Graphical method of exploring patterns and trends in data




               Data Mining Course, Sharif University of Technology   36
What Tasks Can Data Mining
Accomplish? (cont’d)
• (2) Estimation
   – Similar to Classification task, except target variable numeric
   – Models built from complete data records
   – Records include values for each predictor field and numeric
     target variable in training set
   – For new observations, estimate of target variable made

   – For example, estimate a patient’s systolic blood pressure, based
     on patient’s age, gender, body-mass index, and sodium levels

   – Here, estimation model built from training set records
   – Model then estimates value for new case


                   Data Mining Course, Sharif University of Technology   37
What Tasks Can Data Mining
Accomplish? (cont’d)
 – Estimation Tasks in Business and Research:
 – Estimate amount of money, family of four will spend on back-to-
   school shopping

 – Estimate percentage decrease in rotary movement sustained to
   NFL player with knee injury

 – Estimate number of points basketball player scores when
   double-teamed in playoffs

 – Estimate GPA of graduate student, based on student’s
   undergraduate GPA


                Data Mining Course, Sharif University of Technology   38
What Tasks Can Data Mining
Accomplish? (cont’d)
 – Figure 1.2 shows scatter plot of graduate GPA against
   undergraduate GPA




 – Linear regression finds line (blue) best approximating
   relationship between two variables
 – Regression line estimates student’s graduate GPA based on their
   undergraduate GPA

                Data Mining Course, Sharif University of Technology   39
What Tasks Can Data Mining
Accomplish? (cont’d)
 – Minitab statistical software produces regression
   equation ŷ = 1.24 + 0.67x
 – Therefore, estimated student’s graduate GPA = 1.24 plus 0.67
   times their undergraduate GPA

 –   For example, suppose student’s undergraduate GPA = 3.0
 –   According to estimation model
 –   Estimated student’s graduate GPA = 1.24 + 0.67(3.0) = 3.25
 –   Point (x = 3.0, ŷ = 3.25) lies on regression line

 – Statistical Analysis uses several estimation methods: point
   estimation, confidence interval estimation, linear regression and
   correlation, and multiple regression

                 Data Mining Course, Sharif University of Technology   40
What Tasks Can Data Mining
Accomplish? (cont’d)
• (3) Prediction
   – Similar to classification and estimation, except results lie in the
     future
   – Prediction Tasks in Business and Research:
   – Predict price of stock 3 months into future, based on past
     performance

                                                                         ?


               Stock                                                     ?
               Price


                                                                         ?
                          Q1            Q2            Q3            Q4

                   Data Mining Course, Sharif University of Technology       41
What Tasks Can Data Mining
Accomplish? (cont’d)
 – Predict percentage increase in traffic deaths next year, if speed
   limit increased

 – Predict whether molecule in newly discovered drug leads to
   profitable pharmaceutical drug

 – Methods used for classification and estimation applicable to
   prediction
 – Includes point estimation, confidence interval estimation, linear
   regression and correlation, and multiple regression




                Data Mining Course, Sharif University of Technology    42
What Tasks Can Data Mining
Accomplish? (cont’d)
• (4) Classification
   – Classification requires categorical target variable such as Income
     Bracket
   – Three values include “High”, “Middle”, “Low”
   – Data model examines records containing input fields and target
     field
   – Table shows several records from data set

               Subject     Age      Gender            Occupation               Income Bracket
               001         47      F           Software Engineer          High
               002         28      M           Marketing Consultant       Middle
               003         35      M           Unemployed                 Low
               …           …       …           …                          …




                         Data Mining Course, Sharif University of Technology                    43
What Tasks Can Data Mining
Accomplish? (cont’d)
 – Records of persons in data set used to “train” classification
   model
 – First, Model built from data records, where value of categorical
   target variable (Income Bracket) already known
 – Algorithm “first learns about” which combinations of input fields
   are associated with Income Bracket values in training set
 – For example, algorithm may determine that older females
   associated with high income

 – Next, trained model examines new records
 – Information regarding Income Bracket not available



                Data Mining Course, Sharif University of Technology   44
What Tasks Can Data Mining
Accomplish? (cont’d)
 – Based on classifications in training set, new records classified
 – For example, 63-year old female professor might be classified in
   “High” income bracket

 – Classification Tasks in Business and Research:
 – Determine whether credit card transaction fraudulent

 – Assessing mortgage application to determine “good” or “bad”
   credit risk

 – Diagnosing whether particular disease present


                Data Mining Course, Sharif University of Technology   45
What Tasks Can Data Mining
Accomplish? (cont’d)
 – Determine if will written by deceased, or fraudulently by
   someone else
 – Identify whether certain financial behavior represents terrorist
   threat




 – Scatter plot shows Na/K ratio against Age for 200 patients
 – For example, classify drug type to prescribe based on patient’s
   age and sodium/potassium ratio

                Data Mining Course, Sharif University of Technology   46
What Tasks Can Data Mining
Accomplish? (cont’d)
 – Actual drug type prescribed symbolized by shade (light, medium,
   dark) of points
 – Suppose prescription of new patient based on this data set?
 – Prescribe which drug for young patient with high Na/K ratio?
 – Young patients plotted on left
 – High Na/K plotted on upper-half
 – Quadrant of graph shows light points
 – Recommended drug = Y (corresponds to light points)
 – Prescribe which drug for older patient with low Na/K ratio?
 – Lower-right half of graph shows patients prescribed different
   drug types


                Data Mining Course, Sharif University of Technology   47
What Tasks Can Data Mining
Accomplish? (cont’d)
 – Definitive classification cannot be made
 – More information required to make decision

 – Examples show graphs are helpful for understanding two-
   dimensional data
 – However, classification often requires many input attributes
 – More sophisticated methods of classification required
 – Commonly used algorithms for classification include k-Nearest
   Neighbor, Decision Trees, and Neural Networks




                Data Mining Course, Sharif University of Technology   48
What Tasks Can Data Mining
Accomplish? (cont’d)
• (5) Clustering
   – Refers to grouping records into classes of similar objects
   – Clustering algorithm seeks to segment data set into
     homogeneous subgroups
   – Where similarity of records in clusters maximized, and similarity
     to records outside clusters minimized
   – Target variable not specified

   – For example, Claritas, Inc. PRIZM software clusters demographic
     profiles for different geographic areas according to zip code




                   Data Mining Course, Sharif University of Technology   49
What Tasks Can Data Mining
Accomplish? (cont’d)
 – Table shows 62 distinct “lifestyle” types used by PRIZM
  01 Blue Blood Estates    02 Winner's Circle            03 Executive Suites     04 Pools & Patios

  05 Kids & Cul-de-Sacs    06 Urban Gold Coast           07 Money & Brains       08 Young Literati

  09 American Dreams       10 Bohemian Mix               11 Second City Elite    12 Upward Bound

  13 Gray Power            14 Country Squires            15 God's Country        16 Big Fish, Small Pond

  17 Greenbelt Families    18 Young Influentials         19 New Empty Nests      20 Boomers & Babies

  21 Suburban Sprawl       22 Blue-Chip Blues            23 Upstarts & Seniors   24 New Beginnings

  25 Mobility Blues        26 Gray Collars               27 Urban Achievers      28 Big City Blend

  29 Old Yankee Rows       30 Mid-City Mix               31 Latino America       32 Middleburg Managers

  33 Boomtown Singles      34 Starter Families           35 Sunset City Blues    36 Towns & Gowns

  37 New Homesteaders      38 Middle America             39 Red, White & Blues   40 Military Quarters

  41 Big Sky Families      42 New Eco - topia            43 River City, USA      44 Shotguns & Pickups

  45 Single City Blues     46 Hispanic Mix               47 Inner Cities         48 Smalltown Downtown

  49 Hometown Retired      50 Family Scramble            51 Southside City       52 Golden Ponds

  53 Rural Industria       54 Norma Rae-Ville            55 Mines & Mills        56 Agri - Business

  57 Grain Belt            58 Blue Highways              59 Rustic Elders        60 Back Country Folks

  61 Scrub Pine Flats      62 Hard Scrabble

                          Data Mining Course, Sharif University of Technology                              50
What Tasks Can Data Mining
Accomplish? (cont’d)
 – What do the clusters mean?
 – According to PRIZM, Clusters for Beverly Hills, CA 90210 include:

     •   Cluster 01:        Blue Blood Estates
     •   Cluster 10:        Bohemian Mix
     •   Cluster 02:        Winner’s Circle
     •   Cluster 08:        Young Literati


 – Description of Cluster 01, “...’old money’ heirs that live in
   America’s wealthiest suburbs...accustomed to privilege and live
   luxuriously...”



                   Data Mining Course, Sharif University of Technology   51
What Tasks Can Data Mining
Accomplish? (cont’d)
 – Clustering Tasks in Business and Research:
 – Target marketing niche product for small business that does not
   have large marketing budget
 – For accounting purposes, segment financial behavior into benign
   and suspicious categories
 – Use as dimensionality-reduction tool for data set having several
   hundred inputs
 – Determine gene expression clusters, where many genes exhibit
   similar behavior or characteristics

 – Clustering often used as preliminary step in data mining
 – Resulting clusters used as input to different technique
   downstream, such as neural networks
                Data Mining Course, Sharif University of Technology   52
What Tasks Can Data Mining
Accomplish? (cont’d)
• (6) Association
   – Find out which attributes “go together”
   – Market Basket Analysis commonly used in business applications
   – Quantify relationships in the form of Rules
                     IF antecedent THEN consequent
   – Rules measured using support and confidence
   – For example, discover which items in supermarket are purchased
     together
   – Thursday night 200 of 1,000 customers bought diapers, and of
     those buying diapers, 50 purchased beer
   – Association Rule: “IF buy diapers, THEN buy beer”
   – Support = 200/1,000 = 5%, and confidence = 50/200 = 25%
                 Data Mining Course, Sharif University of Technology   53
What Tasks Can Data Mining
Accomplish? (cont’d)
 – Association Tasks in Business and Research:
 – Investigating proportion of subscribers to cell phone plan
   responding positively to service upgrade offer
 – Predicting degradation in telecommunication networks
 – Discovering which items in supermarket purchased together
 – Determining proportion of cases where administering new drug
   exhibits serious side effects

 – Two commonly-used algorithms for generating association rules
 – A Priori and Generalized Rule Induction (GRI)




               Data Mining Course, Sharif University of Technology   54
Case Study 2

• Predicting Abnormal Stock Market Returns
   – Business Understanding
   – Alan Safer reports stock market trades by insiders have
     abnormal returns
   – Profits by outsiders can be increased, by using legal insider
     trading information
   – Safer attempts to predict abnormal stock price returns
   – Data Preparation
   – Rank of company insiders not considered important
   – Also, company insiders omitted where not involved in company
     decisions


                  Data Mining Course, Sharif University of Technology   55
Case Study 2                      (cont’d)

 –   Modeling
 –   Data split training = 80% and validation = 20%
 –   Neural Network model uncovered results:
 –   (a) Several groups had most predictable abnormal returns:
      •   Electronic equipment, excluding computer equipment
      •   Chemical products
      •   Transportation equipment
      •   Business services
 – (b) Predictions looking farther into future increased ability to
   predict unusual variations
 – (c) Abnormal stock returns of small companies easier to predict


                   Data Mining Course, Sharif University of Technology   56
Case Study 2                   (cont’d)

 – Evaluation
 – Multivariate Adaptive Regression Spline (MARS) also applied to
   data
 – Uncovered similar findings to Neural Network model
 – Confluence of results is powerful method of evaluating validity of
   model
 – This increases confidence in results
 – Deployment
 – Safer published findings in Intelligent Data Analysis




                Data Mining Course, Sharif University of Technology   57
Case Study 3

• Mining Association Rules from Legal Databases
   – Business Understanding
   – Ivkovic, Yearwood, and Stranieri mine association rules from
     large database of applicants for government-funded legal aide in
     Australia
   – Legal data highly unstructured
   – Goal is to improve delivery of legal services
   – Data Understanding
   – Data provided by Victoria Legal Aid (VLA)
   – Contains 380,000 applications with 300 attributes



                  Data Mining Course, Sharif University of Technology   58
Case Study 3                   (cont’d)

 – Domain experts consulted in effort to reduce dimensionality
 – Researchers selected seven of the most important inputs for
   inclusion in data set
 – Data Preparation
 – VLA data set relatively clean
 – VLA database administration system responsible for high-quality
   data
 – Modeling
 – Rules restricted to having both single antecedent and
   consequent
 – Many interesting and uninteresting rules uncovered


                Data Mining Course, Sharif University of Technology   59
Case Study 3                      (cont’d)

 – Researchers adopted premise that interesting rules spawn
   interesting hypotheses
 – For example, possible reasons for rule “If place of birth =
   Vietnam, then law type = criminal law” include:
     • Vietnamese applicants applied for criminal law assistance only
     • Vietnamese applicants committed more crimes than other groups
     • Perhaps high proportion of males applied, and males more closely
         associated with criminal activity
     •   Vietnamese didn’t have access to VLA promotional material
 – Researchers concluded first hypothesis most likely
 – Note intense human activity required for data mining process



                   Data Mining Course, Sharif University of Technology    60
Case Study 3                   (cont’d)

 – Evaluation
 – Three domain experts estimated confidence level of 144 rules
 – These estimates compared to confidence level reported by
   software
 – Deployment
 – Web-based application developed (WebAssociator)
 – Non-specialists able to access rule-building engine
 – Researchers suggest WebAssociator deployment to enhance
   judicial system
 – May identify unjust processes




                Data Mining Course, Sharif University of Technology   61
Case Study 4

• Predicting Corporate Bankruptcies using Decision Trees
   – Business Understanding
   – Recent economic crisis has spawned many corporate
     bankruptcies in East Asia
   – Sung, Chang, and Lee developing models predicting
     bankruptcies that maximize interpretability of results
   – Therefore, decision trees used for modeling
   – Data Understanding
   – Data included two groups of Korean companies
   – Those that went bankrupt 1991-1995 during stable period
   – Companies that went bankrupt during crisis years 1997-1998


                 Data Mining Course, Sharif University of Technology   62
Case Study 4                     (cont’d)

 – 29 firms identified, mostly manufacturing
 – Financial data collected from Korean Stock Exchange
 – Data Preparation
 – 56 financial ratios identified through literature search
 – 16 dropped due to duplication
 – Measures included growth, profitability, etc.
 – Modeling
 – Separate decision tree models were applied under “normal” and
   “crisis” conditions
 – Normal-conditions rules uncovered:
     • If productivity of capital > 19.65, then predict non-bankrupt with
       86% confidence

                  Data Mining Course, Sharif University of Technology       63
Case Study 4                       (cont’d)

     • If productivity of capital <= 19.65 and ratio of cash flow to total
         assets <= -5.65, then predict bankrupt with 84% confidence
 – Crisis-conditions rules uncovered:
     • If productivity of capital > 20.61, predict non-bankrupt with 95%
         confidence
     •   If ratio of cash flow to liabilities > 2.54, predict non-bankrupt with
         85% confidence
 – “Cash flow” and “productivity of capital” important predictors,
   regardless of economic conditions
 – Evaluation
 – Panel of domain experts concluded “productivity of capital” most
   important attribute for differentiating firms at risk
 – Domain experts verified results of decision tree

                    Data Mining Course, Sharif University of Technology           64
Case Study 4                   (cont’d)

 – Group confirmed results would generalize to population of
   Korean manufacturing firms
 – Discriminant analysis determined many of the 40 financial ratios
   were important predictors
 – Deployment
 – No specific deployment took place
 – However, financial institutions in Korea became more aware of
   important predictors of bankruptcy




                Data Mining Course, Sharif University of Technology   65
Case Study 5

• Profiling the Tourism Market using k-Means Clustering
  Analysis
   – Business Understanding
   – Hudson and Richie were interested in studying intra-province
     tourism behavior in Alberta, Canada
   – Goal was development of marketing campaign for tourism in
     Alberta (sponsored by Travel Alberta)
   – Models created in effort to quantify factors for choosing
     vacations in Alberta
   – Data Understanding
   – Data collected from 13,445 Albertans using phone survey in
     1999
   – Only 3,071/13,445 records included in modeling
                  Data Mining Course, Sharif University of Technology   66
Case Study 5                     (cont’d)

 – Data Preparation
 – One question asked respondents to indicate which of 13 factors
   most influenced their travel decisions
 – Factors included accommodations, weather conditions, etc.
 – Modeling
 – Between two and six clusters explored with k-Means
 – Five-cluster solution chosen with profile names:
    •   Young Urban Outdoor Market
    •   Indoor Leisure Traveler Market
    •   Children-first Market
    •   Fair-weather-friends Market
    •   Older, Cost-conscious Traveler Market

                  Data Mining Course, Sharif University of Technology   67
Case Study 5                   (cont’d)

 – Evaluation
 – Discriminant analysis verified “reality” of clusters
 – Classified 93% correctly
 – Deployment
 – Findings resulted in launch of new campaign, “Alberta, Made to
   Order”
 – More than 80 projects launched
 – Travel Alberta found increase of 20% in number of Albertans
   considering Alberta “top-of-the-mind” travel destination




                Data Mining Course, Sharif University of Technology   68
Summary
• Data mining: discovering interesting patterns from large amounts of data
• A natural evolution of database technology, in great demand, with wide
   applications
• A KDD process includes data cleaning, data integration, data selection,
   transformation, data mining, pattern evaluation, and knowledge
   presentation
• Mining can be performed in a variety of information repositories
• Data mining functionalities: characterization, discrimination, association,
   classification, clustering, outlier and trend analysis, etc.
• Data Mining tasks
• Case studies


                       Data Mining Course, Sharif University of Technology      69
Readings
• Chapter 1 Larose book
• Chapters 1 Han and Kamber book
• Intro to Data Mining and Knowledge
  Discovery article
• DataMining-1.html
• From Data Mining to Knowledge Discovery
  in Databases article
• Data Mining an overview article
           Data Mining Course, Sharif University of Technology   70
A Brief History of Data Mining Society

• 1989 IJCAI Workshop on Knowledge Discovery in Databases (Piatetsky-Shapiro)
    –   Knowledge Discovery in Databases (G. Piatetsky-Shapiro and W. Frawley, 1991)

• 1991-1994 Workshops on Knowledge Discovery in Databases
    –   Advances in Knowledge Discovery and Data Mining (U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R.
        Uthurusamy, 1996)

• 1995-1998 International Conferences on Knowledge Discovery in Databases and Data
   Mining (KDD’95-98)
    –   Journal of Data Mining and Knowledge Discovery (1997)

• 1998 ACM SIGKDD, SIGKDD’1999-2001 conferences, and SIGKDD Explorations
• More conferences on data mining
    –   PAKDD (1997), PKDD (1997), SIAM-Data Mining (2001), (IEEE) ICDM (2001), etc.




                            Data Mining Course, Sharif University of Technology                              71
Where to Find References?
• Data mining and KDD (SIGKDD: CDROM)
    –   Conferences: ACM-SIGKDD, IEEE-ICDM, SIAM-DM, PKDD, PAKDD, etc.
    –   Journal: Data Mining and Knowledge Discovery, KDD Explorations
• Database systems (SIGMOD: CD ROM)
    –   Conferences: ACM-SIGMOD, ACM-PODS, VLDB, IEEE-ICDE, EDBT, ICDT, DASFAA
    –   Journals: ACM-TODS, IEEE-TKDE, JIIS, J. ACM, etc.
• AI & Machine Learning
    –   Conferences: Machine learning (ML), AAAI, IJCAI, COLT (Learning Theory), etc.
    –   Journals: Machine Learning, Artificial Intelligence, etc.
• Statistics
    –   Conferences: Joint Stat. Meeting, etc.
    –   Journals: Annals of statistics, etc.
• Visualization
    –   Conference proceedings: CHI, ACM-SIGGraph, etc.
    –   Journals: IEEE Trans. visualization and computer graphics, etc.




                               Data Mining Course, Sharif University of Technology      72
    Recommended Reference Books
•   R. Agrawal, J. Han, and H. Mannila, Readings in Data Mining: A Database Perspective, Morgan
    Kaufmann (in preparation)
•   U. M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy. Advances in Knowledge Discovery
    and Data Mining. AAAI/MIT Press, 1996
•   U. Fayyad, G. Grinstein, and A. Wierse, Information Visualization in Data Mining and Knowledge
    Discovery, Morgan Kaufmann, 2001
•   J. Han and M. Kamber. Data Mining: Concepts and Techniques. Morgan Kaufmann, 2001
•   D. J. Hand, H. Mannila, and P. Smyth, Principles of Data Mining, MIT Press, 2001
•   T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference,
    and Prediction, Springer-Verlag, 2001
•   T. M. Mitchell, Machine Learning, McGraw Hill, 1997
•   G. Piatetsky-Shapiro and W. J. Frawley. Knowledge Discovery in Databases. AAAI/MIT Press, 1991
•   S. M. Weiss and N. Indurkhya, Predictive Data Mining, Morgan Kaufmann, 1998
•   I. H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques with Java
    Implementations, Morgan Kaufmann, 2001

                                 Data Mining Course, Sharif University of Technology                     73

								
To top