Docstoc

Instrument selection980726ppt - Instrument selection reliability

Document Sample
Instrument selection980726ppt - Instrument selection reliability Powered By Docstoc
					Instrument selection: reliability
         and validity

        School of Nursing
             CGU
        Weng, Li-Chueh

                                    1
 The  purpose of measurement is to produce
  trustworthy data that can be used in
  statistic analyses.
 The rule of measurement established for
  research are similar to those used in nursing
  practice .


                                              2
Concepts of measurement theory
 Directness of measurement
 Measurement error
 Levels of measurement




                              3
       Directness of measurement
 To measure, the researcher must clarify
  the object, characteristic, or element to
  be measured.
 Concrete factor: direct measure
 Abstract idea: indirect measure (indicator
  or attributes of the concept)



                                               4
             Measure error
 Measure   error is the difference between the
  true score and what in reality is measured. It
  can exist both in direct and indirect
  measure.
 Random error : difference between obtained
  value and true value is without pattern or
  direction
 Systemic error: with pattern and same
  direction
                                               5
                    Systemic error

                              true score

80
75
70
65
60                                                        true score
55
50
45
40
     40   45   50   55   60       65       70   75   80


                                                                       6
                Random error

                         true score

80
75
70
65
60                                              true score
55
50
45
40
     40   45   50   55          60    65   70


                                                             7
              Random error
 As random errors increase , that the estimation is
  less precise.
 Factors can result RE: transient personal factor,
  Situation factor, variation in measurement
  procedure, Processing of data
 The sum of the random errors is expected to be
  zero.
 The random error is not expected to be correlated
  with the true score.

                                                   8
             Systematic error
 Systemic   error occurs because something
  else is being measured in addition to
  concept.
 Weight scale.
 A close link between the abstract concept
  and instrument can decrease systematic
  error

                                              9
      Reference of measurement
 Referencing   involves comparing a
  subject’s score against a standard.
 Norm-referenced testing: how does the
  average person score on this test?
  “standardization”
 Criterion-referenced testing: what is
  desirable in the perfect subject?
  “comparison of a subject’s score with a
  criterion of achievement”
                                            10
                Reliability
            is concerned with how
 Reliability
  consistently the measurement technique
  measures the concept of interest.
 Consider a measure of the amount of
  random error in the measure technique.
 Correlation coefficient 0~1 (0.8 for well
  developed tool, 0.7 for newly instrument )


                                               11
 Specific  to the sample being tested.
 Reliability testing needs to be performed on
  each instrument used in a study before
  other statistical analyses are performed.
 Stability 穩定性
 Equivalence 對等性
 Homogeneity 同質性




                                                 12
                Stability
 The consistency of repeated measures.
 Test-retest reliability.
 The factor to be measured remains the
  same at the two testing time
 Any change in the value or score is a
  consequence of random error.


                                          13
              Equivalence
 Comparing    measurements made by two or
  more observers measuring the same thing
  inter-rater reliability.
 0.9 is desirable.




                                             14
             Homogeneity
 Address   the correlation of various items
  within the instrument.
 Split-half reliability
 Test-retest reliability
 Internal consistency reliability: examine the
  extent to which all items in the instrument
  consistently measure the construct.
 Cronbach’s alpha coefficient, K-R 20
  (Kuder-Richardson)(0.8~0.9)                 15
                       Reliability
                  Item-total Statistics
                 Scale        Scale       Corrected
         Mean          Variance       Item-          Alpha
           if Item      if Item     Total        if Item
       Deleted        Deleted Correlation            Deleted
SE01     36.0960          30.6782        .6462         .8619
SE02     36.0395          31.8905        .5316         .8694
SE03     35.3277          35.9261        .3242         .8785
SE04     35.3164          35.7402        .3326         .8782
SE05     36.3107          30.8404        .5081         .8736
SE06     35.9548          30.1229        .7240         .8567
SE07     36.1921          30.7697        .5924         .8659
SE09     35.8531          31.4442        .6622         .8614
SE10      36.0791         30.3460        .6587         .8611
SE11     35.7514          32.9606        .6241         .8654
SE12     35.9322          32.4386        .5737         .8668
SE13      36.0113         31.1135        .6687         .8608
                 Reliability Coefficients
  N of Cases = 177.0                    N of Items = 12
                    Alpha = .8767
                                                               16
                  Validity
 The  extent to which the instrument actually
  reflects the abstract construct (or concept)
 Not a “all or nothing” phenomenon but rather
  a matter of degree.
 An instrument may be very valid in one
  situation but not valid in another
 Influenced by systemic error

                                            17
         Content –related validity
 This is the extent to which the measurement
  included all the major elements relevant to
  the concept being measured.
 Literature review, feedback form individual,
  experts
 Content validity index (CVI)



                                             18
請依題目內容的適合度進行評分,若用詞遣字不妥之處,
煩請直接修改於內容上


題 題目內容                   不尚可佳良
數                        佳可3 4 5
                         1 2
1 告訴別人我換腎,我覺得很安心         1 2 3 4 5
2 我覺得我應該隱瞞我有換過腎          1 2 3 4 5
3 我可以接受我有換過腎             1 2 3 4 5
4 我把換過腎這件事看作是我生活中的一部份    1 2 3 4 5
5 我樂觀看待我換過腎這件事           1 2 3 4 5
6 我願意和其他也換過腎的人分享我自己的經驗   1 2 3 4 5
7 我覺得自己比別人差,因為我有換腎       1 2 3 4195
Contrasting groups or known group

 By identifying groups who are expected to
  have contrasting scores on the instrument.
 Hope: newly married individuals and In
  hospitalized suicidal individuals




                                               20
        Convergence validity
 By  comparing the instrument with other
  instruments that measure the same
  concept.
 Administrating all of selected instrument
  concurrently to a sample of subjects
 Correlation between the score of all
  instruments, 0~+1 (positive correlation)

                                              21
          Divergence validity
 Other instrument that measure a concept
  opposite to the newly instrument. Hope /
  hopeless
 Administrating to the same sample of
  subjects
 Correlation -1 to 0 (negative correlation)



                                               22
         Discriminant analysis
 Closely  related to concept. Coping and
  adaptation
 Administrating to the same sample of
  subjects
 Statistical analyses to test the extent to
  which both instruments can discriminate
  finely between these concept

                                               23
       Prediction of future events
                預測效度
 Testing  the ability of the instrument to
  predict future phenomenon or attitude
  based on instrument score
 Health related behavior to predict the future
  health status




                                                  24
     Predicting concurrent events
               同時效度
 Testing the ability of the instrument to
  predict concurrent phenomenon or attitude
  based on instrument score
 Coping and self-esteem. High score of
  coping to predict the self-esteem score.




                                          25
             Factor analysis
 Exploratory  factor analysis (EFA) and
  confirmatory factor analysis (CFA)
 Examine the relationships among the
  various items of the instrument
 Statistical analysis method




                                           26
                            Rotated C omponent Matrix a

                                            Component
               1            2            3           4            5           6
憂 鬱 07          .748    9.212E-02          .292        .143        .201        .120
憂 鬱 01          .743         .122    2.646E-02   -5.02E-02    -1.75E-03        .158
憂 鬱 05          .569    -2.58E-03          .111        .543   -3.45E-03        .158
憂 鬱 11          .563         .305          .308        .218        .122   7.528E-02
憂 鬱 20          .105         .712          .283        .150   -3.38E-02        .231
憂 鬱 17     1.398E-02         .626          .400        .121        .232   -7.83E-04
憂 鬱 18n         .407         .584         -.109  3.694E-02         .152   -8.82E-02
憂 鬱 15     4.115E-02         .583          .231        .508   5.686E-02        .277
憂 鬱 16n         .178         .557    3.586E-02         .169        .179        .240
憂 鬱 10          .186    2.614E-02          .681       -.127   -2.66E-02        .322
憂 鬱 13          .171         .318          .678        .316   -3.27E-02   -3.87E-02
憂 鬱 04          .148         .142          .601  4.264E-02         .410        .260
憂 鬱 19          .417         .272          .486        .379   9.201E-02   2.667E-02
憂 鬱 21     4.620E-02         .161    -1.41E-02         .766   5.619E-02   -5.83E-03
憂 鬱 12          .145         .218          .157        .615   8.505E-02        .381
憂 鬱 06     9.013E-02    3.668E-02    6.583E-02         .199        .786   2.839E-02
憂 鬱 02          .103         .266    -6.68E-02        -.183        .699        .270
憂 鬱 08     6.592E-02         .112          .524  8.713E-02         .602   3.781E-03
憂 鬱 03     5.339E-02    5.244E-02    6.921E-02         .322   7.716E-02        .810
憂 鬱 09          .273         .282          .201       -.122        .139        .542
憂 鬱 14          .376         .185          .218        .132        .209        .475
         on Method: Pri
Ex trac ti                                   s.
                      ncipal Component Analysi
                             th
Rotation Method: Varimax wi Kaiser Norm alization.                              27
    a. Rotation converged in 9 iterations.
      Physiology and biochemical
              measures
 Accuracy    (content-related validity): the
  instrument measures the concept defined in
  the study. It is an evaluation of the
  adequacy of the operational definition.
 Selectivity: is the ability to identify correctly
  signal under study and to distinguish it from
  other signal

                                                  28
 Precision:  the degree of consistence or
  reproducibility of measurements using
  physiology instrument (reliability, stability).
 Manufacturer and quality of control
 Recalibration
 Because of fluctuations in most physiology
  measures, test-retest reliability is
  inappropriate
                                                    29
 Sensitivity:   the amount of change of a
  parameter that can be measured precisely.
 If changes are expected very small, the
  instrument must be highly sensitive
 Stability of instrument




                                              30
 Error
 Environmental   factors such as temperature
 Variation in operation of equipment: user,
  supply, procedure…
 Biological variability: age, gender, body size,
  diurnal rhythms and seasonal cycle
 Other: medication, exercise, emotion

                                               31
        Reliability and Validity
A  measuring procedure can be highly
  reliable but have low validity.
 A measuring procedure that has low
  reliability when the measurement situation
  remains constant cannot have an
  acceptable degree of validity


                                               32
      Measurement strategies
 Physiologic  measurement: quantify the
  changes that occur as a consequence of
  nursing practice (observation)
 Observational measurement: un-structure
  and structure (careful define/ checklist)
 Interviews: un-structure and structure
 Questionnaires: open-end and close-end



                                              33
A smoking questionnaire
 Do you currently smoke cigarettes?
□Yes □No
 How old were you when you started
  smoking?
□under 15 □15 years □16 years
 How many program have you attended to
  help you quit smoking
□none □one □two □three
                                      34
 Fact  about events and situation, belief,
  attitude, opinions, level of knowledge
  etc.
 Scales: precise than questionnaire.
 Summated scales score as a single score
 Rating scale
 Likert scale: agreement response (agree to
  disagree), frequency response (rarely,
  sometime, always)
                                               35
                 A rating scale:
          nurse patient communication
   Nurses come into my room
a. Rarely
b. sometimes
c. whenever I call
d. Frequently
 When a nurse enters my room, I would prefer
    that she
a. Talk very little
b. talk only when necessary
c. talk about causal things
d. be willing to listen or discuss what concerns me
                                                      36
              A Likert scale
 People with cancer almost always die.
□1strongly disagree □2disagree □ 3uncertain □4
  agree □5strongly agree
 Chemotherapy is effective in treating cancer
□strongly disagree □disagree □ uncertain □
  agree □strongly agree
 We are close to finding a cure for cancer
□strongly disagree □disagree □ uncertain □
  agree □strongly agree

                                                 37
         Selecting instrument
 A critical process in study
 Locating an exiting instrument (website
  searching, literature review).
 Must closely fit the conceptual definition of
  variable (concept). Extensive literature review
  is needed
 Evaluating exiting instrument: how the
  instrument was developed? what the
  instrument measure? how to administrate it ?
                                                38
           Constructing scales
 Define the concept
 Design the scale
 Seek item review
 Conduct preliminary item tryouts
 Perform a field test
 Conduct item analyses
 Select items to retain
 Conduct validity studies
 Evaluate the reliability
 Compile norm on the scale          39
 Two  part of item : a leading question and a
  response set
 Leading question should be careful
  designed and clearly expressed
 Response set: every answer need to be
  exhaustive, and mutual exclusive
 Problems: ambiguous, vague meaning,
  assume a pre-exiting state of affair, double
  question, double negative


                                                 40
     出院計畫遵循度-題目藍圖
     健康知 藥物   日常活 營養    合計
     識        動
對內容 3    5    5    5    18
的了解
程度
對內容 2    2    2    2    8
的態度
實際遵 4    10   10   20   34
循情形
合計   9   17   17   17   60
                             41

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:3
posted:5/13/2010
language:English
pages:41