L-4b Chapter 8
~ Measuring ~
• Measurement basics
Main Ideas • Measurement valid and not valid
• Measurement accurate not
accurate
• Improving reliability and
reducing bias
1
Measurement basics
• We measure a property of a person or thing when
we assign a number to represent the property.
• We often use an instrument to make a
measurement. We may have a choice of the units
we use to record the measurements.
• the result of measurement is a numerical variable
that takes different values for people or things that
differ in whatever we are measuring.
2
Examples
• Measure the length of my bed: use a tape measure
as the instrument.
• Measure student’s readiness for college: use the
SAT (Standardized Achievement Test) score:
the variable is the student’s score.
• How we measure the safety of traveling on the
highway: count the number of people die in motor
vehicle accidents .
3
Measuring unemployment
• Unemployment rate = # people unemployed/# people in
the labor force.
• The slide below shows the unemployment rate from
August 1991 to July 1994. The gap shows the effect of a
change in how the government measures unemployment.
•
4
5
Valid measure
• A variable is a valid measure of a property
if it is relevant or appropriate as a
representation of that property
• It is valid to measure length with a tape
measure. It is not valid to measure a
student’s readiness for college by recording
her height.
6
Examples
• Example. Does the length of the \life line" on your palm
measure your life expectancy? No. A British study showed
\no correlation" between the two. The length of the life line
is not a valid measure of life expectancy. Note that the
study showed an absence of predictive validity.
• Example. A news article says that the percent of students
in the New York City public schools who read at or above
their grade level has increased. Surely this means student
reading performance is improving? Sorry |not a valid
measure. The school system tightened standards for
promotion from grade to grade, holding the poorest readers
in lower grades. By changing the grade in which students
are enrolled, test scores for a given grade can be changed at
will. A valid measure is reading score by age, not by grade.
7
Rates and counts
• Often a rate is fraction, proportion, or
percent at which something occurs id a
more valid measure than a simple count of
occurrences.
• Rather than count we should use a rate
• Death rate= motor vehicle deaths / 100s of
miles driven=41471 / 26190
8
Predictive validity
• A measurement of a property has
predictive validity if it can be used to
predict success on tasks that are related to
the property measured.
EXAMPLE
• Do SAT scores help predict college grades?
Is it much clearer question than “ do IQ test
scores measure intelligence?
9
Examples
• Example. You might want to discuss the restricted range
problem in the context of, for example, predictive validity
of SAT scores. Almost all Princeton students have high
SAT scores. This restricted range reduces the correlation
• between SAT scores and grades for Princeton students. If
Princeton were to admit more students with SAT scores
\too low" by its usual standards, we would expect to see a
higher correlation. The restricted range problem means that
observed correlations between SAT scores and college
GPAs (like the r2's ) in the Statistical Controversies
feature) may be misleadingly low.
10
Error in measurement
• We can think about errors in measurements this way:
Measured value = true value + bias +
random error
A measurement process has bias if it systematically
overstates or understates the true value of the property it
measures.
A measurement process has random error if repeated
measurements on the same individual give different
results. If the random error is small, we say the
measurement is reliable.
11
Improving reliability and reducing
bias
• No measuring process is perfectly reliable.
The average of several repeated
measurements of the same individual is
more reliable (less variable) than an single
measurement.
12
bias and reliability
• The notions of bias and reliability in measurement are
easily absorbed from such examples as the bathroom scale
in the text. Be sure students do not confuse bias in
measurement with lack of validity. I don't put much
emphasis on the \true value + bias + random error“
equation| at this level, it is just a device to aid
understanding. I do try to get students to see that \true
value" is a tricky idea in behavioral measurements. In fact,
I would like them to be more suspicious of our ability to
measure constructs such as \authoritarian
• personality" (Example 11 in Chapter 8).
13
• Users of previous editions of SCC will notice that the
distinction among nominal, ordinal, and interval/ratio
scales of measurement has vanished. That distinction does
make the helpful point that not all numbers carry the same
information, but it doesn't correspond to standard statistical
practice. In practice, variables are usually treated as just
quantitative or categorical. \Ordinal" variables are divided
among these two classes. Some variables that may in
principle be only ordinal (e.g., IQ and SAT scores) are
universally treated as quantitative. Some clearly ordinal
variables (education level measured as \no high school
• diploma," \high school graduate but no college," and so on)
are treated as categorical. Advanced statistical methods do
take account of the ordering of categories in such cases, but
that's a specialist topic that still falls under the heading
\categorical data analysis
14