Get documents about "Home Based Business Surveys
Description
Home Based Business Surveys document sample
Document Sample
DigiPoll Ltd P.O.Box 4059. Hamilton East. Hamilton. 3247 New Zealand
Telephone: 0064 7 834 7655 Email: info@digipoll.com Web: www.digipoll.com
Photo by United Press International,Inc
Harry Truman displays a copy of the Chicago Daily Tribune
erroneously reporting the election of Thomas Dewey in 1948.
Truman’s narrow victory embarrassed pollsters, members of
his own party, and the press who had predicted a Dewey
landslide.
1
Table of Contents
1. Introduction 3
2. Scattered RDD vs Clustered RDD 4
3. DigiPoll’s RDD vs White Pages 14
4. DigiPoll’s Online-Based RDD Survey 21
5. DIGIPOLL’S RDD VS WHITE PAGES
2
Introduction
Since 1996, DigiPoll Ltd has been providing top quality survey services to clients
around the world.
DigiPoll’s surveys maintain a comparative advantage as a result of improvements in:
1. DIGIPOLL’S RDD VS WHITE PAGES
1. Sampling method - scattered Random Digit Dialing (RDD)
2. Statistical weights - second tier socio-economic parameters added to
conventional demographic weights.
This presentation illustrates the improvement through a three-way comparison:
1. DigiPoll’s RDD compared to conventional RDD surveys
2. DigiPoll’s RDD compared to White Pages surveys
3. Socio-economic weights compared to demographic weights in the context of
online surveys.
Overall, the presentation supports DigiPoll’s claim of a precision pollster which offers
clients high quality survey services at competitive charges.
3
RDD (Random Digit Dialing) is a telephone survey sampling method
that creates a telephone number by adding sets of random digits to
the prefix assigned to all telephone numbers in a common
Telephone Exchange Area (TEA).
RDD TEA is the area that a single telephone cable of a
telephone company serves. The same area may be
served by several telephone companies
The term TEA originates from the era
when telephone operators were
manually connecting the two ends of
telephone contacts.
4
Clustered RDD
Conventional RDD still follows the spirit
of old sampling methods used for face-to-
face surveys, where the sample was
restricted to the walking distance of the
individual canvasser.
This sample technique was used well
before the telephone had become a
common household commodity. It is
essentially a stratified or clustered RDD.
Clustered RDD is a time consuming selection of
socio-economic strata in the same ratios existing in
the general population.
5
Antiquated Sampling Still in Use
Clustered sampling continues to be a “necessary
evil” today, even after the telephone has become a
common household commodity.
This is because competing telephone companies
have been reluctant to expose the complete
lists of prefixes of the telephone exchange
areas (let alone the release of such prefixes
alongside maps).
6
The Deficiency of Clustered RDD
A fundamental deficiency of clustered RDD has been
the repeated calls to the same area, and neglecting
too many other telephone exchange areas.
For example - prior to the creation of its
own Australian RDD system, DigiPoll had
purchased an Australian RDD sample of
5000 telephone numbers from a UK
company which specializes in the provision
of world-wide national samples.
Only 33% of the telephone number prefixes in
that clustered RDD were unique. In addition,
the sample included only Telstra numbers.
DigiPoll Australia’s RDD today includes a
sample of 7017 unique prefixes for Australia
(4 times the number of the above clustered
RDD).
7
Garbage in
>>> Garbage out
As a result, a clustered RDD sample fails to represent
the complete socio-economic landscape.
8
DigiPoll’s Scattered RDD
Dr Gabriel Dekel, the founder of DigiPoll, has overcome
these serious sampling limitations by developing an
algorithm to identify the complete scope of the telephone
grid of any country.
Once all existing telephone exchange areas
have been detected, Dr Dekel has applied
Scattered RDD on the basis of calculating the
probability of active telephone usage within
the telephone exchange area, or the
existence of a sub-range (groups of
sequentially valid telephone numbers).
This method, then, allows any household to have an equal
chance of being included in a sample.
9
The Benefit of Scattered RDD
The end result of the Scattered RDD
sampling method is like the image
quality of a high resolution digital
camera: when repeated many times
over, the samples will reflect the
actual national population
distribution, illustrated here.
Clustered RDD will show only certain
localities which are hand-picked by
the pollster.
10
Statistical Weights
Even the best sampling method suffers
from a permanent response bias (e.g.
young males responses are always under-
represented).
In addition there are other random
effects which can make the response
pattern un-representative of the
population (e.g. time of call, the
weather, TV sports events).
All random samples therefore need
rigorous statistical weighting to adjust
the sample to reliably reflect the socio-
economic landscape of the general
population.
11
DigiPoll’s Unique Weights
Most researchers are using only “IF THE ELECTIONS WERE HELD
demographic variables for weights TODAY - WHERE WOULD YOU HIDE?”
(gender, age, regional location).
DigiPoll also employs a specific,
commercially-sensitive socio-
economic weight factor.
The graph illustrates the success of
predicting the 2008 general
election in New Zealand. In fact
DigiPoll has been successfully
predicting all election polls since
1996.
An analysis of DigiPoll accuracy is “THIS WOULD BE SO MUCH EASIER
WITH A ONE-PARTY SYSTEM!”
given in two web links: http://en.wikipedia.org/wiki/Talk:Opinion_polling_for_the_
New_Zealand_general_election,_2008
http://www.nzherald.co.nz/section/story.cfm?c_id=280
&objectid=10542145&ref=emailfriend
12
Maori Political Survey
It is known that the Maori vote is hard to poll. The voters can be highly mobile
geographically, lack landlines, and vote according to Iwi (tribe) affiliation. There
were no Maori telephone election polls until DigiPoll launched such polls for the
New Zealand national television TVNZ Marae programme in 1996.
In every election from that time on DigiPoll has shown highly accurate
predictions for the results in the electoral Maori Seats,. In 1996 DigiPoll
correctly predicted the loss of all seats from the Labour Party to the New
Zealand First Party, and showed Labour's recovery in 1999.
In Sept 2004, DigiPoll registered first among all pollsters the spontaneous
creation of a new Maori Party and its takeover of 4 Maori Seats in 2005.
In 2008, DigiPoll was the only pollster to show exactly which 5 seats would be
taken.
“As usual, thanks to Gabriel, Nandan and the rest of the
DigiPoll team for pulling out all the stops again for us. Everyone
out there uses the Marae DigiPoll as the yardstick for measuring
Maori political opinion. We’re now part of the country’s
political fabric!”.
Victor Alan, Manager, TVNZ Marae, 18 Nov 2008
13
DIGIPOLL’S RDD VS WHITE PAGES
DigiPoll has never used White Pages for surveys. In the case of business
surveys, DigiPoll adds random business numbers to Yellow Pages (many
home-based small business, or branches of larger corporations are not
listed in the Yellow Pages).
Some of the survey research companies who claim to provide random
samples do not specify their use of the White Pages phone book or
Electronic White Pages (EWP) as a source.
When this is the case, the survey research company
randomly selects telephone numbers from EWP or,
sometimes manually picks up numbers from printed
White Pages in a skipping order.
Today, the public release of EWP is illegal,
due to privacy laws. Therefore existing
EWPs, if legally purchased, are outdated.
14
1 in 5 Connections is Unlisted
When respondents to DigiPoll RDD surveys are asked if they
are listed in the current telephone book, at least 1 in 6
respondents reply negatively, either by choice or
circumstances.
In addition, these lists exclude unlisted phone
numbers and are biased against the frequently moving
or the recently moved-in. These population segments
all have less chance of getting their phone numbers
published on time.
The socio-economic profile of the unlisted respondent is
significantly different from the general population,
illustrated in the next slide.
15
Listed vs Unlisted in White Pages
1. Age Groups and annual Income
Age Group
30.00%
Annual Household Income
25.00%
25.00%
20.00%
20.00%
Listed
15.00%
Unlisted
15.00%
Listed
10.00% Unlisted
10.00%
5.00%
5.00%
0.00%
18-24 25-29 30-39 40-49 50-59 60-69 70+
0.00%
White Pages are biased against younger ages
0
0
0
0
0
0
0
0
00
0
00
00
00
00
00
00
00
00
00
00
,0
,0
0,
5,
0,
0,
0,
0,
0,
0,
5,
00
00
$2
$2
$3
$4
$5
$6
$7
$8
$1
$1
$1
(below 50).
-
-
-
-
-
-
-
-
an
r
-
0
1
1
1
1
1
1
1
ve
th
00
00
00
00
00
00
00
00
0
00
O
s
5,
0,
5,
0,
0,
0,
0,
0,
s
0,
Le
$1
$2
$2
$3
$4
$5
$6
$7
$8
White Pages are biased against households with annual
low income (below $20,000) and the $50-60,000 bracket.
White Pages over-represent the $80,000+ income earners.
16
Listed vs Unlisted in White Pages
2. Ethnicity and Residence
Ethnicity
90.00%
80.00%
70.00%
Urban/Rural Residence
60.00%
50.00%
Listed
Unlisted
40.00%
100.00%
30.00%
90.00%
20.00%
80.00%
10.00%
70.00%
0.00%
60.00%
NZ European Other
Listed
50.00%
Unlisted
White Pages are biased against all 40.00%
non-New Zealand-European ethnicities 30.00%
20.00%
10.00%
0.00%
Urban Rural
White Pages are biased against urban residents
17
Test:
The RDD vs White Pages
The table in the next slide illustrates the differences between White
Pages (WP) sample, RDD sample and the actual vote in the New
Zealand 2008 Elections. The analysis is based on the poll conducted
during the week prior to the 2008 Elections. The White Pages sample
has been extracted from DigiPoll’s RDD survey and includes only
respondents who indicated that they are listed in the White Pages.
The same statistical weights were applied to
each sample.
Table 1 – Final election poll (7th Nov 2008):
White Pages Versus DigiPoll RDD samples.
18
Table 1:
1 2 3
White Pages vs DigiPoll RDD
DigiPoll RDD Final White Pages Sample Actual Party vote
PARTY VOTE Poll 7th Nov 2008 Using Demographic 8th Nov 2008
Using Demographic weights
weights
National and Allies
(National, ACT, 50.4 54.5 49.4
United Future)
Labour and Allies
(Labour, Greens, NZ
First, Progressives 46.3 42.2 45.7
Others 3.2 3.3 4.9
Sample Size 982 721
Margin of Error 3.1% 3.6%
Actual Combined
1.6% (=0.8%) 8.4%(=4.2%)
Error (National &
Labour Blocks) (50.4-49.4)+(46.3-45.7)=1.6 (54.5-49.4)+(45.7-42.2)=8.4
19
Bias Follows Bias
The comparison shows a 3.4% bias against Labour & Allies voters, which
confirms the bias of the White Pages sample against the lower income
voters. The bias is wider than the statistical margin of error.
At the same time DigiPoll’s RDD showed a 0.6% bias in favour of the same
block of parties, well within the margin of error.
The combined error for the sample from
DigiPoll’s RDD was 1.6%, well within the
standard margin of error (±3.2% = combined
error of +6.4%), while the combined error for
the White Pages sample was 8.4%, well over
the margin of error.
20
Online surveys have been widely and rightfully
criticized for being non-representative of the
general population. However, if applied the right
way - online surveys can offer quality surveys at
low cost.
Laundry on line
Unlike many other online survey providers who use online panels recruited
from internet surfers, DigiPoll recruits its online samples through its
telephone scattered RDD surveys.
DigiPoll’s online samples are not from self-selected respondents, who use
this opportunity to partially derive their income from survey participation.
The big risk is that panels reflect narrow socioeconomic segments of the
population and therefore are unrepresentative.
21
Test:
Common Demographic Weights
vs Socio-economic Weights
The table in the next slide illustrates DigiPoll’s improved
socio-economic weights applied to an online-equivalent
survey. The on-line sample was extracted from
respondents to the 2008 pre-elections poll, who agreed to
participate in a future on-line panel for DigiPoll.
Column 1 shows a DigiPoll RDD of a sample of 982,
Column 2 shows only those in the sample who gave us an
email address (402), weighted by standard demographics
Column 3 shows the same sample weighted also by a
socio-economic factor
Column 4 is the actual vote.
22
Table 2: 1 2 3 4
Demographics vs DigiPoll RDD Online Sample Online Sample Actual Party
Socio-economic Final Poll 7th using adding socio- Vote
Weights
Nov 2008 using demographic economic 8th Nov 2008
demographic weights only weights
PARTY VOTE weights
National and
Allies
(National, ACT, 50.4 51.2 50.6 49.4
United)
Labour and
Allies
(Labour, 46.3 44.9 45.3 45.7
Greens, NZ1
Progressives)
Others 3.2 4 4.2 4.9
Sample Size 982 402 402
Margin of Error 3.1% 4.9% 4.9%
Combined
Error 1.6%(=0.8%) 2.6%(=1.8%) 1.6%(=0.8%)
(National and (50.4-49.4)+(46.3- (51.2-49.4)+(45.7- (50.6-49.4)+(45.7-
Labour Blocks) 45.7)=1.6 44.9)=2.6 45.3)=1.6
23
Good, and Even Better
Using again the combined error as a benchmark of quality, the results show
that the application of the common demographic weights on their own has
been a remarkable success of an online equivalent sub-sample of just 402 -
a combined error of 2.6%. The sample of 402 (with margin error of ±4.9%)
allows a combined error of 9.8%!
Note that the mode of selection of DigiPoll’s online sample is still
DigiPoll’s scattered RDD, whereas the common mode of recruitment
of online panels is through the solicitation of internet surfers,
resulting in a self-select bias by career panelists. DigiPoll therefore,
does not endorse other online surveys as reliable tools.
However, after applying socio-economic weights on DigiPoll’s
online sample, the results are even more outstanding with only 1.6%
combined error, similar to DigiPoll’s larger telephone RDD sample of
982!
24
DigiPoll Saves You Money
In times of economic austerity we are cost conscience. The precision
of scattered RDD means that you may be able to compromise on the
sample size.
Sampling error is a function of both sample size (error type A,
measured as a margin of error), and sample design (un-measurable
error type B). Poorly designed samples will deliver a higher error
than the indicated statistical margin of error for a given sample size.
Since DigiPoll’s sample design is superior, you can have a trade-off
where smaller sample size will satisfy your quality requirement.
We expect that a 15 minutes sample of 600 with conventional RDD,
calling from Australia, will cost you AUS$24,000, at minimum.
DigiPoll can provide you with the same quality calling
from New Zealand a sample of 400 for about
AUS$15,000 (inclusive of statistical weights).
25
Thank you
If you need more information please email info@digipoll.com
26
Related docs
