Customer satisfaction made easy

Document Sample
Customer satisfaction made easy Powered By Docstoc
Made Easy
A plan for actively listening
      to customers of
Labor Market Information
  Products and Services

     A resource handbook from the
Customer Satisfaction Work Group of the
    Workforce Information Council

           Copyright 2003
                  Table of Contents

Topic                                                   Page

Introduction – Why Measure Customer Satisfaction?        1
How to Measure Customer Satisfaction                     2-3
Step 1 – Pick a Product or Customer Group                4-6
Step 2 – Set the Stage for Assessment                    7-31
         What do you want to find out?          7-10
          Sample Problem Definition Statement   8-10
         How you can best find the answers?     10-31
          Qualitative Approach                  11-12
          Quantitative Approach                 13-14
          Guide for Using Focus Groups          16-18
          Guide for Using A Mail Survey         19-20
          Guide for Using A Telephone Survey    21-23
          Guide for Using a Personal Interview  24-26
          Guide for Using an Internet Survey    27-29
          What to Ask                           30
          Sample Survey                         31

Step 3 – Conduct Research and Take Action!              32
             Documenting Demand
Conclusion                                              33
            I n t r o d u c t i o n                         –
 Why measure customer satisfaction?

        Because it’s required! The Workforce Investment Act of 1998
says states need to “consult with customers about the relevance of the
information disseminated through the statewide employment statistics
system, in order to continuously improve the system.” The two key
parts of this statement are:
      • Consult with customers
      • Continuously improve the system
It’s also a requirement of the One Stop LMI grant from the Employment
and Training Administration!

      Because we have to prove “return on investment!” In times
of increased accountability in government, every activity needs to prove
that taxpayers’ money is used efficiently and effectively. To ensure that
LMI products retain their value to customers, we need to continuously
ask customers if our products are meeting their
needs. We also need to create new products that are
responsive to customer demands. Asking our cus-
tomers about our products provides proof that LMI
products are a good investment.

     Because we can’t afford not to! With
limited funding, we have to make sure that every dol-
lar we spend on developing and disseminating infor-
mation products (whether its for printing or updating
a website) is being spent on a product people will
use. The BEST way to make sure our products are
being used is to ask the people who use them.

    Customer satisfaction assessment is a great opportunity
       for states to make positive changes in the future
            of the labor market information system!
         How to measure
      customer satisfaction?
         This handbook will help states make customer
satisfaction assessment as easy as 1, 2, 3! In order to
make it this easy, there are some things you should do
and should know.

Form an evaluation team. The team will oversee the collection of
customer satisfaction feedback and then perform three responsibilities.

           1. Make changes to products and services based on that feedback;
           2. Share satisfaction processes and results with other states;
           3. Share satisfaction processes and results with federal
              representatives to make systemic changes.

       Consider having a publications specialist, an analyst and a projections
professional on your team. This team may already exist as the group who deter-
mines your LMI product content.

Customer satisfaction assessment does not
necessarily have to be a statistically rigorous exercise.
       The process outlined in this handbook is based on solid descriptive
research methodology, but it is different from the types of research normally
conducted in “LMI shops.” While you may have to meet strict standards for
response rate based on a stratified random sample to meet the requirements of
the Bureau of Labor Statistics, customer satisfaction assessment is based on a
“dialogue” with your customers. This dialogue may not be conducted the way
you would conduct a BLS survey.

You should document the demand for your
products and services.
        In addition to finding out if customers are satisfied with your products,
find out how many are actually using them, how they are using them and where
those users are located. We have a special section at the end which addresses
this issue.
Create Customer Lists.
        Many of the processes outlined in this handbook will require you to have
a list of your customers. You may want to consider creating these lists if you do
not have them already. These are lists of customers who receive or use a partic-
ular document or LMI product you produce.

You have many tools in the toolbox of customer
satisfaction research.
      You may want to conduct a survey, you may want to have a focus group
or conduct interviews with your main customers of a particular product.
Remember, when you ask just one customer their opinion of a product, you
have started to listen to the customer. When you change a document based on
what a customer said, you are engaged in the process of continuous improve-
ment. This leads to the final point:

You can’t do it all at once!
       This handbook will walk you
through conducting customer satisfac-
tion research. At first, it may seem a
daunting task to find out what all your
customers think about all of your prod-
ucts. But, you have to start somewhere
and you can’t do it all at once. Remain
calm and dig in!

Step           Pick a Product or Customer Group
                      State LMI providers should be producing docu-

   1           ments for particular customer groups. According to the
               Workforce Investment Act, states should be serving the
               following customer groups:

• Businesses make decisions about product and financial markets, busi-
  ness location, and employee recruitment, compensation, and training.
• Individuals, including young people and adults, make choices about
  careers, education, training, and job search.
• Elected Officials and Policy Makers, including Workforce Investment
  Boards, make decisions about law, policy, budgets, and regulations.
• Program Planners determine what workforce and economic develop-
  ment services to provide, and evaluate program performance.
• Education and Training Providers, including teachers and curriculum spe-
  cialists, design, deliver, and evaluate programs that develop students’
  knowledge and skills.
• Intermediaries, such as parents, counselors, teachers, mentors,
  placement workers, and case workers, assist others in choosing edu-
  cation and training opportunities, and in finding employment, and
• Researchers study how the labor market works, and conduct policy
  research. The mass media may be included in this category.

        Take a look at the LMI products
and services your state produces.
Which customer groups use each of the
products? Create a grid, like the sam-
ple that follows, which shows the cus-
tomer groups and the products you pro-
vide for them.

                            CUSTOMER                                            GROUPS

                                           Elected officials and

                                                                                       Training Providers
                                                                   Program planners/
                                                                   Workforce Boards
                                           Policy Makers

                                                                                       Education and
                            Job Seekers




Handbook                         X                                                         X                X
Hot Job Sheet                    X                                                                          X                               X
Labor Market
Trends           X                              X                        X                                                   X              X
Wages 2002       X
Patterns                                        X                                                                            X
Rate                                            X                                                                            X
Conference                                                                X
Website 1       X                                                                                                            X              X
Website 2                        X
Website 3                                                                              X

      At each intersection on the grid,
    you should be assessing satisfaction.

Once you identify which customers are using what products and services,
you can measure customer satisfaction by starting with either of two paths:

                • start with a product, or
                • start with a customer.

         As noted on the previous grid, if your state is about to re-publish a
Career Handbook, you may want to ask people in the three customer groups
that use the document if they are satisfied with it. In that case, you would be
starting with a product.

        You can be in tune with the changing needs of customers by conduct-
ing needs assessment without discussing an existing product. For example,
each year employment statistics agencies must confer with local workforce
investment boards to discuss information needs. This is an opportunity to
conduct a needs assessment. Ask them about how they use data in their core
business functions and if workforce information could help them do their
work. This may help you prioritize which information products and services
you provide and customize those products for their specific needs.

Step               Set the Stage for Assessment

    2                There are two things
                  you need to do in step two:

1. determine what you want to find
   out, and
2. determine how best to find the

     The rest of this step walks you through
this process and gives you examples.

What Do You Want to Find Out
You should develop an Assessment Statement. This merely helps you “think
through” your descriptive research process. It may seem obvious when you
complete it, but it will help guide you. This is what should be in and should
be answered in your statement:

• A general description what you want to find out and from whom.
• A basic question the research needs to answer
• What you want to determine in your research?
• Management objectives, such as:
    ✓ Why you are doing the research?
    ✓ What will be the uses of the information you learn?
    ✓ Why is it important that you do the research?
    ✓ What is the general setting for the research?
    ✓ Who are you going to assess?
• What data collection technique will you use and why?
• When will you do it?
• How much time/money can you spend doing it?
• Who will do it?

                 The following pages provide an example of a
                      completed Assessment Statement.


Assessment Statement

General Description:
                                  To assess satisfaction with career guidance mate-
                                  rials delivered by the State Department of Labor
                                  (DOL) and determine methods for improving
                                  those products for customers.

Basic Research Question:          In what ways can career guidance materials be
                                  improved to assist those who advise students on
                                  career goals?

Research Information Questions:
                                  - To determine the level of satisfaction with
                                  existing career materials, specifically Guides to
                                  Career Choices and accompanying posters;

                                  - To determine the most effective attributes of
                                  career materials which should be retained in
                                  future publications;

                                  - To determine the least effective attributes of
                                  career materials which should be omitted in
                                  future publications;

                                  - To determine attributes of career materials from
                                  other states which may provide better
                                  alternatives for presenting information to career
                                  guidance personnel.

Management’s Objectives:
                                  Why is this research being done?
                                  We are required to assess customer satisfaction
                                  with products as part of its funding grant from
                                  the U.S. Dept. of Labor, Employment and
                                  Training Administration (ETA), and needs cus-
                                  tomer satisfaction input to guide its continuous
                                  improvement efforts. This assessment is in par-
                                  tial fulfillment of this obligation. We also want
                                  to create a more useful product.

                             What are management’s intended uses of the

                             DOL management will document comments from

                             customers and, as necessary and fiscally possible,

                             make changes to career information materials.

                             How important is the research?
                             Limited resources to fund printing require DOL
                             management to ensure that only the most useful
                             information is published.

                             What is the general setting for the research?
                             Labor Market Information has been published by
                             DOL for many years under cooperative agreements
                             with the U.S. Dept. of Labor, Bureau of Labor
                             Statistics and Employment and Training
                             Administration. Career guidance materials are part
                             of meeting these and other mandates (e.g., the
                             Workforce Investment Act of 1998) to serve edu-
                             cators. DOL wants to ensure that this key target
                             group is being effectively served.

                             Who is to be assessed?
                             The career guidance personnel in two vocational
                             districts in the state will be invited to participate in
                             focus groups.
Data Collection Technique:   Because the group members can be identified by
                             name, a focus group will be held in each of two
                             districts with approximately 7 to 12 persons in
                             each group. Most of the questions will be open-
                             ended to allow for maximum response by the per-
                             sonnel. One group is from a suburb. The other
                             group is from the more rural area. The two career
                             guidance centers in the above-mentioned commu-
                             nities will be contacted for a list of names of per-
                             sonnel who use the materials. These individuals
                             will be invited to attend the focus group until a
                             maximum of 12 people are confirmed in each loca-
                             tion. The focus groups will be held at the voca-
                             tional offices in the communities noted.

Timing:                      The research will be conducted
                             May 19 - 30, 2003.                                   9
Researchers:                   The focus groups will be conducted by
                               an independent research company.

           How You Can Best Find the Answers
Notice that in the above example a focus group was selected. There are many
options in addition to focus groups. The Customer Satisfaction Work Group
recommends one or a combination of these five:

        • Focus Group
        • Focused Personal Interview
        • Mail Survey
            (can be handed out at
             a conference, too)
        • Internet Survey
        • Telephone Survey

Notice that the first two data collection techniques mentioned above involve
one-to-one personal contact with a group of customers. The last three involve
sending or handing out a survey. You can get both qualitative (comments) or
quantitative (numbers) information from either source. So, which should you
use? There are some ideas on the following pages.
                A GUIDE FOR USING

General Description and Purpose
This is an approach that is primarily designed to gather information
using open-ended questions to allow for more in depth interviewing.
When you want more insights into a question or general opinions,
this is a good method to use.
Recommended LMI Products or Services
This can be for all products and services that you provide.
Types of Questions
Normally open-ended questions are used. (Such as “how do you use
these” or “how do you feel about this.”) However, do not hesitate to
ask closed-ended questions but not too many. (Such as “how many
students do you serve.”)
Customer Groups
All customer groups are appropriate.
Selection of Customers
A nonrandom, subjective method may be better than a random,
objective method. Select people who you feel will share feelings
and ideas.
Number involved
A relatively small number, say fewer than 20, can be very effective.
But, you may want to ask open-ended questions on a formal,
quantitative survey of a particular customer group. This is appropriate
but remember that the answers to open-ended questions are the most
difficult to analyze. If you ask 300 people a “why” question, you
will spend many hours just analyzing that one question!
Typical Response rate
Unless you are asking the open-ended question of a person, you may
find that many people do not answer open-ended question on a
written survey.                                                   11
Method of Conducting the Exploratory Study
There are many methods available. Focus groups are commonly
used. However, you can do individual interviews. This differs from
a focus group since there is not the dynamics of reaching group con-
census. Another technique is the “think aloud” method. Comment
cards can also provide insights.

Buying a program to document and tabulate the open-ended
responses can be beneficial.

Typical Costs
Will normally be cheaper than quantitative methods due to a smaller
sample size.

Time Required
Can be shorter than doing quantitative research.

Staffing Needs
You will need persons skilled in leading a group or interviewing
persons. Having a staff member trained in analyzing open-ended
responses would be very helpful.

Benefits of Using this Approach
You can obtain in-depth customer feedback and gain an understand-
ing of how information is actually used by the customer. You have
much flexibility in your approach. The approach can be very infor-
mal. Using a complex sampling method is not needed. You also
will not normally need, nor want, a large sample size.

Cautions in Using this Approach
If you are doing a focus group or interviews, you cannot project the
results to a larger group of people. If you are analyzing the results
of open-ended questions asked on a quantitative survey, you can
project the results to the larger group. The problem becomes
analyzing the answers in some meaningful manner.
                 A GUIDE FOR USING

General Description and Purpose
The quantitative approach is used when you want to “count” a certain
characteristic and project the results to a defined population or cus-
tomer group.

This method is most appropriate when you have a randomly selected
group of persons. In addition, having a fairly large sample is very helpful.

Recommended LMI Products or Services
All LMI products and services can be evaluated using a quantitative

Types of Questions
Normally, closed-ended questions are used in a quantitative approach.
But one can count the number of ideas generated from open-ended
questions. Behavior, attitude or demographic questions can all be
asked using this approach.

Customer Groups
All customer groups can provide information using this approach.

Selection of Customers
Normally, you would want to select the respondents using a random,
objective method such as systematic sampling.

Number involved
If you have a fairly large number in your population, for example,
employers or job seekers, then a random sample that will result in 300
responses is very appropriate. This will yield a margin of error of no
more than 5.6%. If you have a small number in a customer group such
as 200 on a mailing list, survey all persons.

Typical Response rate
By surveying a large number of persons, we expect that not everyone
will respond. Response rates of 10% to 50% can be expected depending
on the customer group surveyed.

Method of Conducting the Quantitative Study
You can use any method: mail, telephone, Internet, self administered
(hand out), computer-assisted, Web page analysis, or comment cards.

Typical Costs
Quantitative methods normally involve more persons than a qualitative
and therefore cost more. You may need to purchase special computer
packages for tabulating and analysis.

Time Required
Normally, the approach will take longer
than a qualitative approach.

Staffing Needs
People with different skills in developing
questions, collecting data, entering data,
analyzing data and writing reports will
be needed.

Benefits of Using this Approach
You will be able to provide consistent
information from one time period to
another. By taking a sample, you can project results to the entire cus-
tomer group.

Cautions in Using this Approach
This approach will normally take more time and cost more than a quali-
tative approach.                                                 14
      How likely are people to respond to different techniques?
             (1 is the best and 5 the worst)

                                                                 Personal Interview
                            Focus Group




Speed of Collection         1   3   1   2   1
Geographic Flexibility      5   2   1   5   1
Respondent Contact          90% 95% 30% 30% 95%
Response rates             50%             30% 70% 80% 30%
Structure: Flexibility     3               2   2   1   2
Questionnaire length:
Flexibility                 5              3     2               1                    3
Least Respondent
misunderstanding            2              3     1               1                    3
Easiest to follow up
on nonrespondents           1              1     3               5                    1
Most confidentiality        5              2     4               5                    1-5
Most anonymity
of respondents              5              1     4               5                    1
Least cost                  2              2     1               4                    1-3
List of names needed        1              1     1               1                    1
Easiest to enter data       3              3     1               3                    2
Highest quality
of information              1-5            1-5   1-5             1-5                  1-5


General Description and Purpose:
A qualitative method of obtaining in depth answers to open-ended
questions from a small group of knowledgeable persons who share a
common interest.

Many times this is an exploratory method that obtains ideas and
insights about a particular question such as “What do career counselors
think about our new website?”

Recommended LMI Products or Services Any of your products or
services can be studied using this method.

Types of Questions:
You will be using primarily 10 to 15 open-ended questions but do have
a list of them before you begin the session. Any behavior or attitudinal
question can be asked. But be certain to ask the all important “Why do
you use this product/service” and “why is it important to you?”

Customer Groups:
Any group of employers, job seekers, elected officials and policy makers,
program planners, education and training providers, researchers, and
others can be used in a focus group. It is best to find people with
similar LMI product experiences. Some will be easier to contact
than others. The people should be located close to each other
Selection of Customers:
You want people who are familiar with the product or service being
discussed. If at all possible, invite persons who will share their ideas.
A subjective, non random method is normally better than a random
Number involved:
Having a group of 8 to 12 persons is typical. You want to be certain
that you have persons who will share their opinions. You should plan
on having 15 to 18 persons commit to coming; some will not show.
Remember, even 5 important users could make a good group.
Typical Response rate:
About 80% of those invited will attend. You will want to get everyone
involved in the discussion.
Method of Conducting the Focus
Have a table for the people to sit
around. The leader can use an easel to
write down ALL comments. Encourage
all comments. Use a cassette recorder
and a camcorder to tape the comments.
Provide refreshments. Get permission to
record people.

Typical Costs:
Many focus groups will provide an honorarium for each participant of
$25-$50. Many do not. Also, budget for refreshments. A consulting
firm will typically charge $2500 to $5000 for one session.

Time Required:
Depending on the group, the focus group may be organized within a
week. It may take much longer if the group is not accessible. A
focus group will last about 90 to 120 minutes but can be accomplished
in 60 minutes if needed.                                           17
Staffing Needs:
You will need a trained facilitator and possibly another person to
record the session. If the facilitator does not use an easel, then another
person must be present to take notes. Be sure not to use the focus group
to correct or educate the participants. You may want to consider a facili-
tator who is not from the LMI unit.

Benefits of Using this Approach
Many times this can be a very efficient method of obtaining many ideas.
You can also obtain very in depth answers and antidotes not apparent in a
quantitative survey. The sampling method does not need to be complex.

Cautions in Using this Approach
Focus groups provide qualitative and not quantitative information.
Do not group people of different levels of the chain of command
(employees and supervisors). Be careful about grouping people who
know each other. You will need an effective leader of the session. Do
not let one or two persons dominate the discussion. Analyzing the results
of the focus group is an art within itself. Drawing key concepts out of
the information is important. Be careful of getting defensive or lecturing
if an LMI professional moderates.

General Description and Purpose:
You want to survey persons using the
US mail and have several questions to
ask. Normally, there is a large number
of persons to be surveyed and you want
to project the results to a particular
The mail survey is very appropriate when the respondents are easily
identified but may be located fairly far from each other. It is useful
when the questionnaire is two to four pages in length. You can also
enclose a brochure for the recipient to evaluate.
Recommended LMI Product s or Services:
All of your products or services can be evaluated through the mail.

Types of Questions:
The mail survey normally uses closed-ended questions that provide
answers. You should be able to have about 25 questions per page. Too
many open-ended questions will be left unanswered.
Customer Groups:
Any group that can be identified such as recipients of a LMI brochure
or job seekers, can be surveyed using the mail.
Selection of Customers:
You will want to use an objective, random method of selecting your
respondents. A frequently used method is systematic sampling.
Randomly select every “n”th person from the list.
Number involved:
If you have fewer than 1000 persons in a customer group, select every-
one. If you have more, then select 1000 to be surveyed. You will reach
about 95% of your selected group.
Typical Response rate:
Depending on the group, the response rate may range from 10% to 50%.
However, 30% is more typical. Having responses from of 300 persons
will result in a margin of error of 5.6% when projecting the results to
the appropriate customer group.
Method of Conducting the Mail Survey:
You will need a list of names and letterhead stationary. Outgoing #10
envelopes and postage-free #9 return envelopes are necessary. A follow-
up post card is needed as well. Copy the questionnaire onto both sides of
the paper. Make it look professional. Including an incentive such as a
bookmark can be effective.
Typical Costs:
Costs must be allowed for copying, envelopes, outgoing postage for
both the surveys and a postcard and return postage for the surveys.
Time Required:
Allow at least 2 weeks to obtain the names, design the questionnaire,
and obtain the envelopes. About 4 days after mailing the questionnaires,
send out a follow-up postcard. Allow at least 2.5 weeks for responses.
A second questionnaire mailing might increase response rate, but would
take longer.
Staffing Needs:
Different skills are needed. Good word processing skills for designing
the questionnaire are desirable. You will need persons to stuff the ques-
tionnaires, enter the data correctly, tabulate and analyze the results.
Benefits of Using this Approach
You have a high certainty of contacting the people you want. The
design of the questionnaire can be very flexible. You can do a survey
from a centralized location and have control over the process.
Cautions in Using this Approach
The response rate is somewhat low. The survey must look professional.
Include a postage free return envelope. If the group being surveyed is
not interested in the product or service, the response rate may be
extremely low. Do not ask too many open-ended questions.
General Description and Purpose:
Calling a group of customers can be a very
efficient and effective method of collect-
ing information from them. The tele-
phone survey allows for both quantita-
tive and qualitative data collection.

Use this method when you have
a specific list of persons to be
called who will be receptive to
your call. Many people are too busy
to answer, or may not be available to take your call.

Recommended LMI Products or Services:
All products and services can be surveyed using the telephone

Types of Questions:
Most telephone surveys will ask fewer than 40 questions. Yes, you can
ask more but the respondent may become irritated. These questions
will tend to be closed-ended with answers. Asking too many open-
ended questions will also irritate the respondent. For example, asking
“Why” to each question will soon yield very little response.

Customer Groups:
All groups can be difficult to contact. Having the specific telephone
number and name of a person who is a user of information will greatly
assist in the use of the telephone technique.

Selection of Customers:
You will want to use an objective, random method of selecting your
respondents. A frequently used method is systematic sampling.
Randomly select every “n”th person from the list.
Number involved:
If you want to have 300 persons included in your data base, then you
will need to select about 900 persons. This can be less if you have a
longer time period to collect the data. But if you are not going to use
call backs, then selecting 3 times the number desired is a good rule of

Typical Response rate:
Of those persons actually contacted, you should expect about an 80%
response rate.

Method of Conducting the Telephone Survey:
A centralized method of calling persons is best. But you may not have
a center for telephoning. Therefore, employees will need to call from
their desks. You will need to develop a computer program that will
allow the interviewers to ask the questions that are on the computer
screen so the information can be directly entered into a data base. If
persons who are called are not available, then a callback should be
made the following day. Do not leave a message on a recorder.

Typical Costs:
This could be the least expensive method since persons are already
hired and telephones are available. Even long distance costs are very
reasonable today. Interviewers would already have computers. If the
office does not have a computer program to facilitate the interviewing
and data collection, then it would need to be purchased.

Time Required:
This can be a relatively quick method of collecting information. Allow
about one week to establish the survey, a few days for training and per-
haps another week to collect the information. If the data are entered
directly into the computer, a preliminary report can be generated within
a day of the final collection of data.

Staffing Needs:
In addition to people constructing the
questionnaire, you will need experienced
interviewers. The more you have, the less
time it takes to collect the data. You may
need several people to do the calling. This
will take them away from their other

Benefits of Using this Approach
You can contact many different people
very quickly. You have control over the
survey. You can call other people to
replace those who do not answer or refuse
to answer. Of those you contact, most will
participate. This can be the quickest and least costly method.

Cautions in Using this Approach
Be careful about asking too many open ended questions. Do not have
the survey last more than about 7 minutes. Be careful of the ques-
tions you ask. Do not ask questions that rely upon the customer’s
memory too much. You will have fewer options in forming your
questions and answers. It is very helpful to have the questionnaire
entered into a computer so the responses can be easily tabulated. You
will need pleasant people with a strong self confidence to do the

                A GUIDE FOR USING

General Description and Purpose:
A personal interview survey involves directly asking people on a one-to-
one basis. It allows direct interaction with the respondent.

If you can easily contact people, you might want to use this technique.
But unless the respondents really want to be surveyed, this can be a
time consuming, frustrating technique. This would be most appropriate
for persons coming into One-Stop Center or those attending a confer-
ence, workshop or training session.
Recommended LMI Products or Services:
Since this technique is the most flexible, one can ask about any LMI
product or service using a variety of techniques.

Types of Questions:
Any type of question is appropriate for the personal interview tech-
nique. Even though many of the questions will be closed-ended, you
can use several open-ended ones. People are more likely to respond to
open-ended question when personally asked. You are not as limited to
the number of questions as in the other techniques. More than 50 ques-
tions would be possible.

Customer Groups:
Since many of the customer groups would be difficult to contact in an
efficient manner, this technique would be best limited to persons com-
ing to One-Stop Center (if you are asking job seekers, for example) or
those attending a conference, a workshop or a training session. This
can also work for employers who may not have time or want to attend
a focus group.

Selection of Customers:
If you limit your choice of respondents to one of the suggested groups,
you will be more limited in who you will talk to. Therefore, interview
everyone who comes into the office during a specified week. Also
interview everyone who attends a conference, workshop or training
session, or every willing employer on a list.

Number involved:
The number involved in this study will be limited to the number who
will come to your office, welcome you to their office or attend a confer-
ence, workshop or training session.

Typical Response rate:
About 80% of those coming to your office will most likely have time
to answer your questions. Over 90% of those attending a conference,
workshop or training session will respond. Some will have left. Some
are in a hurry.

Method of Conducting the Personal Interview Survey:
Have a person with a pleasant personality conduct the interviews.
Having a clip board facilitates the interview process and projects a
“research atmosphere.” Questions are directly asked and the answers
recorded. Some questions may require the respondent to look at them
in order to answer them. Be certain that all open-ended questions are
directly asked and that the respondent is encouraged to respond.

Typical Costs:
This can be relatively inexpensive since only one or two persons will
be needed to do the interviewing (such as an economic planner.)

Time Required:
This can be relatively quick if the persons are at a session. It can be rel-
atively slow if you need to wait for a particular type of person such as
an economic planner to call you back for an appointment.

Staffing Needs:
In addition to people constructing the questionnaire, you will need an
experienced interviewer. Someone to enter the information will be
needed. Someone will be needed to analyze the open-ended questions.

Benefits of Using this Approach
You have control over the survey and the asking of the questions. Of
those you contact, most will participate. Can be a quick and a low cost

                                      Cautions in Using this Approach
                                      You may need extra time to collect
                                      information using this approach.
                                      Some people will not want to be
                                      interviewed. You are trying to
                                      determine customer satisfaction
                                      not to create customer dissatisfac-
                                      tion. If you ask open ended ques-
                                      tions, you will need to have some-
                                      one analyze them. You may not
                                      have a very large sample size on
                                      which to make decisions. You will
                                      be limited to those persons who are
                                      available to you. Trying to person-
                                      ally interview employers at their
                                      place of business can be a major
                                      problem.                       26

General Description and Purpose:
Providing an opportunity for a person to respond to a survey using the
Internet. This can be accomplished using an e-mail message only or
linking with a Web based survey. Using the Internet to collect
information is very similar to a mail survey.

The Internet survey is most appropriate when you have the e-mail
addresses of all persons to be surveyed. Also, you must be confident
that the respondent would not treat a message as “spam”.

Recommended LMI Products or Services:
All products and services, including a Website for your LMI materials,
could be surveyed in this manner.

Types of Questions:
Any type of question is
appropriate for the Internet
based survey. Even though
many of the questions will be
closed-ended, you can use a
few open-ended ones. You
are more limited to the num-
ber of questions using this
technique since people are
accustomed to spending a
short time on Websites. More
than 50 questions would be
somewhat impractical.

Customer Groups:
Any customer group with a computer is appropriate for this technique.
But you need the e-mail addresses.
Selection of Customers:
If you do sampling, be certain to take a random sample of the people to
be contacted. Systematic sampling works very well.

                              Number involved:
                              A target number of responses varies
                              depending upon the customer group.
                              However, 300 to 400 responses is an
                              adequate number normally. But if the
                              total number available to survey is only
                              100, survey everyone.

                              Typical Response rate:
                              The response rate will vary greatly
                              depending upon the knowledge of the
                              LMI products or services and the overall
                              interest of the customer group in the
                              information. You can expect 60% from
                              LMI-savvy individuals but only 10%
                              from job seekers.

Method of Conducting the Internet Survey:
The methods used can vary widely. The simplest way is to include
questions in an e-mail message and have the person hit REPLY and
click on the appropriate answer. You cannot have very many questions
though. This is cumbersome.
If you use an e-mail message with a link to a Web page, you will
probably need a computer package to build the survey on a Web page.
The questionnaire is automatically sent to a collection point ready for
A major consideration of both methods is how to load the email address-
es for an easy dissemination of the survey.

Typical Costs:
The e-mail only method is very cost efficient. It requires only employ-
ee time to enter the survey form and then to enter the responses when
they return.

The e-mail with Web page version is more costly. A software package
may cost $6000 with a $2000 annual service fee. However, the data
are already entered.

Time Required:
This can be the most efficient method of collecting information. It will
take a week to build the customer base and develop the survey and
enter it. Within an hour, you can send out the survey. You can then
send out a reminder within a day or two.

Staffing Needs:
In addition to people constructing the questionnaire, you will need an
experienced person to work with the Internet If you have open-ended
questions, someone will need to analyze them.

Benefits of Using this Approach
You can send reminders very easily to the respondents. Do not send
too many though. Data can be collected very quickly. This can be a
very low cost method.

Cautions in Using this Approach
People may view your survey as another spam related message. The
response rate will vary greatly. Not all customers you want to survey
will have email addresses. Some of your customers will not be very
computer literate and will refuse to do the survey. If you do the e-
mail/Web page version, you may need to attach a Word or Word
Perfect file containing the survey.

                          What to Ask
No matter what kind of research method you choose, there are cer-
tain questions you should ask.

The Customer Satisfaction Work Group analyzed many surveys from
state and federal LMI agencies and developed nine (9) attributes which
should be assessed about LMI products. They are:

Accuracy: The information is accurate enough for the customer’s use.
Relevancy: The information is relevant to solving the customer’s
Accessibility: The customer could easily access the information.
Understandability: The customer could understand what the content
Comparability: The customer is able to compare the information with
other information they use.
Geographic Detail: The information is in sufficient geographic detail.
Timely: The information is timely enough for the customer’s needs.
Completeness: The customer could solve the problem with this infor-
mation only.
Importance: The information is important in the customer’s overall
problem solving work.

You should also ask a general satisfaction question (“Overall, how
would you rate your safisfaction with this product?”), some demo-
graphic questions (“Are you in business
or an educator?”), some questions about
customer service (“Was our staff help-
ful?”), and, most important, there should
be room for open ended comments. The
following page is an example of a written
survey with all of those characteristics.
This survey can also be interpolated into
questions to ask during a focus group or
  This is a sample survey which includes basic important questions for all sur-
  vey methods on LMI products. Feel free to modify it as needed. Notice at the bot-
  tom of the form you can indicate from which data source(s) the information origi-
  nates from. You also may want to enter in the top line for the customer.

                              Customer Satisfaction Survey
What is the name of the information product you are using.
Describe the product

Please indicate all the purposes for which you use this product (Check all that apply) :
❑ Product markets                                   ❑ Compensation                                        ❑ State/local budgeting
❑ Financial market                                  ❑ Training                                            ❑ State/local policies
❑ Business location                                 ❑ Career planning                                     ❑ Develop curriculum
❑ Employee recruitment                              ❑ State/local policy making                           ❑ Assist others finding employment
❑ Business planning                                 ❑ Economic development                                ❑ Inform the pubic
❑ Business budgeting                                ❑ Find a job

What is the one main purpose for which you use this information:
Please indicate the broad category which applies to the person or organization using this information:
❑ Business        ❑ Researcher                  ❑ Job Seeker              ❑ Researcher ❑ One-Stop Employee                 ❑ Media
❑ Student         ❑ Elected Officials           ❑ State or Federal Policy Maker               ❑ Workforce Program Planners
❑ Education & Training Provider                 ❑ Intermediary helping people find employment (such as a career counselor)

On a scale of 1 to 5, please circle the appropriate rating of the following attributes of the information product you are using:
 The information is accurate enough for my use                            The Information is timely enough for my needs
 Strongly Disagree Disagree   Neutral   Agree    Strongly Agree                                          enough Agree Strongly
                                                                           The information is timely Neutral for my needs Agree
                                                                           Strongly Disagree  Disagree
        1             2         3         4           5           N/A      Strongly1             2
                                                                                    Disagree Disagree       3
                                                                                                       Neutral   Agree 4Strongly Agree
                                                                                                                                     5       N/A
                                                                                   1            2        3        4           5        N/A
 The information is relevant to solving my problem                        I am able to solve my information needs with this
 Strongly Disagree Disagree   Neutral   Agree    Strongly Agree            I am able only
                                                                          information to solve my information needs with this
        1             2         3         4           5           N/A      information
                                                                          Strongly Disagreeonly
                                                                                              Disagree   Neutral     Agree    Strongly Agree
                                                                           Strongly1             2
                                                                                    Disagree Disagree       3
                                                                                                       Neutral                       5
                                                                                                                 Agree 4Strongly Agree       N/A
 I could easily access the information                                             1            2                 4           5
                                                                          The information is important3 my decision making process
                                                                                                          to                           N/A
 Strongly Disagree Disagree   Neutral   Agree    Strongly Agree           Strongly Disagree    Disagree      Neutral      Agree    Strongly Agree
        1             2         3         4           5           N/A     The information is important3to my decision
                                                                                   1            2                      4            5     N/A
                                                                          making process
 I understand what the content of the information means                   Strongly I am satisfied with this informationStrongly Agree
                                                                          Overall,Disagree Disagree Neutral      Agree   product
 Strongly Disagree Disagree   Neutral   Agree    Strongly Agree           Strongly 1
                                                                                   Disagree    2
                                                                                             Disagree   3
                                                                                                        Neutral   4 Agree Strongly Agree
                                                                                                                             5        N/A
        1             2         3         4           5           N/A             1               2             3           4             5         N/A
                                                                           I was treated courteously by a labor market information
 I am able to compare this information with other                           was treated courteously by a labor market information
 information I use                                                        contact Disagree Disagree Neutral
                                                                           Strongly                            Agree Strongly Agree
 Strongly Disagree Disagree   Neutral   Agree    Strongly Agree           Strongly 1Disagree  2
                                                                                             Disagree  3
                                                                                                       Neutral  4 Agree Strongly Agree
                                                                                                                           5        N/A
        1             2         3         4           5           N/A              1            2          3         4             5     N/A
                                                                          My questions were adequately answered by a staff person
 The information is in sufficient geographic detail for me                My questions were adequately answered by a staff person
                                                                          Strongly Disagree Disagree  Neutral   Agree Strongly Agree
 Strongly Disagree Disagree   Neutral   Agree    Strongly Agree           Strongly 1
                                                                                   Disagree    2
                                                                                             Disagree   3
                                                                                                        Neutral  4 Agree Strongly Agree
                                                                                                                            5        N/A
        1             2         3         4           5           N/A              1            2          3         4             5     N/A


                                                                                                           Thank You for your participation
                                                                   OFFICE USE ONLY
  OES ❑                                 LAUS ❑                             LEHD ❑                                      Job vacancy ❑
  CES ❑                                 ES202 ❑                            Benefits ❑                                  Other ❑_______________

    Step 3 – Conduct Research and Take Action!
    Now that you have outlined what you will do and how you will do it, DO
IT! The results of your research should be delivered as a report. If you use a
survey instrument you will need to pretest it to ensure that it can be under-
stood by those you are surveying. Ask one of your best customers to try it!

    You may want to consider two sections to the final report of your find-
ings: comments and recommendations. Comments are the general comments
or quantitative results from the respondents. Recommendations are action-
able items the research has uncovered. Give the report to the Evaluation
Team for action!

Documenting Demand
        In addition to finding out the level of customer satisfaction, the
Customer Satisfaction Work Group recommends that states document
the demand for products and services. This may help in future market-
ing efforts, creating a customer survey base, and proving return on
investment for LMI products. There are four types of demand that
should be documented:

        1. Statistics (counting usage)
           Web site visits
           Specific page hits (Web Metrix)
           Visits – time on each page, data downloaded

        2. Customer lists (counting customers)
           By document
           By customer group

        3. Other Formal Satisfaction Feedback
           Special Surveys
           Focus Groups
           Directed Interviews

       4. Anecdotal (counting and quoting comments)
            “I can’t live without it”
          Proof of Usage
            “This is how I use it and its value to me”

Very Important
Notice that part of documenting demand calls for the creation of
customer lists. Since many states are delivering products over the
Internet, it is difficult to attach a name and address to many product
users. States may want to consider having users register, get onto
mailing lists or update lists, and other methods of gathering names,
addresses, and phone numbers of users.

It is also important to record customer contact information if they call,
write e-mail or fax requests for information or data.

In an era of continually need-
ing to prove the value of labor
market information products
and services, the nationwide
LMI system needs to build
relationships with the cus-
tomers it serves. To do that,
federal and state agencies need
to undertake an aggressive pro-
gram to actively listen to cus-
tomers and respond with
improved and innovative prod-

This handbook is designed to
help agencies measure cus-
tomer satisfaction.                                                  33

Shared By: