Marketing Project Report on Customer Satisfaction

Document Sample
Marketing Project Report on Customer Satisfaction Powered By Docstoc
					                             Final Report of the
                      Customer Satisfaction Work Group
                                    of the
                       Workforce Information Council

Section I – Executive Summary

         The Customer Satisfaction Work Group (Work Group) of the Workforce
Information Council (WIC) was appointed to explore the issue of customer satisfaction as
it relates to labor market information (LMI) providers and users of the information. The
Work Group was composed of eleven members, two support personnel and a consultant.
They met formally three times between October 2002 and June 2003, and gave quarterly
updates to the WIC.

        The Work Group utilized existing documents and gathered other reports to guide
its work. Many of the concepts employed are private sector marketing concepts. The
group verbalized the process of developing labor market information products and
services as being a process of data collection, data analysis and distribution of
information products and services. The process defined in this manner seems void of
customer satisfaction measurement and needs assessment. Therefore, the Work Group
concentrated on incorporating the needs and satisfaction of customers into the LMI
creation process.

       Two models were constructed to demonstrate how customer satisfaction and
needs assessment can guide the creation of meaningful labor market information and
move toward a demand-driven workforce information system. One model charts a state
scenario and the other extends the model to federal products. The result is a dynamic,
customer-driven labor market and workforce information system which is responsive to
customer needs and wants in a changing environment. The process embraces not only
the concept of assessing customer satisfaction, but also of working (through evaluation
teams and the WIC) toward continuous improvement of the system. This is the objective
of the requirement to “consult” with customers as set forth in Section 309 of the
Workforce Investment Act of 1998.

       As part of infusing customer satisfaction into the labor market information
development process, some critical components have been identified. They include:

    •   The establishment of evaluation teams in each state, charged with turning
        customer satisfaction assessment results into improved products and services, and
        with sharing customer satisfaction processes and improvements with other states
        and federal agencies;
    •   Nine measurable attributes which the Work Group believes will lead to more
        meaningful customer assessment activities and provide for easier information
        sharing;
    •   A methodology that state and federal agencies can employ to “think through” the
        customer satisfaction process. The methodology is displayed in a handbook
        developed by the Work Group which will be made available to assist federal and
                                                                                       1
        state agencies as they work to develop protocols and choose methods for
        assessment;
    •   Strategies for documenting demand for labor market information products which
        can provide guidance to funding agencies, in addition to encouraging broader use
        of available information.

     This document has five sections including this executive summary. Section II
outlines the Work Group’s approach to work. Section III discusses and models the
relationships between customer satisfaction and labor market information. Section IV
reviews pilot studies, and Section V of this document responds specifically to the charter
given to the Work Group by the WIC.

     The response in Section V notes findings by the group and then provides seven
actionable recommendations. These recommendations include establishing or identifying
a customer satisfaction focal point for sustaining the effort and providing on-going
customer satisfaction technical assistance, offering training to agency representatives on
implementing customer satisfaction programs, exploring customer satisfaction as a
deliverable in the BLS cooperative agreement and continuing it as a deliverable in the
ETA core products grant, and expanding the efforts of the Work Group into a demand-
driven marketing effort for workforce information.

     There are also attachments under various tabs. Tab III is a customer satisfaction
handbook. Tab IV reviews pilot study results. Tab V is the draft document “Workforce
Information Customer Satisfaction Assessment: A Primer for State and Local Planning”
developed by Mediacall, Inc. and the Heldrich Center. Tab VI documents BLS customer
satisfaction activities. Tab VII is the project work plan which set the course for the Work
Group. Tab VIII is feedback information from federal agencies on their customer
feedback programs. Tab IX is a tally of state customer satisfaction activities, and Tab X
is the first progress report of the Work Group.




                                                                                         2
Section II – Approach to Work

Introduction

    The Customer Satisfaction Work Group (Work Group) of the Workforce Information
Council (WIC) was appointed to explore the issue of customer satisfaction as it relates to
labor market information providers and users of the information. The WIC defined the
scope of the Work Group by providing a charter with five specific outcomes:

   1. What are the broad parameters for a customer satisfaction measurement system?
   2. What customers and products/services should the system address?
   3. How should the activities related to customer satisfaction measurement be
      communicated to state LMI offices and state employment statistics agencies?
      How can their support for the effort be encouraged?
   4. What technical assistance should be provided to states in implementing customer
      satisfaction measurement within their states?
   5. How should customer satisfaction measurement be incorporated into federal
      funding agreements for LMI activities?

   The Work Group was composed of eleven members, two support personnel and a
consultant, including:

   •   Olaf Bjorklund, Employment and Training Administration, Federal Co-chair
   •   James McFadden, Chief, Office of Occupational and Labor Market Information,
       Delaware Department of Labor, State Co-chair
   •   Garry Breedlove, Florida Agency for Workforce Innovation
   •   Tom Gallagher, Manager, Research and Planning, Wyoming Department of
       Employment
   •   Rod Fortran, New York Department of Labor, Division of Research and Statistics
   •   Yolanda Harris, Marketing Communication Specialist, Illinois Department of
       Employment Security
   •   Betty McGrath, Economist, North Carolina Employment Security Commission
   •   Peter Phelan, Center For Workforce Information and Analysis, Pennsylvania
       Department of Labor and Industry
   •   Mary Ann Regan, Director, Center for Workforce Information and Analysis,
       Pennsylvania Department of Labor and Industry
   •   Vivian Shapiro, Assistant Commissioner, New Jersey Department of Labor
   •   Stanley Stephenson, San Francisco Regional Commissioner, Bureau of Labor
       Statistics

Support Staff

   •   Dixie Sommers, Center on Education and Training for Employment, Ohio State
       University
   •   Nancy Pyon, Center on Education and Training for Employment, Ohio State
       University


                                                                                         3
Consultant

   •   Don Norton, Vice President, Mediacall Marketing and Advertising, Inc.

    The Work Group began meeting quarterly with the consultant in October 2002. Other
meetings were held in January and May 2003. The WIC was apprised of status of the
project at meetings in December 2002, March and June 2003. A presentation was made
on the Work Group efforts at the BLS LMI conference in May 2003.

       Several documents were used to guide the effort. Some of the documents were
developed by staff specifically for the project and others were collected to build in
previous efforts. These documents include:

             1. A draft “Customer Satisfaction Primer,” funded by the ETA and written
                by Mediacall and the Heldrich Center for Workforce Development at
                Rutgers. This document also notes what actions states are implementing,
                as well as private sector customer satisfaction practices;

             2. A report from the Bureau of Labor Statistics (BLS) on the type of
                customer satisfaction activities it has undertaken;

             3. A “Project Work Plan” developed by the Work Group;

             4. Examples of “Customer Feedback Information” collected from federal
                agencies; and

             5. Tally of Customer Satisfaction Activities currently being undertaken by
                states.

       Copies of these documents are attached as appendices to this document.
Similarly, the group developed a report at the mid-point of the process to document
resources. It is also attached as “Report 1.”




                                                                                        4
Approach to Work

        The Work Group began to approach customer satisfaction with concepts set forth
in the Primer, noted above. The group set forth working definitions. These include:

          o Customer satisfaction: meeting or exceeding the expectations of the
            customer. It is recognized that there may be many attributes to satisfying
            the customers.

          o Customer expectations: attributes that are in the mind of the customers,
            and consist of all parameters of what they, the customers, need, want and
            think they need and want.

          o Customer feedback: information gathered from the customers to assess
            their level of satisfaction.

        There were many discussions of “marketing” as it relates to labor market
information. The group embraced the concept of marketing as a process. In the private
sector, the process involves conducting research (listening to the customer to ascertain
needs and wants), developing products to meet those needs, promoting products, and then
re-evaluating product effectiveness. Products are changed through innovation or
discontinued, as noted below.




                                                                                      5
       The group verbalized the process of developing labor market information
products and services as being a process of data collection, data analysis and distribution
of information products and services. The process has not historically recognized the
customer as part of the development process, as noted below.

                      LMI Product Development Model




       Section 309 of the Workforce Investment Act of 1998 clearly mandates that
customer satisfaction data be gathered in an effort to continuously improve the LMI
system. The Work Group has been keenly aware of this mandate, and worked toward
developing a model which could incorporate private sector concepts into the labor market
information creation process.

        In that regard, the group was less concerned with a process that developed a
single number for measuring satisfaction. Rather, the group focused on ways to collect
information regarding how labor market information products can be improved to better
meet the needs of users. The resultant model could be seen as follows:

  Integrating Customer Satisfaction into LMI Product Development




                                                                                         6
      Based on this fundamental model of incorporating the needs and satisfaction of
customers into the LMI creation process, the work group then set about establishing
models which could allow this process to thrive in the complex state and federal
workforce information development system.

        In addition to seeking and responding to customer satisfaction input, the Work
Group identified the lack of documentation regarding the usefulness of and demand for
workforce and labor market information. Therefore, the Work Group’s effort focused on
developing a process for assessing customer satisfaction with and documenting demand
for labor market information.

Section III – Customer Satisfaction and Labor Market Information

Modeling the Process

        In addition to framing the approach to work with the models noted in Section II,
the Work Group established two models to further define the process that states and
federal agencies should use in assessing customer satisfaction. The first model is for
states and is outlined below:
              State Product Customer Needs and Satisfaction




       The following is a verbal description of this process:
          1. The process notes the three fundamentals of creating labor market and
              workforce information: data collection, data analysis and information
              distribution.


                                                                                      7
          2. Information distribution encompasses both products and services. These
             products and services are the items that are evaluated by the customers.
          3. Results of the assessment of customer satisfaction are given to an
             evaluation team at the state level. Work Group research determined that
             these teams already exist in many states as the entity which develops and
             distributes current information products.
          4. The evaluation team then makes changes at any one of the three levels of
             creating labor market information, as it is able depending on funding and
             other considerations.
          5. Furthermore, the evaluation team shares information with a customer
             satisfaction focal point – a central collecting point yet to be determined.
             Training on customer satisfaction best practices and techniques, as well as
             lessons learned on document preparation, then should be offered to states,
             perhaps through the Labor Market Information Training Institute.
          6. Finally, the state evaluation shares information with representatives at the
             federal level to facilitate improvement or changes to the workforce
             system.

        This same process should be used for developing new programs, products and
services. The second model is noted below. The right side of the model below is the
same as the state model above, but the model is completed by adding customer
satisfaction measurement at the national level and the relationship between federal and
state entities.
    System-wide Customer Satisfaction for Workforce Information




                                                                                       8
The following is a verbal description of this process:
   1. Federal activities are in blue, state activities in green, and those entities which are
       state and federal cooperative have both colors.
   2. The process notes the three fundamentals of creating labor market and workforce
       information as the central activity of both federal and state entities: data
       collection, data analysis and information distribution.
   3. The process described above is mirrored at the federal level for national or federal
       product customers. Results are reported to the WIC, which may choose to work
       through an on-going Customer Satisfaction Work Group. For products which
       have consortia or task forces, satisfaction results will be sent to that entity for
       change recommendations.
   4. The entity for making change may either be the Bureau of Labor Statistics or the
       Employment and Training Administration, which, like the state model, will effect
       change at any of the three levels of data collection, data analysis or information
       distribution.
   5. Federal entities then communicate the customer satisfaction results to the state
       focal point to facilitate possible improvements in state products and assessment
       practices.

    The end result of this entire process is a dynamic, customer-driven labor market and
workforce information system that is responsive to customer needs and wants in a
changing environment. The process embraces not only the concept of assessing customer
satisfaction, but also of working (through evaluation teams, federal agencies and the
WIC) toward continuous improvement of the system.

Components of Assessment

       Part of the charter from the WIC was to determine how to encourage support for
customer satisfaction assessment from the states. Work Group members concluded that
the process of assessment would have to be broken into easy-to-follow components to be
embraced by states. In addition, the group assumed that there would not be one
mandatory survey imposed upon the states. Therefore, one particular survey instrument
or assessment mechanism would not be recommended, but rather several assessment
options would be offered.

       To meet the objective of encouraging states to participate, the Work Group is
providing a handbook entitled “Customer Satisfaction Made Easy for Labor Market
Information Professionals.” The information discussed in this section of the report has
been discussed by the group and formulated into the development of the handbook.

Who to Assess:
   The Work Group consulted Section 309 of the Workforce Investment Act of 1998
which clearly indicates the customer groups need to be assessed. They include:

•   Businesses make decisions about product and financial markets, business location, and
    employee recruitment, compensation, and training.
•   Individuals, including young people and adults, make choices about careers, education,
    training, and job search.

                                                                                             9
•   Elected Officials and Policy Makers, including Workforce Investment Boards, make decisions
    about law, policy, budgets, and regulations.
•   Program Planners determine what workforce and economic development services to provide,
    and evaluate program performance.
•   Education and Training Providers, including teachers and curriculum specialists, design,
    deliver, and evaluate programs that develop students’ knowledge and skills.
•   Intermediaries, such as parents, counselors, teachers, mentors, placement workers, and
    case workers, assist others in choosing education and training opportunities, and in finding
    employment, and
•   Researchers and the Media study how the labor market works, and conduct policy research.

       The Work Group recommends that agencies place the customer into one of these
groups. This will facilitate information sharing between states and federal organizations
in terms of how customer groups utilize core products.

        In addition, the Work Group believes it is important that products be assessed at a
customer-familiar level. As an illustration, a candy customer sees a chocolate bar. The
chocolate bar manufacturer sees that same bar as sugar, cocoa, milk and other
ingredients. In LMI terms, the customer sees a brochure that lists projected job
opportunities. The LMI shop sees that same brochure as projections data, skills sets and
wage records. State and federal agencies should ask the customer about the satisfaction
level in terms familiar to the customer, but report a level of satisfaction in terms of each
core data product.

State Evaluation Team:
        The models for customer satisfaction noted above feature a state evaluation team.
This team has three purposes:

            o To oversee the customer satisfaction effort at the state level, and
              incorporate results of that effort into continuous improvement of state
              products;
            o To share information gathered in the customer satisfaction effort, both in
              terms of process and conclusions, with other states;
            o To share information gathered in the customer satisfaction effort with
              representatives and organizations at the federal level to encourage
              systemic change.

        States may want to consider having a publications specialist, an LMI projections
professional and an information analyst on the team to represent some of the individuals
involved in product production. It should be noted that the composition of these teams
will vary by state. During research developed by the group it was discovered that many
states already have such teams that also function as the teams that develop LMI products.

Attributes:
        A review of “customer satisfaction” surveys at the state and federal levels found
that most surveys actually are an attempt to measure customer service. The Work Group
believes that customer service is one aspect of the total customer satisfaction picture.
These service-oriented questions include “Was our staff helpful?” “Could you find the
                                                                                       10
information easily?” To provide further guidance here, the Work Group identified nine
(9) attributes which states should consider measuring when assessing customer
satisfaction. They include:

       Accuracy: The information is accurate enough for the customer’s use.
       Relevancy: The information is relevant to solving the customer’s problem.
       Accessibility: The customer could easily access the information.
       Understandability: The customer could understand what the content means.
       Comparability: The customer is able to compare the information with other
       information they use.
       Geographic Detail: The information is in sufficient geographic detail.
       Timely: The information is timely enough for the customer’s needs.
       Completeness: The customer could solve the problem with this information only.
       Importance: The information is important in the customer’s overall problem
       solving work.

    Not all of these apply to every information piece all the time. But these attributes can
give information providers guidance on how to make products more meaningful, and
provide evidence to support new product development.

Methodologies:
        It is important to emphasize that customer satisfaction assessment is not just a
survey or a number. As noted earlier, the Work Group believes that customer assessment
is a process and there is a risk of being number-oriented. If a state or federal agency
scores a 77% on a satisfaction rating, what does this tell them about how to alter a
product to make it more meaningful? Quite possibly, the agency knows nothing more
about how to improve the product.

         This number-orientation could further be complicated by the very nature of labor
market information research. As an industry, LMI professionals are numbers people.
Methodologies that profess to gauge customer satisfaction through anything less than a
stratified random sample with significant return rate could be rejected outright. Customer
satisfaction measurement through other methods could represent a paradigm shift within
the LMI industry.

       Furthermore, the Employment and Training Administration has discouraged the
development of one preferred methodology and has, in fact, encouraged utilizing a
variety of approaches to customer satisfaction assessment. The Work Group also
embraces that approach.

       However, customers can and should (as the sample survey in the handbook points
out) be asked a relative level of satisfaction with a document on a Likert scale of 1 to 5
with 5 being highest.

       To provide a framework for this descriptive research, the Work Group endorses a
process of Problem Definition and Problem Solution, which has been refined to a simple
“Assessment Statement.” This protocol helps enable agencies to choose a methodology
which best meets the needs identified in the problem definition, as noted below.
                                                                                         11
        This process provides a valid framework for a state to “think through” the
customer satisfaction assessment process. Based on this framework, the Work Group
recommends that a variety of assessment mechanisms be used to assess customer
satisfaction, including but not limited to phone, mail and Internet surveys, focus groups
and interviews.

      In fact, many states have used even more assessment methods than those listed
which provide actionable customer satisfaction feedback. Two of the most common
methods being utilized are surveys and focus groups. To facilitate this effort, the Work
Group developed a survey which can be adapted for states to administer, and a document
on how to conduct a focus group. Again, details of these processes are available in the
handbook.

       Section IV of this document discusses how the methodologies in this document
have been piloted.

Needs Assessment:
        Customer satisfaction implies that there is a product with which a customer can
express satisfaction or dissatisfaction. A truly proactive approach is to identify a target
customer group and assess their needs for products that may or may not exist. The next
step would be to create a product to meet those needs and then assess satisfaction with
that product. One of the piloting efforts taking place in Pennsylvania is conducting this
proactive technique with local workforce board professionals utilizing a similar research
protocol as outlined above.




                                                                                        12
Documenting Demand

       The Work Group believes that there is not a clear picture of the utility of or
demand for labor market information. Statistics regarding demand and usage are
important in that they can provide guidance to funding agencies, in addition to
encouraging broader use of available information.

       With this in mind, the group suggests that states document items such as:

   •   Statistics of product usage;
   •   Web site visits, using Web Metrix and other programs;
   •   Developing customer lists by product;
   •   Other Formal Satisfaction Feedback, such as special surveys, focus groups and
       directed interviews;
   •   Anecdotal information, such as testimonials from customers regarding how
       information is used to solve problems;
   •   Proof of usage, through letters or other measurements where people have noted
       “This is how I use it and its value to me.”

Marketing Implications

       Part of the issue of documenting demand concerns the fact that labor market
information products have never been aggressively marketed except at the state level in a
few states. The Work Group believes that while it is important to satisfy existing
customers, there are tens of thousands of potential customers who are in need of
information products but do not know they exist. As noted earlier, customer satisfaction
is a marketing concept. The marketing cycle is not complete unless products are
adequately promoted.

        Only when LMI products are effectively marketed can their demand and usage be
fully document. This effort has not yet been undertaken, and possibly to the detriment of
economic development and other efforts in communities throughout the nation.

Section IV – Pilot Studies

        The Work Group believes it is important to provide states and federal agencies
with customer satisfaction assessment tools which can be understood and then
implemented. Toward that end, several of the methodologies outlined in the previous
section are being piloted.

       The purposes of the pilot studies include:

           •   To test the usability of the nine attributes in assessing customer
               satisfaction measurement;
           •   To test a proposed survey instrument;
           •   To provide examples of focus group questions and processes;
           •   To refine protocols that will be outlined in the handbook.

                                                                                      13
        Three states volunteered to be included in the study. The states and the projects
are as follows:

        Delaware:     A survey followed by directed interviews on a monthly labor
                      market information brochure with a defined audience;

                      A focus group with a finite group of state leaders to determine the
                      use of existing products and need for new products;

        Illinois:     Focus groups with two similar but geographically diverse customer
                      groups regarding a set of career counseling documents;

        Pennsylvania: Open-ended survey and directed interviews/focus groups with
                      local workforce board representatives to determine products and
                      services that will be needed during the coming year (needs
                      assessment.)

      An attachment to this document provides detailed protocol, comments and
recommendations which have resulted from these pilots.

Section V – Response to Charter

       The Customer Satisfaction Work Group is providing the following direct
responses to the charter from the Workforce Information Council. The responses take the
form of findings and actionable recommendations.

   1.       What are the broad parameters for a customer satisfaction measurement
            system?
Response
        The Customer Satisfaction Work Group believes that customer satisfaction should
be incorporated into the process of developing labor market information products.
Following the model set forth in this report, customer satisfaction should be measured at
the local, state and national levels. There are nine actionable attributes which should be
measured, in addition to assessing unmet information needs. State and federal agencies
should use a variety of assessment mechanisms including but not limited to phone, mail
and Internet surveys, focus groups and interviews. States and federal agencies should
recognize the importance of mechanisms such as the WIC and state evaluation teams
(many already exist) for the implementation of customer recommendations for product
improvement.

        2. What customers and products/services should the system address?
Response
     We believe in assessing customers as outlined in WIA section 309 with emphasis on
the business community. While we recognize that states and federal agencies do not have
the resources to assess customer satisfaction on every product at all times, over time, we
believe agencies should work toward assessing all products and services that are offered.
It is important that assessment be made of the products and services as our customers
                                                                                       14
know them (pamphlets, brochures, websites, documents, other information), and not as
core products or statistical data products (CES, OES) as they are known in LMI
cooperative agreements. However, for some intensive data users, the product may indeed
be the statistical data product.


           3. How should the activities related to customer satisfaction measurement be
              communicated to state LMI offices and state employment statistics
              agencies? How can their support for the effort be encouraged?
Response
        The Work Group has overseen the development and publication of a handbook,
which is a logical outgrowth of the Primer. We also believe that narrative portions of
state one-stop grant annual reports to the ETA should be summarized and published. And
the group believes it is important to include other stakeholder groups in our customer
satisfaction work group activities. Common LMI websites should be used to post results
of activities and success stories, and seminars on customer satisfaction assessment should
be presented at national and regional LMI and workforce development meetings.

           4. What technical assistance should be provided to states in implementing
              customer satisfaction measurement within their states?
Response
        In addition to providing the workbook, the Work Group has piloted studies of
methodologies noted in this report. We encourage on-going independent validation of
customer satisfaction methodologies recommended by the work group, and encourage
making available the handbook, a tutorial, and collateral materials to implement LMI-
specific methodologies for conducting customer satisfaction assessments and forming
customer service evaluation teams in states.

           5. How should customer satisfaction measurement be incorporated into
              federal funding agreements for LMI activities?
Response
        The Work Group is putting forward as an actionable recommendation the
continuation of the ETA core products grant requirement to assess and report customer
satisfaction, and exploring customer satisfaction measurement as a funded deliverable in
the BLS cooperative agreement.

Recommendation for Action
   1. We recommend that states, ETA and BLS implement the process outlined in this
      report.
   2. We recommend that a work group or organization be established, appointed or
      identified for sustaining the customer service effort. This group or organization
      should be charged with ensuring that on-going customer satisfaction technical
      assistance is provided and that customer satisfaction activities are documented,
      synthesized and made available to the WIC and states.
   3. We recommend the WIC use the documented state customer satisfaction results to
      guide recommendations regarding the workforce information system.
   4. We recommend training sessions be offered through existing training mechanisms
      (such as the LMI Institute) on implementing customer satisfaction programs and
                                                                                        15
      the importance of the customer service activity to a more stable funding
      environment.
   5. We recommend that capacity be built to sustain a customer satisfaction
      assessment process for Labor Market Information.
             • We recommend that customer satisfaction measurement be explored as
                 a deliverable in the BLS cooperative agreement, and that it be
                 continued as a deliverable in the ETA core products grant.
   6. We recommend that customer satisfaction assessment be expanded into a
      demand-driven marketing effort for workforce information.

Section VI – Conclusion

       The Customer Satisfaction Work Group of the Workforce Information Council
was appointed to explore the issue of customer satisfaction as it relates to labor market
information providers and users of the information. The group believes that listening to
the voice of the customer is a critical component in the process of developing more
meaningful labor market information products and services.

        The group has been able to document a process which, while not mandated to be
followed, provides a practical and valid approach to infusing the needs and wants of
customers into product development. A handbook is being provided as a part of this
effort that can serve as a guide to agencies seeking to utilize this process.

         Work Group members believe that the success of building a demand driven
workforce and labor market information system is dependent upon having states and
federal agencies conduct customer satisfaction assessment, share the results of both the
processes and information gathered, document demand for information products and
services and begin a process of fully marketing and promoting the wealth of information
that is available to benefit the public and private sectors in the nation.




                                                                                      16

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:180
posted:1/16/2012
language:English
pages:16
Description: Marketing Project Report on Customer Satisfaction document sample