Quality and Standardisation of Processes by gyvwpsjkko


                                    Contents                                                                1 of 14


                                      Quality and Standardisation of Processes
               Hans Viggo Sæbø
               Statistics Norway. Department for Planning and Finance, Kongens gate 6, 0033 Oslo, Norway.
               E-mail: hvs@ssb.no


               Quality of statistical products is only achieved through good production processes. Improvement and

               standardisation of these processes have been emphasised by several statistical institutes and international
               statistical organisations during the last years. Quality work in statistics has gradually developed from
               assessing and reporting on output quality to a process-oriented approach, following the whole statistical
               value chain. The development has been facilitated by some basic and common principles of quality

               This paper presents a summary of the recent work on improving quality and standardisation of statistical
               production processes in Statistics Norway. This includes continuous improvements on the level of each
               process, a process standardisation program across the whole organisation and the development of completely
               new processes, due to dramatic changes in technology and other factors.
                                    Contents                                                             2 of 14

               Emphasis is put on some preconditions for succeeding with such work, in particular management support.
               Ensuring sufficient resources for development work requires strong prioritisation. Project portfolio
               management has been introduced in Statistics Norway to facilitate good prioritisation, and is briefly
               described. The paper also refers to some international efforts supporting national standardisation and
               development work.

               Process variables

               Sæbø (2007) has discussed measurement of process variables within the framework of quality assessment, an
               important technique for continuous improvement. Figure 1 illustrates the main links between process

               variables and other data quality assessment methods. Eurostat (2007) provides an overview of such methods.

               Systematic monitoring of the process variables with a high influence on resulting product quality (such as
               accuracy, timeliness and comparability of statistics) is necessary to improve the processes. Examples of
               process variables that have been used by statistical institutes are measurements of interviewer performance,
               non response of different types, costs and use of time for different processes.

                                    Contents                                                                3 of 14

                               Figure 1. Process variables in the framework of data quality assessment.

                                                                          Certification and labelling

                                           Users                       Self assessments      Audits and
                                                                                             peer reviews

                                      Statistical Output:               Quality indicators   Quality reports
                                         Product quality

                                    Statistical Processes:

                                         Process quality                Process variables

               What has often been lacking is a systematic approach to identifying and measuring such variables. Work on
               establishing quality assessment systems including self-assessments and audits, is important to facilitate this.
               In Statistics Norway a quality assessment has been carried out by using the DESAP self-assessment
               questionnaire, see Eurostat (2003). All of our approximately 200 statistics have been covered in 2008. Even
               if this has been a self assessment, people responsible for the different processes have been supported by
               central quality or methodology staff in filling out the questionnaires. Main areas of improvement identified
                                     Contents                                                           4 of 14

               comprise data collection and editing. In both these areas more study of process variables are recommended.
               These areas are also emphasised in our standardisation programme discussed below.

               Standardisation of Processes
               Statistics Norway is running a programme on the standardisation of processes, systems and methods
               (abbreviated FOSS). FOSS can be regarded as a business reengineering project based on our quality work,

               strategies and international trends.

               The main objective of FOSS is to improve quality and efficiency in the production of statistics.
               Standardisation will reduce risks and facilitate internal rotation. FOSS also covers quality assurance and
               human resource and organisational development supporting standardisation.

               Previously planned development activities have been reviewed and fitted into this framework. As a basis for
               the standardisation work we have used a detailed process description or business process model, see
               Statistics Norway (2008). All projects are related to this model. Only systems and methods that are applied
               for the same or comparable processes can in principle be standardised. Hence the business model represents
               a precondition for FOSS. Figure 2 shows the model on an overall level (there are up to 3 or 4 underlying
                                   Contents                                                              5 of 14


                                       Figure 2. The Statistics Norway business process model.

               Important projects that are related to the development of corporate framework systems include:
               •    Check-list and indicators for data collection
               •    System for interview surveys
               •    Coordination of samples for business surveys
               •    A framework system for data editing and estimation
               •    A coherent metadata systems environment
               •    Development of a geo database for ground property maps

           T   •    Improving data archiving and secondary use of data for researchers
                                    Contents                                                                  6 of 14

               •     Extensive developments in dissemination and redesign of website
               •     Overview of guidelines and handbooks
               •     Quality assessment covering all statistical products and processes (mentioned above)

               The central organisation of FOSS has consisted of a board headed by the Director General and a coordination
               group with members from different parts of the organisation. This group provides support to the projects
               managers, and ensures that projects are implemented according to the objectives and form a totality. In many

               cases members of the coordination group have participated in the projects.

               More than 30 projects have started by now, of which about 20 have been completed. Most of the FOSS
               projects are short (3 – 6 months). That makes it easier for participants to allocate a large part of their work to
               these projects as long as they last. That means that large projects have been split into several smaller
               projects. Possible gains in quality and efficiency are considered for each project.

               Processes and New Technology
               A continuous monitoring system is needed to be sure that quality remains high, and there should always be
               an objective to improve processes over time. However, continuous improvement work does not necessarily
               encourage replacing a process with a completely different one. Some times a shift in technology or
           T   methodology occurs, which offers completely new ways of doing things. In this case one should consider
                                    Contents                                                               7 of 14

               replacing the old process with a new one. A historical example of such a shift was the introduction of
               Internet in 1995. This changed the way we disseminate statistics completely, even if it still took some years
               before our statistical products had been adapted to the new medium.

               A similar situation regards electronic data collection routines, which recently have been introduced on a
               larger scale. In Norway all businesses have had the opportunity to submit data electronically since 2004. The
               number of businesses reporting their data electronically has increased slowly from 33 percent in 2005 to 53

               percent in 2008. However, paper questionnaires have normally been sent out to all participants in surveys

               anyway. For some surveys we have cut this service, and informed the participants that they have to ask for
               paper forms if they are not going submit data electronically. This has led to a boost in electronic data
               submission for these surveys, even if the number of phone calls for support also increased for a period.

               Table 1 is an example showing the development of electronic data collection reporting for wage statistics.
               Here, automatic distribution of paper questionnaires was cut in 2007. The number of telephone calls
               increased significantly the first year, but is now decreasing. Non response has not increased significantly.
               Based on experiences like this, we are considering cutting distribution of paper for all business surveys.
               After a transition period this will probably increase efficiency and quality (at least timeliness) of data
               collection and reduce costs.
                                    Contents                                                               8 of 14

                   Table 1. Electronic data submission and number of support telephone calls of wage statistics in
                                                   Statistics Norway. 2006 –2008.

                                                    Share of respondents reporting    Number of phone calls for
                                                    electronically. Percent           support
                2006                                53                                5901
                2007 – Cut in paper distribution    87                                9240

                2008                                95                                6557

               Another example where new solutions are needed is the household expenditure survey. Decreasing response
               rates has led to decreased quality, and more resources have to be spent on keeping the response rates on an
               acceptable level. New data sources such as electronic cards and traces are considered, and sooner or later this
               kind of data will probably have to be collected in a new way rather than by the traditional household
               accounting questionnaire.

               What is important when working with processes is to be open to new possibilities. Management has to
               encourage new ideas and allocate enough time for rethinking solutions.

                                    Contents                                                              9 of 14


               Management Support
               Any quality work in an organisation is deemed to failure without management support. This in particular
               goes for standardisation since different parts of the organisation do not necessarily benefit from a standard
               solution even if the organisation as a whole of course does. The FOSS programme has been initiated by top
               management. As mentioned, the Director General has chaired its steering group. Close follow up with
               monthly steering meetings, coordinated with Statistics Norway work on planning and budgets has been an

               important success factor for our standardisation efforts. This coordination has been further emphasised by a

               recent reorganisation, including centralisation of IT resources and the formation of a Department for
               Planning and Finance. This department is responsible for central coordination of quality, standardisation,
               security, international cooperation, central planning and follow up including budgeting and accounting.

               Centralisation of IT is an advantage for standardisation, but it represents a challenge when it comes to
               management of projects and activities demanding scarce IT resources. Statistics Norway has therefore started
               development and implementation of portfolio management of all large development projects including the
               standardisation projects.

                                      Contents                                                            10 of 14

               Portfolio Management
               The purpose of portfolio management is to improve quality and efficiency by better prioritisation of projects
               based on systematic information. In addition to the normal information on projects included in project
               mandates and plans, a set of key parameters that can be measured or given values (points) as a basis for
               prioritisation has been developed. In addition to these parameters with summary points, final prioritisation
               has to be based on judgements including other information such as the purpose and cost of projects, available
               resources and competence, interconnections between projects and the totality of the project portfolio.

               The parameters used for prioritisation of Statistics Norway are:
               Parameters which are given values on a scale
                      Coherence with strategies and plans
                      Contribution to standardisation of processes
                      Laws and EU-regulations
                      Reduced risk in Statistics Norway
                      Quality (relevance, accuracy, timeliness and punctuality, coherence and comparability, accessibility
                       and clarity)
                      Response burden
                                   Contents                                                             11 of 14


               Other parameters to be taken into account
                      Competence requirements

                      Links with other projects, totality of the project portfolio

               Judgements have to be used for final prioritisation carried out by the top management, but these judgements
               will be based on systematic information about the projects.

               Portfolio management in Statistics Norway is partly based on experiences from the standardisation
               programme. It will comprise the projects included in the standardisation programme, which will be adapted
               to portfolio management.

                                         Contents                                                                         12 of 14

               International Support
               Quality work on a national basis can benefit from international work, through sharing of experiences and
               knowledge of best practices, but also by support from common frameworks, recommendations and measures
               taken by international statistical organisations.

               In Europe, the implementation of the European Statistics Code of Practice followed by self-assessments and
               peer reviews during the period 2006 – 2008, has been an important milestone in the work on quality in

               statistical institutions. During the last year European Task Forces following the Cracow Action Plan 1 have

               completed their work, also emphasising the need for improving and standardisation of statistical processes.

               On the Nordic level there has been cooperation on statistical solutions for many years. This includes working
               groups in many fields, among them quality and standardisation of processes.

                   Three Task Forces were established by the European Statistical Programme Committee in its meeting in Krakow in February 2007.
           T       Their reports were discussed in the conference “Work in Progress” in The Hague 15. and 16, January 2009
                                    Contents                                                              13 of 14

               The work on improving processes is the key to improving quality of statistics as well as the effectiveness of
               statistical organisations. In this area there has been a lot of work and progress in recent years. Work on the
               national level has been supported by international efforts.

               Study of process variables is crucial for continuous improvements, whereas standardisation by implementing
               best practices has been focused in many countries. Important preconditions for success in this area comprise

               development and use of business models describing the processes. However, the single most important

               success factor in implementing standardisation and improvements is management support. In fact,
               standardisation normally has to be management driven. Management should also facilitate completely new
               solutions by creating a climate for new ideas and ensuring time for pursuing these.

               [1]   Eurostat (2003): Checklist for Survey Managers (DESAP).
                     Published on http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/quality_reporting
               [2]   Eurostat (2007): Handbook on Data Quality - Assessment Methods and Tools.
                     Published on http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/quality_reporting

                                   Contents                                                           14 of 14

               [3]   Statistics Norway (2008): Statistics Norway’s Business Process Model. Documents 2008/17, Statistics
                     Norway 2008. Published on:
               [4]   Sæbø. H.V. (2007): Measurement of process variables within the framework of quality assessment.
                     Paper for the ISI Conference 2007, Lisboa, 22-29 August 2007.


To top