KPI-1-User-satisfaction

Document Sample
KPI-1-User-satisfaction Powered By Docstoc
					KPI 1
Objective

User satisfaction

To measure the extent to which users are satisfied with the ICT services they receive.

Reason for the KPI
User perceptions of the service are a critical element of any performance management system. The KPI reflects the emphasis in both the Best Value regime and the e-government policy in looking at services offered from the viewpoint of those who use it. In the case of support services such as ICT this philosophy means listening carefully to the viewpoint of the user. In general, this means employees in the organisation but, as ICT is increasingly used to provide services direct to the customer, this will mean the general public. By adopting this KPI, Socitm is reinforcing the importance of the user perception. Measuring this encourages organisations to understand how services may be improved. It should also encourage ways of measuring satisfaction other than just a survey by questionnaire, eg measuring the satisfaction of those who have experienced a problem and its resolution.

Definition
This is the perception of the service user as measured in any form of survey. Socitm believes that a survey carried out by an independent party is more valid than a survey carried out by the ICT unit.

Composition
The KPI is the answer to the question ‘How do you rate the overall ICT service you receive?’ The service users responding to the survey should be invited to indicate on a scale of 1 to 7 where 1 is poor and 7 is excellent. Please note that no other words should be attributed to the intermediate points on the scale. The number of service users surveyed should also be provided. A minimum sample of 100 users is recommended. Survey information that is more than 24 months old is invalid and should not be submitted. The use of an agent external to the ICT unit should also be stated.

Example
A survey where 300 services users respond and 10 users score 1 20 users score 2 30 users score 3 80 users score 4 70 users score 5 60 users score 6 30 users score 7 gives an average score of 4.60.

KPI 1

Result

Full survey of all users     Enter the average score per service user where a confidential survey of all/most users has been carried out (score to 2 decimal places) Enter the number of service users invited to respond Enter the number of services users who responded Indicate it you have used an agent external to the ICT unit for carrying out the survey.

1



Enter date the survey was carried out

Short survey of users with recent problems     Enter the average score per service user where a poll of those who reported problems or received services has been carried out (score to 2 decimal places) Enter the number of service users invited to respond Enter the number of services users who responded Enter the date the survey was carried out

The public        Enter the average score per member of the public who have been surveyed on any ICT service provided by the council (score to 2 decimal places) Enter the number of members of the public invited to respond Enter the number of members of the public who responded Enter the date the survey was reported Indicate whether you have improved your overall performance on this KPI in the past twelve months (yes/no) Indicate whether this KPI has been validated by an external organisation (yes/no) Enter any qualifying comment

2

KPI 2
Objective

Resolution of reported incidents

To measure the efficiency of the ICT unit in restoring the service within an agreed time scale after an operational incident has been reported by a user.

Reason for the KPI
Much of the resource and organisation of any ICT service is focused on resolving operational incidents quickly and accurately. This KPI concentrates on the success in dealing with this requirement. The definition assumes that the ICT service will set targets for problem resolution that are part of a formal service agreement (or published as standards widely throughout the council, perhaps via the intranet) rather than just relying on best endeavours. Socitm supports this as the way to manage this part of the service.

Definition
This is the percentage of calls when a service is restored for use within a time scale agreed by both service user and provider after an operational incident has been reported. An operational incident is defined as an unexpected event that prevents the user from carrying out normal work. The aim of this KPI is to measure performance against a promise or standard timetables rather than against a precise technical definition of a problem. The agreed time scale starts when the service user reports the incident (by phone, fax, e-mail or in person) and ends when the user is able to return to normal working (not necessarily when the underlying problem has been rectified).

Composition
a) b) For each reported incident keep a record of both the agreed time scale for restoring the service and the actual time taken. Take the total number of operational incidents reported on average each week to the ICT unit (almost always via a help desk or service desk). A representative sample will be sufficient if a full record is not available. Divide the number of incidents, when the service is restored within the agreed time scale, by the total number of incidents and then express as a percentage. Divide the number of incidents, when the service is restored within four hours, by the total number of problems reported and then express as a percentage.

c) d)

Note
Some incidents will be very easy to solve (eg can be solved immediately) and others may take weeks to solve. Most ICT units will prioritise problems by severity, by ease of fixing etc or by the number or type of persons affected They will agree beforehand the categories to apply (probably through some service level agreement) and will confirm the appropriate category when the incident is reported, or very soon after. The service is deemed to be restored when the user can continue with normal work. It might be that the ICT unit achieves this through a temporary measure (eg loan of equipment) rather than through a permanent resolution of the problem. In this case it is the restoration of the service that matters, not the technical fix to the problem. The number of operational incidents should also be provided to give some scale of the overall work. This in itself is one of the most useful indicators for customer service, especially when measured over time, because it indicates the success in reducing the number of reported incidents (eg actions taken to deal with common problems, etc).

3

KPI 2
 

Result

Enter the percentage of calls (operational incidents) when a service is restored within the time scale agreed. Enter the percentage of all calls when a service is restored within the following working hours. a) b) c) d) 0 – 4 hours 4 – 8 hours 8 – 16 hours over 16 hours

    

Enter the average number of operational incidents reported per week. Indicate whether the resolution of operational incidents is covered by a formal service agreement with users, specifying time to resolve (yes/no) Indicate whether you have improved your overall performance on this KPI in the past twelve months (yes/no) Indicate whether this KPI has been validated by an external organisation (yes/no) Enter any qualifying comment

4

KPI 3
Objective

Percentage of successful projects 2003/04

To measure the degree of success of ICT projects as determined by those who are the main beneficiaries of the project.

Reason for the KPI
Project activity counts for a significant part of any ICT resources and managing projects successfully should be an important indicator for any ICT service. This, however, is not just in the hands of the head of ICT. Project success reflects how well the organisation as a whole recognises project management as a key competency. The Head of ICT should at least have some influence in securing this recognition by the organisation, hence the emphasis in the definition of assessing project success from the viewpoint of the project sponsor (or project review board). The implementation of e-government plans, in particular, will test the capacity of the organisation to manage projects successfully.

Definition
A project involves the delivery of a product or service that meets a set of objectives agreed at the start. It may include internal ICT projects for the ICT service. Success is measured by the customer agreeing that the objectives have been met. A project can be defined as a planned set of deliverables using an agreed budget delivered to an agreed timetable and at an agreed standard. The project activity can be any work that involves a significant ICT component (say, more than one person month of resource). It can be defined as a single project, or a stage within a larger project. To complete this KPI the project should have been defined as a project at its inception and not later in order to complete this KPI. Socitm believes that a formal project management methodology (eg PRINCE2, or a customised version) should be used in order to know whether projects are successful or not. If no such methodology is in use, then this KPI will be much more difficult to collect and will have less credibility than in those organisations where a methodology has been applied. The key features of a methodology include:       Definition of a project Agreement of a project sponsor Ownership of the project from those who have a stake in its outcome (eg Project Review Board) Mechanism for managing and organising a project Stages to completion of a project Process for completing and reviewing project

Socitm recommends using a project success index for each project, which is built up as follows: (i) At the start of each project the project sponsor (or Project Review Board) agrees in advance the critical success criteria for that project (approximately 3 to 6 such criteria) and weight them out of 100. (Some criteria might be more critical than others). (ii) At the end of the project the same person, or group, assesses the performance of the project against each of those criteria. (iii) The index is calculated by applying the assessment of the performance against each of the weighted criteria, before totalling the score which will be expressed as a percentage. The criteria will vary from one project to another but they are likely to include:    completion within budget completion by agreed time scale achievement of business benefits

5



smooth transition to new facilities.

Composition
Use a standard project success index for each project Calculate the rating for each project completed Calculate an average rating for all projects assessed by the project success index.

Example
Project 1 has a project success index of 70% Criteria Criteria 1 Criteria 2 Criteria 3 Total Weighting 50 30 20 100 Assessment 30 30 10 70

The average weighting for all projects is the addition of the individual project success results, in this case 70%, divided by the number of projects.
KPI 3
All projects  Please state whether your organisation uses a formal project management methodology for all projects, whether or not projects include any ICT component. All projects that include any ICT component          Please state whether your organisation uses a formal project management methodology for ICT projects. (yes/no) Advise on the methodology used: PRINCE/ PRINCE 2 /PRINCE modified/ Internal system/ Other Enter the number of projects completed in the last financial year using a project management methodology. Enter the number of projects subject to formal post-implementation review. Enter the number of projects entered in the project success index. Enter the project success index averaged across projects as agreed by customers. Indicate whether you have improved your overall performance on this KPI in the past twelve months (yes/no) Indicate whether this KPI has been validated by an external organisation (yes/no) Enter any qualifying comment

Result

6

KPI 4
Objective

Acquisition costs of workstation 2003/04

To measure the total procurement cost of a workstation.

Reason for the KPI
Buying ICT facilities should be a test of the organisation’s purchasing ability, in a market that changes constantly, for a set of commodities that take up a gradually increasing percentage of overall resources.

Definition
The definition of a workstation is any device that accesses data. For the purposes of this KPI it is restricted to PCs, laptops, or ‘thin clients’ (or mainframe terminals). It excludes printers, scanners and devices such as PDAs, palmtops, electronic organisers. The three categories of workstation covered by this KPI are:             personal computer laptop ‘thin client’/mainframe terminal Keyboard Monitor CPU Network card Sound card Speakers CD, floppy and DVD drives Desktop operating system The equipment and software costs

The workstation includes, where relevant:

The costs should comprise:  The initial costs of procurement and installation which include: Administrative cost of procurement Physical cost of installation Initial hand over training Asset management Setting up help desk Management overheads, including an appropriate share of support service costs charged to the ICT service (costs of financial, personnel, legal and premises) Travel time and vehicle costs The costs should include work covered by the following Socitm definition of service elements: SE306 Technology provision

Composition
(i) (ii) Take both items of cost in turn for each category. Divide item 1 by total number of workstations procured in each category during the year. Notes:

7

 

Any annual lease/rental costs should be multiplied by 3 to give the equivalent to an outright purchase cost. Equipment costs should include operating system software but not general office products such as MS Office.

Example
Equipment costs a) Total purchase costs b) Annual rental/lease costs (where applicable) c) No of years to rent/lease d) Equivalent purchase costs e) Total equipment/ software costs (a+d) f) No workstations acquired g) Unit cost per workstation (e/f) Procurement and installation costs h) Salaries, NI and superannuation i) Travel expenses j) ICT expenses k) Management overheads l) Total procurement and installation costs (a+b+c+d) m) No workstations acquired n) Unit cost per workstation (e/f) o) Total acquisition cost per workstation (g+n)
KPI 4 Result
Equipment     Enter cost per PC Enter cost per laptop Enter cost per thin client Indicate whether you have improved your overall performance on this KPI in the past twelve months (yes/no) Procurement & installation Total cost per workstation

PCs £80,000 £40,000 3 £120,000 £200,000 200 £1,000

Laptops £51,000 £8,000 3 £24,000 £75,000 50 £1,500

Thin clients £8,000 £4,000 3 £12,000 £20,000 50 £400

£8,000 £1,000 £2,000 £3,000 £14,000 200 £70 £1,070

£3,000 £500 £400 £600 £4,500 50 £90 £1,590

£2,000 £200 £500 £300 £3,000 50 £60 £460

8

 

Indicate whether this KPI has been validated by an external organisation (yes/no) Enter any qualifying comment

9

KPI 5
Objective

Cost per connection to voice network

To measure the total life-cycle costs of a connection to the corporate voice network.

Reason for the KPI
Managing ICT costs is a critical part of any ICT service in a public sector organisation. Bad decisions and poor organisation can quickly lead to escalating costs. Life-cycle costing is acknowledged to be the best approach to managing costs of any ICT facility. This KPI focuses on an indicator for the voice network.

Definition
A device is taken to be any device that uses voice transmission and is connected to a voice network, taking up an active network port. The actual or notional charge that would apply to the cost of providing a connection to an internally managed voice network. This comprises the initial capital costs of any investment and the ongoing support costs. The initial capital costs should be:  The costs of procurement and installation of any new network in the last three years, including the current year (converted as an annual cost per connection assuming that costs are recovered over three years). If costs are in fact covered over, say, five or seven years, then this will need to be converted to a three-year figure to ensure ‘like-for-like comparisons. The ongoing costs of supporting the telephone services (excluding the cost of call charges and the costs of switchboard operators). The costs of moves and changes (but not the costs of any new devices, eg handsets). The cost of leased lines Management overheads should be included including an appropriate share of support service costs charged to the ICT service (costs of financial, personnel, legal and premises) for delivering this service SE309 Network management (voice only) SE310 Network support (voice only) The cost of calls per connection:   The average cost of call charges per connection should also be calculated and reported. The total cost of contracts and calls for mobile phones for the previous year should be calculated and divided by the number of mobile phones in use at the year end 31 March. The cost per mobile:

The support costs should be:    

The support costs should cover the following Socitm service elements:

Composition
a) b) c) Calculate the number of devices attached to the voice network, counting the number of ports actively used on each hub, and provide an overall total. Take all initial capital costs for any new voice network (annualised over a three-year period). Divide the total initial capital cost by the number of voice connections to give initial capital cost per connection.

10

d) e)

Take all annual support costs (excluding any costs of new devices). Divide annual support costs by total number of voice connections to give the support cost per connection.

Example
4,000 voice devices in total are supported at a cost of £700,000. A new network cost £600,000 12 months ago. Initial capital costs a) New network costs b) Procurement and installation costs c) No of years annualised d) Total capital costs e) No of voice connections f) Initial capital cost per connection (d/e) Support cost per connection g) h) i) j) Salaries, NI and superannuation Travel expenses ICT expenses Management overheads £600,000 £25,000 £35,000 £40,000 £700,000 4,000 £175 £230 £600,000 £60,000 3 £220,000 4,000 £55

k) Total support costs l) No of voice connections m) Support cost per connection (k/l) n) Total cost of voice connection (f+m)

KPI 5
       

Result

Enter any initial capital cost per connection. Enter support cost per connection Enter cost of calls per connection Enter number of mobiles supplied by the organisation (excluding teachers). Enter the cost per mobile supplied Indicate whether you have improved your overall performance on this KPI in the past twelve months (yes/no) Indicate whether this KPI has been validated by an external organisation (yes/no) Enter any qualifying comment

11

KPI 6
Objective

Cost per connection to data network

To measure the cost efficiency of providing the connection of a device to a corporate data network.

Reason for the KPI
This KPI complements KPI 5 in that it focuses on the data connection. This has been selected as one of the key elements of any ICT environment where costs should be controlled. Again, life-cycle costing is used as the best approach to managing costs of any ICT facility.

Definition
A device is any device that accesses data and is connected to the corporate data network (eg including PCs, printers, servers etc), taking up an active data network port. The network to which the devices are attached. They include Local Area Networks (LANs), Wide Area Networks (WANs) etc and may cover any technical standard (eg digital, analogue, token ring, X25 etc). The initial capital costs should be the costs of procurement and installation of any new investment in the last three years, including the current year (converted as an annual cost per data connection assuming that costs are recovered over three years). If costs are in fact covered over, say five or seven years, then this will need to be converted to a three year figure to ensure ‘like-for-like’ comparisons. The support costs should be the proportion of support costs (in-house plus contract costs) for all workstations plus the support costs for the network infrastructure. They should exclude the costs of any new workstations connected to the network but include the costs of moves and changes. They comprise in so far as they relate to support of networks: SE303 Security control SE306 Technology provision SE307 Equipment maintenance SE308 System software support SE309 Network management SE310 Network support SE312 Virus protection SE312 Asset management SE320 Service desk SE321 Problem management

Composition
a) b) c) d) e) f) g) Calculate the number of devices attached to the network by counting the number of data ports actively used on each hub, and provide an overall total. Take all initial capital costs for any new data network (annualised over a three year period). Divide the total initial capital costs by the number of device connections to give initial capital cost per connection. Calculate the indirect support costs of the network infrastructure. Calculate the direct support costs for all types of device. Add indirect and direct support costs to give total support costs. Divide total support costs by the total number of device connections to give support cost per connection.

12

The infrastructure costs include all equipment and software costs associated with the network, annualised where necessary.

Example
3,000 data devices in total are supported at a cost of £800,000. A new network cost £600,000 12 months ago. Initial capital costs a) b) c) d) e) f) New network costs Procurement and installation costs No of years annualised Total capital costs No of data connections Initial capital cost per connection (d/e) £600,000 £60,000 3 £220,000 3,000 £73

Support cost per connection g) h) i) j) k) l) m) n) o) p) Salaries, NI and superannuation Travel expenses ICT expenses Management overheads Total support costs (in-house) (g+h+i+j) Total contractor costs Total support costs (k+l) No of data connections Support cost per connection (m/n) Total cost of data connection (f+o) £380,000 £30,000 £40,000 £50,000 £500,000 £300,000 £800,000 3,000 £267 £340

KPI 6
     

Result

Enter any initial capital cost per data connection. Enter support cost per data connection Enter the number of individuals/ small sites (3 or less connections) supported. Indicate whether you have improved your overall performance on this KPI in the past twelve months (yes/no) Indicate whether this KPI has been validated by an external organisation (yes/no) Enter any qualifying comment

13

KPI 7
Objective

Support costs per workstation

To measure the cost efficiency of providing support for users of workstations.

Reason for the KPI
This KPI looks at cost from the point of view of costs per workstation in order to assess for example whether there might be a more efficient way of organising support.

Definition
The definition of a workstation is any device that accesses data. For the purposes of this KPI it is restricted to PCs, laptops, or ‘thin clients (or mainframe terminals). It excludes devices such as PDAs, palmtops, electronic organisers. Costs are not expected to be identified separately for these three categories. The costs are the annual costs of all support whether provided in-house or bought in via external service suppliers (ie the labour costs). They include the support costs of standard office products (eg Windows, MS-Office) but not the costs of the products themselves. They exclude the costs of client software to operate applications (eg Oracle Client) and the costs of supporting the network to which the device may be connected. They include the costs of supporting attached printers (but these printers are not counted as workstations). Support comprises the following Socitm service elements: SE208 SE209 SE210 SE320 SE303 SE306 SE307 SE308 SE312 SE321 SE322 Application maintenance (general office products only) Application software support (general office products only) Systems administration Service desk Security control Technology provision Equipment maintenance System software support Asset management Problem management Virus protection

Composition
a) b) c) Take the total annual employee costs for the relevant ICT specialists, their managers and clerical and other support staff. Take the annual contract costs for any support provided by third parties. Management overheads should be included, including an appropriate share of support service costs charged to the ICT service (costs of financial, personnel, legal and premises for delivering this service) Take the number of workstations supported by those resources. Divide the total costs by the number of workstations supported.

d) e)

Example
3,500 workstations cost £800,000 to support.

14

a) b) c) d) e) f) g) h) i)

Salaries, NI and superannuation Travel expenses ICT expenses Management overheads Total in-house costs (a+b+c+d) Total bought-in costs Total support costs (e+f) No of workstations supported Support cost per workstation (g/h)

£450,000 £35,000 £45,000 £70,000 £600,000 £200,000 £800,000 3,500 £229

KPI 7
   

Result

Enter the support cost per workstation. Indicate whether you have improved your overall performance on this KPI in the past twelve months (yes/no) Indicate whether this KPI has been validated by an external organisation (yes/no) Enter any qualifying comment

15

KPI 8
Objective

Workstations supported per support specialist

To measure the efficiency of the technical infrastructure and the productivity of support specialists.

Reason for the KPI
This KPI assesses the productivity of the most valuable resource for any ICT service, the ICT specialists. It is critical that those who support users do this in the most efficient manner. This is not just a question of organisation, or individual competence but also of the degree of investment in automated support tools (eg remote diagnostic tools).

Definition
The definition of a workstation is any device that accesses data. For the purposes of this KPI it is restricted to PCs, laptops, or ‘thin clients’ (or mainframe terminals). It excludes devices such as PDAs, palmtops, electronic organisers. Costs are not expected to be identified separately for these categories. The total number of support specialists is the total number of full time equivalent staff (FTEs) employed by the ICT unit who carry out these support activities. It also includes those who are brought in as contractors. Support comprises the following service elements: SE208 Application maintenance (general office products only) SE209 Application support (general office products only) SE210 Application administration SE302 Virus protection SE303 Security control SE306 Technology provision SE307 Equipment maintenance SE308 System software support SE312 Asset management SE320 Service desk SE321 Problem management Where any agreement for external support specifies numbers of support specialists, that number should be used. Where this is not specified, the number should be estimated by dividing the annual costs of the external support by the average salary of in-house ICT staff (including all overheads) and this should be added to the number of staff employed.

Composition
a) b) c) Add the number of FTEs supporting workstations employed by the ICT unit to any externally provided (actual or estimate) to give the total number of FTEs supporting workstations. Take the total number of workstations supported by those resources. Divide the total number of workstations supported by the number of FTEs.

Example
a) b) c) d) e) Number of in-house support specialists Number of external support specialists Total number of support specialists (a+b) Number of workstations supported Workstations supported per support specialist 14 2 16 3,500 219

16

KPI 8
   

Result

Enter the total number of workstations supported per support specialist Indicate whether you have improved your overall performance on this KPI in the past twelve months (yes/no) Indicate whether this KPI has been validated by an external organisation (yes/no) Enter any qualifying comment

17

KPI 9
Objective

Access to ICT

To measure the penetration of ICT throughout the organisation.

Reason for the KPI
Access to ICT indicates the degree of maturity and dependence on ICT. It is important to know the access to facilities that are provided for all employees, including elected members.

Definition
Employee access is the proportion of employees (including elected members) in the whole organisation who have reasonable access to a range of ICT facilities from the workstation at their desktop. Access does not mean ownership of a personal workstation. The definition of a workstation is any device that accesses data. For the purposes of this KPI it is restricted to PCs, laptops, or ‘thin clients’ (or mainframe terminals). It excluded devices such as PDAs, palmtops, electronic organisers. All workstations used just for training purposes, as spares or are redundant should be excluded. The total number of employees is the total number of full-time equivalent staff (FTEs) employed by the organisation. It should include all employees except manual workers and teachers (non-teaching employees at schools included if possible). There should be a separate record of access by elected members using ICT facilities provided for the purpose by the local authority and these might be provided in the home or at council offices (eg 10 workstations for an authority with 50 members counts as 20%). This calculation is very straightforward, but assumes the traditional style of working whereby each employee has a person desk and may or may not have personal workstation. The calculation is not quite as simple when we consider the impact of ‘hot-desking’ when the calculation has to consider the notion of seats rather than employees. A seat is a workstation that is occupied by one employee, or a number of employees working on a shift basis.

Composition
a) b) c) Take the total number of workstations in the organisation. Take the total number of FTEs (or seats), including elected members. Divide the number of workstations by the number of FTEs (or seats), expressing this as a percentage.

KPI 9
       

Result

Enter the percentage of office-based staff with personal workstations Enter the percentage of elected members with personal workstation Enter percentage of office-based staff with access to internal e-mail. Enter percentage of office-based staff with access to external e-mail. Enter percentage of office-based staff with access to the internet at their desktop. Enter the percentage of office based staff with intranet access. Enter the percentage of manual staff with personal workstations Enter percentage of manual staff with access to internal e-mail.

18

      

Enter percentage of manual staff with access to external e-mail. Enter percentage of manual staff with access to the internet at their desktop. Enter the percentage of manual staff with intranet access. Enter the number of PDAs in use in the council. Indicate whether you have improved your overall performance on this KPI in the past twelve months (yes/no) Indicate whether this KPI has been validated by an external organisation (yes/no) Enter any qualifying comment

19

KPI 10
Objective

ICT competence of employees

To measure the competence of employees who use ICT facilities.

Reason for the KPI
Dissatisfaction with training is recognised to be a major issue as far as service users are concerned. The implementation of e-government depends on a well-trained workforce confident and competent in using ICT. In the longer term qualifications such as ECDL might be a standard qualification for job advertisements.

Definition
The objective and subjective assessment of competence achieved by employees using ICT facilities.

Composition
a) Survey a sample of employees with the question: Do you think you have been adequately trained to make the best use of ICT in the workplace? (scale of 1 to 7 where 1 is inadequate and 7 is adequate.) b) c) Take the average score for the number of employees responding. Take the total number of office-based employees (including elected members but excluding manual workers and teachers) who have gained the ECDL qualification for at least one part.

KPI 10 Result
      Enter the employee perception about the adequacy of training to make the best use of ICT in the workplace. Enter number of employees who responded. Enter the total number of employees who have obtained ECDL qualification (any part). Indicate whether you have improved your overall performance on this KPI in the past twelve months (yes/no) Indicate whether this KPI has been validated by an external organisation (yes/no) Enter any qualifying comment

20

Draft KPI Service availability
Objective
To measure the availability of the ICT service to users during core working hours.

Reason for the KPI
It is assumed that the service is available 100% of core times. This KPI is attempting to measure the extent to which this standard is not met.

Definition
Level 1 – the whole service is unavailable affecting all users on the corporate network. ‘Unavailable’ means that users cannot use the network, however, they may be able to work on their PCs and store to a local drive until the network is available. ‘Unavailable’ includes the system being shut down, for example, because of a virus or other security measure. Level 2 – major applications are unavailable for all users. ‘Unavailable’ means that all users of a specific application cannot use that application(s) but are still able to use their computer for other things, eg the e-mail system is not working but word processing is still available. Level 1 unavailability should not be included when measuring Level 2 – do not double count. The specific applications to be measured are:

     

E-mail (internal) Internet access Finance system Revenues and benefits system Housing system Personnel/payroll systems

If multiple applications are used, eg two e-mail systems, count only the one with the majority of users.

Composition
Define the organisation’s core working hours for the whole ICT service and for each application. This will usually be found in existing service level agreements. Core times mean the core times supported by the ICT service as per service level agreements. Measure the time (hours and minutes) the service and the applications was unavailable throughout the previous year (April 1 to March 31) during core working hours. It is assumed that all planned downtime takes place outside core hours. Identify the number of instances for Level 1 and for each Level 2 application when the service/ application was unavailable to all users.

KPI

Result
Level 1.

 

Enter the total time (in hours) that the system was unavailable to all users Enter the number of instances when the service was unavailable

21

Level 2.  Enter the total time and number of instances that the following applications were unavailable to all users Hours E-mail (internal) Internet access Finance system Revenues and benefits system Housing system Personnel/payroll system    Indicate whether you have improved your overall performance on this KPI in the past twelve months (yes/no) Indicate whether this KPI has been validated by an external organisation (yes/no) Enter any qualifying comment Instances

22

KPI 14
Objective

Investment in ICT by employee

This KPI seeks to answer the question of what the organisation invests in ICT for its employees.

To measure the total level of investment in ICT

Reason for the KPI
Preparations for e-government have indicated that funding is critical and is usually seen as a major barrier to its successful implementation. It is crucial that all public sector organisations have some hard information about levels of investment produced in a consistent way that can be compared.

Definition
The number of employees is the number of full-time equivalents (FTEs), excluding all manual workers and teachers at the start of the financial year to which the costs apply. It should also include all elected members. Non-teaching employees at schools should be included if the ICT costs relating to them can be included, but not if the ICT costs cannot be included. Total investment includes all budgeted ICT expenditure for the current financial year, ie staffing, equipment and software, made by the whole organisation, not just the central ICT unit. It should also include an appropriate share of support services (eg personnel, payroll, legal and cost of premises). It excludes initial capital expenditure but includes the annual capital recharge associated with it. It also includes funding that has been made as a result of some competitive process (eg Invest To Save Bid, Pathfinders, IEG etc). Partnership funding should be allocated on whatever basis the partnership has agreed to share out its funds (ie not necessarily an equal share). Although hidden costs (eg informal support and training by users) are a significant element of the true costs, it is not practical to collect this information and such costs are excluded.

Composition
a)       b) c) d) e) Add together all items of revenue expenditure Staffing Equipment Software Communications (excluding call charges) Bought-in services (eg outsourcing, consultancy, subscriptions) Recharge for overheads from outside ICT unit Add together all items of external income (but not housing revenue). Take the total full-time employees (including elected members but excluding teachers and manual workers). Divide the total revenue expenditure by the number of employees to give total revenue investment in ICT per employee. Divide the total external funding by the number of employees to give the total investment from external funds in ICT per employee.

Result
   Enter the total revenue investment in ICT per employee. Enter the total investment from external funds received by the council for ICT per employee Indicate whether you have improved your overall performance on this KPI in the past twelve months (yes/no)

23

 

Indicate whether this KPI has been validated by an external organisation (yes/no) Enter any qualifying comment

24

KPIs 51-75
Objective

Electronic service delivery for the most frequently used transactions (10 only)

To measure the implementation and success of e-government.

Reason for the KPI
It is absolutely critical that we have a measure of the successful implementation of full electronic service delivery that recognises not just availability of electronic service delivery but take-up and satisfaction. It also has to be practical as a data collection method by focusing on the most frequently used transactions rather than attempting to cover every single possibility.

Definition
The top 10 frequently used transactions have been defined as a set. Self-service relates to contact by electronic means such as a website sanctioned by the local authority, or to other remote access facilities such as kiosks in public places, automated telephone transactions, digital TV or text messaging that results in transactions. Service by telephone relates to a mediated service where an intermediary at the end of a telephone line receives requests for information or service calls and processes them directly by making use of electronic transactions without having to pass the request to a third party. Such a facility may be called a contact centre, a call centre, or even a telephone switchboard, but the key characteristic is that the call is not passed on to another for resolution as a result of having immediate electronic access to the right system. Service face-to-face relates to a mediated service where an intermediary in an office, or shop or other service point receives requests for information or service calls from a member of the public visiting in person (or possibly by video-conferencing) and can satisfy the request immediately as a result of having immediate electronic access to the right system without having to pass the request on to a third party.

Composition
The method of data collection will vary according to the three main channels identified, and any variation of those channels. Only general guidance can be given here as follows: a) b) c) Enter under availability Yes/No and, in the case of opening hours, the number of hours open each typical week, ignoring holidays etc. Enter under service standard the existence of a published statement setting out the performance standards for the service using this method.. Enter under take-up the percentage of transactions completed on average each week in the past month using this method. Note that the percentages for the three electronic methods plus the percentage for non-electronic methods should add up to 100%. Enter the month as (02/02 for February 2002) when the take-up calculation was made. Enter under satisfaction the average level of satisfaction with the service from a sample of those who used the service (most likely part of a broader satisfaction survey). The scale used should be 1 to 7 where 1 is poor and 7 is excellent (no intermediate descriptors advised).

d) e)

Context
This set of new KPIs has grown out of the perceived weaknesses of the statutory indicator for e-government (BVPI 157 in England and NAWPI 1.5 in Wales). Whilst a useful target, it has been criticised for being overly complex and not being practical enough for at least four reasons:   It focuses only on availability, ignoring standards, take-up and satisfaction It covers all possible transactions, rather than most commonly used transactions

25

 

It counts availability for just one electronic channel, ignoring the possibility that others might be better used It offers no comparison with non-electronic methods

This new set aims to address these shortcomings and to offer a framework for performance measurement that will outlast the 2005 targets for the statutory indicators. It is fully recognised that it will take time to introduce measurement systems that cover the full scope of the new set of KPIs, but it maps out a future development path that will eventually be a very effective management tool. There are several different ways in which local e-government might be implemented. Measuring the success of the many options (eg which channels and the impacts (eg what take-up) is a vital component of the implementation within the organisation. Sharing the lessons learned between organisations will also help to make the best use of scarce resources. This is a set of KPIs in a new area of performance measurement for the public sector. It will take time to develop fully. While every care has been taken to produce a definition that is clear, robust and practical, there may well be some early difficulties of understanding and definition which Socitm is committed to deal with in the next version. Socitm will also maintain a section on its website for frequently asked questions relating to this set of KPIs. At this time there is no authoritative list of most commonly used transactions to apply for this KPI. Such a concept is in any event difficult to provide for a number of reasons:    The definition of the transaction will be open to interpretation The list of common transactions will vary for counties, districts and unitaries Each local authority is likely to have some local variation of any top ten

The solution to these problems is to produce a long list of 25 suitable transactions, based on the best information available, and to propose that each authority selects its top 10 from this list. The advantages are that not only are the listed flaws dealt with, but that over time we will build up a real picture of what the top 10 are nationally (also what percentage of the total 10% they represent). Such a list has been compiled from local authorities around the country and from that list has been selected a relevant question that requires an interactive response by an electronic method (ie a transaction). There may well be some early problems of definition for such transactions. Currently a list of definitive transactions is being drawn up as part of the Life Events Access Project (LEAP) focused initially on the needs of metropolitan districts and London boroughs. This list is the Customer Unique Process Identifier (CUPID) classification scheme developed in order to produce a list of customer-facing local authorities’ processes.

Electronic signatures
In some cases a signature is required to complete the transaction. At the current time signatures are not accepted electronically. Transactions can be counted as electronic if all the steps up to the signature are electronic but not the signature itself.

26

KPIs 51-75

Electronic service delivery for the 10 most frequently used transactions

Can I make the following transactions by an electronic method? Unitary / shire county KPI 51 KPI 52 KPI 53 KPI 54 KPI 55 KPI 56 KPI 57 KPI 58 KPI 59 KPI 60 KPI 61 KPI 62 KPI 63 KPI 64 KPI 65 KPI 66 KPI 67 KPI 68 KPI 69 General KPI 70 KPI 71 KPI 72 KPI 73 Other KPIs 73-75 Each local authority can add another three transactions from its own database of information that are commonly used. find out the name of my local councillor? report a change of address once only? obtain a list of services and contact points? make an application for a job? renew library books? find out the names and addresses of local schools? find out when my education grant will be paid? enrol on an adult education course? report a street light failure? apply for any trading standards licence? arrange to see a social worker? find out about a county transport service (eg social services, schools)? find out housing benefit entitlement? report a housing repair? enquire about the status of my planning application? make a building control application? make a complaint about some aspect of environmental health? report a missed refuse collection? find tourist information about the area covered by the local authority? book a sporting/leisure event managed by the local authority? pay a parking fine? pay Council Tax? pay business rates?

Unitary / shire district

KPIs 51-75

Results

IMPORTANT – for this year only 10 KPIs will be used: KPIs 51, 54, 55, 60, 61, 64, 66, 67, 68 and 73 (new).

27

Note
The revised external KPIs for benchmarking e-government progress on high profile transactions have been linked, as shown below, to the services that are included in the ESD Toolkit. This is administered on the IDeA website and further details about the Toolkit can be found by registering at www.esdtoolkit.gov.uk
Socitm KPI KPI 51 Transaction description Service level 1 Renew library books Service level 2 ARTS AND LEISURE PID No and Name Libraries 440 - Loans renewing/extending

KPI 54

Enrol on an adult education course

EDUCATION

Adult education

27 - Courses

KPI 55

Report a street light failure

ENVIRONMENTAL SERVICES

Roads, highways and pavements

564 - Lighting - street lights

KPI 60

Report a housing repair

HOUSING

Improvements, repairs and regeneration Planning, building control and town centre management Refuse

141 - Housing repairs council property

KPI 61

Enquire about the status of my planning application Report a missed refuse collection

ENVIRONMENTAL SERVICES

485/487 - Planning residential/ business

KPI 64

ENVIRONMENTAL SERVICES

524 - Household waste - collection

KPI 66

Book a sporting/leisure event managed by the local authority Pay a parking fine

ARTS AND LEISURE

Physical activity and sports

451 - Sports facilities

KPI 67

ENVIRONMENTAL SERVICES

Parking

471 - On street parking enforcement - parking fines 58 - Individual account enquiries (incl. new accounts) 337 - Vacancies

KPI 68

Pay Council Tax

REVENUES AND BENEFITS/FINANCE

Council tax

KPI 73

Make an application for a job

MISCELLANEOUS

Recruitment

28

KPI 51

Renew library books

(unitary/county)

Self-service (eg website) a) b) c) d) e) Is transaction available? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service by telephone f) g) h) i) j) Number of hours service available per week? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service face-to-face kl) l) m) n) o) p) Number of hours service available per week? Is there a service standard for this method? No of locations where transaction is available? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Other methods of service q) r) Estimate % take-up of transactions using all other (non-electronic) methods. Enter any qualifying comment

29

KPI 54

Enrol on an adult education course

(unitary/county)

Self-service (eg website) a) b) c) d) e) Is transaction available? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service by telephone f) g) h) i) j) Number of hours service available per week? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service face-to-face k) l) m) n) o) p) Number of hours service available per week? Is there a service standard for this method? No of locations where transaction is available? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Other methods of service q) r) Estimate % take-up of transactions using all other (non-electronic) methods. Enter any qualifying comment

30

KPI 55

Report a street light failure

(unitary/county)

Self-service (eg website) a) b) c) d) e) Is transaction available? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service by telephone f) g) h) i) j) Number of hours service available per week? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service face-to-face kl) l) m) n) o) p) Number of hours service available per week? Is there a service standard for this method? No of locations where transaction is available? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Other methods of service q) r) Estimate % take-up of transactions using all other (non-electronic) methods. Enter any qualifying comment

31

KPI 60

Report a housing repair

(unitary/district)

Self-service (eg website) a) b) c) d) e) Is transaction available? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service by telephone f) g) h) i) j) Number of hours service available per week? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service face-to-face kl) l) m) n) o) p) Number of hours service available per week? Is there a service standard for this method? No of locations where transaction is available? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Other methods of service q) r) Estimate % take-up of transactions using all other (non-electronic) methods. Enter any qualifying comment

32

KPI 61

Enquire about status of planning application (unitary/district)

Self-service (eg website) a) b) c) d) e) Is transaction available? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service by telephone f) g) h) i) j) Number of hours service available per week? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service face-to-face kl) l) m) n) o) p) Number of hours service available per week? Is there a service standard for this method? No of locations where transaction is available? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Other methods of service q) r) Estimate % take-up of transactions using all other (non-electronic) methods. Enter any qualifying comment

33

KPI 64

Report a missed refuse collection

(unitary/district)

Self-service (eg website) a) b) c) d) e) Is transaction available? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service by telephone f) g) h) i) j) Number of hours service available per week? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service face-to-face kl) l) m) n) o) p) Number of hours service available per week? Is there a service standard for this method? No of locations where transaction is available? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Other methods of service q) r) Estimate % take-up of transactions using all other (non-electronic) methods. Enter any qualifying comment

34

KPI 66

Book a sporting/leisure event managed by the local authority (unitary/district)

Self-service (eg website) a) b) c) d) e) Is transaction available? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service by telephone f) g) h) i) j) Number of hours service available per week? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service face-to-face k) l) m) n) o) p) Number of hours service available per week? Is there a service standard for this method? No of locations where transaction is available? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Other methods of service q) r) Estimate % take-up of transactions using all other (non-electronic) methods. Enter any qualifying comment

35

KPI 67

Pay a parking fee

(unitary/district)

Self-service (eg website) a) b) c) d) e) Is transaction available? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service by telephone f) g) h) i) j) Number of hours service available per week? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service face-to-face kl) l) m) n) o) p) Number of hours service available per week? Is there a service standard for this method? No of locations where transaction is available? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Other methods of service q) r) Estimate % take-up of transactions using all other (non-electronic) methods. Enter any qualifying comment

36

KPI 68

Pay Council Tax

(unitary/district)

Self-service (eg website) a) b) c) d) e) Is transaction available? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service by telephone f) g) h) i) j) Number of hours service available per week? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service face-to-face k) l) m) n) o) p) Number of hours service available per week? Is there a service standard for this method? No of locations where transaction is available? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Other methods of service q) r) Estimate % take-up of transactions using all other (non-electronic) methods. Enter any qualifying comment

37

KPI 73

Make an application for a job (new)

(all)

Self-service (eg website) a) b) c) d) e) Is transaction available? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service by telephone f) g) h) i) j) Number of hours service available per week? Is there a service standard for this method? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Service face-to-face k) l) m) n) o) p) Number of hours service available per week? Is there a service standard for this method? No of locations where transaction is available? % take-up of transactions using this method? Month when take-up was measured? Assessment of satisfaction with service by public (scale of 1 to 7, to 2 decimal places)

Other methods of service q) r) Estimate % take-up of transactions using all other (non-electronic) methods. Enter any qualifying comment

38

KPI 90
Objective

Investment in ICT by head of population

To measure the total level of investment in ICT.

Reason for the KPI
Preparations for e-government have indicated that funding is critical and usually seen as a major barrier to its successful implementation. It is crucial that all public sector organisations have some hard information about levels of investment produced in a consistent way that can be compared.

Definition
Total investment includes all budgeted ICT revenue expenditure for the current financial year, ie staffing, equipment and software, made by the whole organisation, not just the central ICT unit. It should also include an appropriate share of support services (eg personnel, payroll, legal and cost of premises). It excludes initial capital expenditure but includes the annual capital recharge associated with it. It also includes external funding and funding that has been made as a result of some competitive process (eg Invest to Save Bid, Pathfinders, etc). Partnership funding should be allocated on whatever basis the partnership has agreed to share out its funds (ie not necessarily an equal share). Although hidden costs (eg informal support and training by users) are a significant element of the true costs, it is not practical to collect this information and such costs are excluded. Head of population should be based on the official population figures.

Composition
a)       b) c) d) e) Add together all items of revenue expenditure: Staffing Equipment Software Communications (excluding call charges) Bought-in services (eg outsourcing, consultancy, subscriptions) Recharge for overheads from outside ICT unit Add together all items of external income (but not housing revenue) Take the population size for the authority from the official population figures. Divide the total revenue expenditure by the population to give total investment in ICT per head of population. Divide the total external income by the population to give the investment from external funds per head of population.

KPI 90 Result
    Enter the total investment (revenue) in ICT per head of population. Enter the total investment from external funds received by the council for ICT per head of population. Indicate whether this KPI has been validated by an external organisation (yes/no) Enter any qualifying comment

39

KPI 91
Objective

Access to ICT by the community

To measure access to ICT by the community.

Reason for the KPI
Access to ICT indicates the degree of maturity and dependence on ICT. It is fast becoming important to extend access within the community in order to support the implementation of e-government.

Definition
Community access refers to the proportion of the local population that has access to the internet as calculated from a survey (most likely as part of a broader survey of local needs). If workstations have been part-funded by the local authority (eg with NHS, or as part of school homeworking clubs, or with another local authority in a two-tier area), they should be included. If, however, they have not been funded at all by the local authority, but are still publicly available (eg from another government agency), they should be excluded.

Composition
a) b) Carry out a survey of the local community covered by the local authority area (likely to be part of a broader survey). Ask the questions: Have you in the past three months personally used the Internet … … from your home? … from your place of work? … from your school or college? … from another public place? (eg library, community centre, etc) … from anywhere? c) d) Calculate the percentages from the sample surveyed. Record the number of workstations provided by the local authority for community use.

KPI 91 Result
         Enter % of population who have used the internet at home. Enter % of population who have used the internet at work. Enter % of population who have used the internet at school/college. Enter % of population who have used the internet in another public place. Enter % of the population who have used the internet anywhere Enter the number of workstations provided by the local authority for community use. Indicate whether you have improved your overall performance on this KPI in the past twelve months (yes/no) Indicate whether this KPI has been validated by an external organisation (yes/no) Enter any qualifying comment

40


				
DOCUMENT INFO
Shared By:
Tags: KPI-1, -User
Stats:
views:435
posted:11/29/2009
language:English
pages:40
Description: KPI-1-User-satisfaction