Document Sample
bt Powered By Docstoc
					       Proposed requirement on dominant providers to publish Key Performance Indicators

                                        BT Response

    BT’s response to Oftel’s consultation document
   “Proposed requirement on dominant providers to
        publish Key Performance Indicators”
                                19 September 2003

This paper is BT’s formal response to Oftel’s consultation issued on 11 July 2003.

BT would welcome any comments on its position as laid out in this document which
will also be available electronically at

Comments should be addressed to Wendy Dodd, BT Wholesale Regulation, PP502.3,
Mondial House 90-94 Upper Thames Street, London EC4R 3UB, or by email to

BT Group plc                           Septemb er 2003                              Page 1 of 10
       Proposed requirement on dominant providers to publish Key Performance Indicators

                                        BT Response

Further to the Consultation Document “Proposed requirement on dominant providers
to publish Key Performance Indicators” published by Oftel on 11 July 2003, BT’s
comments are set out below.


BT supports the Director’s stated objectives in ensuring transparency and non-
discrimination in the provision of wholesale services where appropriate, namely that
any resulting obligations must:-

•   be objectively justifiable;
•   not discriminate unduly against particular persons;
•   be proportionate to what they are intended to achieve; and
•   provide transparency in what they are intended to achieve.

All of these considerations are critical if a reasonable set of obligations on dominant
providers is to be put in place to ensure that undue discrimination is not occurring.
Any outcomes must ensure that an unduly onerous burden in terms of KPI reporting,
whether in relation to volume of information or frequency of publication, must not be
placed upon dominant providers. BT agrees that a sensible balance must be found
between the need to ensure transparency and the cost/resource implications of any
regulatory obligations. For example, KPI obligations would not be proportionate
where product volumes are low. In addition, we feel that some of the KPIs proposed
in the consultation are not appropriate for all the services covered. We discuss this in
more detail below.


In addition to the stated objectives, it will be important for any definitions of KPIs to
be precisely agreed. It would be unreasonable, for example, if BT were required to
report against any service which included performance parameters not totally within
our control unless the definition were unambiguously qualified to take account of this

As Oftel agreed during previous discussions on Broadband KPIs, certain metrics are
only partially in BT’s control. For example, average time to restore service is not
completely within BT's control as a fault can spend time waiting for SP activity
(parked time) which is included in the count. Similarly, average provisioning time can
be affected by the fact that some SPs work to a minimum lead-time which is reflected
in the submitted customer required date. We believe that similar issues are likely to be
encountered during this review which will have to be approached in the same way.

In view of the above, BT would welcome the opportunity to discuss detailed
definitions with Oftel during the coming months.

In this context, and given that the purpose of the consultation is to ensure that BT
Wholesale does not discriminate in terms of the service that it provides both to BT’s

BT Group plc                           Septemb er 2003                              Page 2 of 10
       Proposed requirement on dominant providers to publish Key Performance Indicators

                                        BT Response

own downstream operations and other providers, any KPIs should, in principle,

(i)    the service provided by BT Wholesale (BTW) to other providers; and
(ii)   the service provided by BTW to BT Retail (BTR) or BT Global Services
       (BTGS) as appropriate.

Such a principle is, of course, subject to the ability to measure such comparisons in
practice. Whilst this may not be practicable in the short term, it should represent the
longer term aim and will involve ensuring that measures for BTW delivery to BTR
are available for whatever is agreed in relation to other BTW customers. It should be
noted that BT Wholesale would be the supplier involved here and we have taken
Oftel’s references to “BT” in the consultation document to equate to BTW.

In some instances there will not be a direct comparison because BTR does not
purchase interconnect products such as PPCs or interconnect links. Further, in some
cases (WLR, for example) the most obvious and readily available comparison may be

(i)    the service provided by BTW to end users of other providers; and
(ii)   the service provided to its own end users by BTR where appropriate.

Therefore, reasonable equivalence is an important concept if there is to be any
meaningful comparison between the service provided by BTW to BTR and other
providers in instances where no direct service equivalence exists.

BT Group plc                           Septemb er 2003                              Page 3 of 10
           Proposed requirement on dominant providers to publish Key Performance Indicators

                                                      BT Response


1. Specific KPIs

Q.1.1 Do you agree with the list of proposed KPIs, and their rankings, as outlined
in table 1? Alternative options will be considered. Ideally the proposals should be
substantiated and ranked in order of preference.

BT’s view on the applicability and rankings of proposed KPIs is in Table 1 below:

Table 1

Proposed                 ATM Note 1              I/X Links              FRIACO                  WLR
pre-ordering             3 – n/a to existing     3 – n/a to existing    3 – n/a to existing     WA - 3 (n/a to
response time            process                 process                process                 existing process)
                                                                                                ISDN - 3
ordering response        3                       3                      3                       WA - 3
time                                                                                            ISDN – 3
% orders rejected        1 – this measure        3 - n/a to existing    3 - n/a to existing     WA - 3
                         would most clearly      process                process                 ISDN – 3
                         show non
                         between SPs
% orders                 1 – this measure        1 – this measure       1– this measure         WA - 1
provisioned on time      would most clearly      would most clearly     would most clearly      ISDN – 1
                         show non                show non               show non
                         discrimination          discrimination for     discrimination for
                         between SPs (NB –       both new routes &      both new routes &
                         not currently           route augmentation     route augmentation
average                  3                       3                      3                       WA - 3
provisioning time                                                                               ISDN – 3
% new provisions         3 – not relevant as     3 – not relevant as    3 – not relevant as     WA - 3
reported as faulty       each cct tested         each cct tested with   each cct tested with    ISDN – 3
                         before provisioning     customer before        customer before
                         completed               provisioning           provisioning
                                                 completed              completed
% installed base         3                       3                      3                       WA - 2
reported as faulty                                                                              ISDN – 2
average time to          1 – existing measure    1 – existing measure   1 – existing measure    WA - 1
restore service Note 2   is faults cleared <40   would fit purpose      would fit purpose       ISDN – 1
                         hours (and should       i.e. % service         i.e. % service
                         exclude parked          restoration in 8 &     restoration in 8 &
                         time)                   24 hours               24 hours
% repeat faults          2                       3                      3                       WA - 2
                                                                                                ISDN - 2

Note 1 – As confirmed by Oftel at the meeting of 12 September, the proposed KPIs relate to end user connections
for the DataStream family of products rather than the provision/repair of ATM Access Ports or In-Span handover
links. Oftel is aware that BTR & BTOW do not use the DataStream product, so non-discrimination is demonstrable
between non-BT SP’s only.

Note 2 - NB – alternative measure for WLR should be % faults cleared on time.

General Note – As stated earlier in this response, many of the above metrics are only partially in BT’s gift to
deliver because of Service Provider interaction.

BT Group plc                                         Septemb er 2003                                   Page 4 of 10
       Proposed requirement on dominant providers to publish Key Performance Indicators

                                        BT Response

Q. 1.2 which option do you prefer?

BT would favour the priority 1 options as detailed in Table 1 above which, we
believe, would demonstrate transparently that no undue discrimination was taking
place. They represent a sensible balance in that they would fit the required purpose of
the consultation but would not involve substantial re-engineering of existing processes
and systems, thereby avoiding excessive costs.

We appreciate that, in drawing up the proposed list, Oftel is trying to capture key
business processes on a theoretical end-to-end basis, the inevitable result of this
approach being that some measures are clearly not appropriate to the chosen services.
On this basis, pre-ordering response time should be removed from the list as it is not

BT considers the most important measures to be timeliness of order provisioning and
fault repair. Not only do both measures impact most directly on the customer
experience, but they also rely upon BT having to take positive action rather than the
main driver being an automated process.

In this respect we believe that measurement of average times is an over-complicated
and potentially costly option in that average time would need to be measured for each
transaction type where a number of different transactions take place for a specific
product. Any differences in average times on an individual basis may also have a
reasonable explanation e.g. if one SP receives a below average provisioning time, it
could be because it does not generally require delivery as quickly as other SPs.
Average time is also influenced by different populations of products (e.g. standard
care, prompt care) and this, combined with the factor of different requested delivery
times as noted above, can skew any measurements. It can also be unduly influenced
by the removal of long waiters or faults, especially for products with lower volumes.

In general, in view of the fact that “% of orders provisioned on time” is already
proposed as a measure, we would question whether it is proportionate to require BT to
report on two measurements of the same type.

We would appreciate clarification on the rankings assigned to each KPI. Anything
with a generally agreed marking of 1 would presumably be a “must-have”, but it is
not clear whether those KPIs with a ranking of 3 would automatically be discarded or
would still be considered in some context. BT would prefer to limit its obligations to
those KPIs (which may differ by product) which have an agreed ranking of 1.

2. Specific Wholesale Services

Q.2.1 Do you agree that LLU, CPS, IA and NTS should not be included within the
scope of the proposed Directions?

BT supports Oftel’s proposals not to include these products in the scope of the
Directions. The focus of the proposed KPIs is to demonstrate that no undue

BT Group plc                           Septemb er 2003                              Page 5 of 10
       Proposed requirement on dominant providers to publish Key Performance Indicators

                                        BT Response

discrimination is taking place. In BT’s view, this should mean that obligations should
not apply where there is no designation of SMP on BT along with a specified remedy
(e.g. IPStream) or where there has never been any evidence of systematic
discrimination (voice service, for example, which has been measured for many years).

As Oftel notes in paragraph 3.11, BT already publishes performance data in relation
to CPS and is continuing to do so. As this meets industry needs at present, it would
seem disproportionate to increase requirements in this area.

With regard to IA and NTS, in addition to the absence of an ordering process for these
services (the interconnect links process covers this aspect), it would be difficult to
separate the services for measurement purposes as they are provided over the same
links. Consequently, the inclusion of interconnect links is more appropriate although
it should be noted that BT does not supply this service to itself internally. Therefore
this can only serve as a measure of non-discrimination in treatment of other providers.

BT agrees with Oftel’s stated principle that the requirements should only apply to
products which generate enough volume to be statistically meaningful. By this
yardstick it is appropriate to exclude LLU and, by extension, any product with very
low volumes (new services should not be reported on, for example).

Q.2.3 Do you agree that FRIACO, WLR etc should be included?

BT has no objections in principle to the list of proposed services. However, we do
have some reservations over the potential applicability of the range of KPIs to certain
of the products specified. Please see our response to Question 1 for more detailed

Where volumes are already low and declining, a threshold should be agreed, whether
in terms of volumes or number of customers, where the requirement to report should
be discontinued. This would apply in cases where the resource and cost involved in
providing the report was disproportionate to any benefits gained from such provision.

As noted above, BT already provides performance statistics on a voluntary basis for
CPS. It may be that such an arrangement would be more appropriate to WLR than the
regulated proposals in the consultation document. BT would be happy to consider
discussing this possibility with Oftel and industry.

3. Process of publication

Q.3.1 Do you agree with the proposed options for the process of publication as
outlined in table 2?

BT agrees that the options specified are all potentially deliverable. However, given
that the objective here is to demonstrate that no undue discrimination is taking place
between the services BT provides to itself and to other providers, BT believes that
Option 2 represents the best balance between transparency and commercial
confidentiality. We believe that providing the figures relating to BTR or BTGS to
Oftel should be sufficient to meet the need for transparency. We consider that if these

BT Group plc                           Septemb er 2003                              Page 6 of 10
       Proposed requirement on dominant providers to publish Key Performance Indicators

                                        BT Response

figures were required to be published separately to industry at large, this could raise
issues with regard to the commercial confidentiality of these BT units but could also
be unduly discriminatory. In our view, BT’s downstream operations should be treated
no differently from any other operator in terms of public disclosure of information i.e.
if a provider wants to see the statistics for BT’s downstream operations then it must
be prepared to have its own individual information published too.

For WLR, BT believes that it would be sufficient to measure all SPs at an aggregate
level. The costs of disaggregating are likely to be prohibitive and WA is a new
product, so no meaningful volume or market share information will be available on
which to base a disaggregated report. Option 2 therefore represents a proportionate

We would be happy to provide such information to Oftel on a confidential basis and
would, of course, publish the aggregated figures in the public domain in the interests
of transparency.

Q.3.2 Are there other options we should consider?

As mentioned under “General” above, the most readily available measurement for
WA and ISDN may be between the performance that BTW provides to SPs’ end users
and that which BTR provides to its own end users. While this does not represent a
different option in itself, this point would need to be clarified in the context of Option
2 as would the appropriate BT Retail comparisons for the other products considered.

Q.3.3. What do you perceive would be the costs and benefits of your preferred

As stated in the response to Question 3.1, we believe that Option 2 represents the best
balance between transparency, commercial confidentiality and proportionality. See
also Question 8 with respect to cost information.

4. Frequency of publication

Q.4.1 How frequently should the KPIs be published to ensure undue discrimination
is detected and responded to in a timely manner?

Where other reporting arrangements have not already been agreed and/or
implemented, or where low volumes exist, BT believes that annual publication would
be proportionate, particularly where there has previously been no indication of unduly
discriminatory behaviour by BT. It may be possible to incorporate a flexible
approach so that the annual report is not set in stone. For example, if undue
discrimination were found to be a problem in a certain product area, the Director
could direct that the frequency of reporting should be increased to, say, quarterly until
such time as the situation is remedied to his satisfaction. Other providers always have
the right to approach Oftel if they feel that they have experienced discriminatory
treatment and such instances can be looked at on an individual basis outside of the
reporting cycle.

BT Group plc                           Septemb er 2003                              Page 7 of 10
       Proposed requirement on dominant providers to publish Key Performance Indicators

                                        BT Response

We do not believe that the option of monthly reporting is sensible as it has the
potential to distort the reported measurements. This is a particular issue with low
volume products where a one-off problem can cast a disproportionately large shadow
over the generality of reported figures. This, however, would have a lesser impact
upon annual or even quarterly figures, which would allow a more reasonable
assessment of whether any systematic discrimination was taking place.

In the case of WLR, the fit for purpose requirements for WA will continue until such
time as the assessment is successfully completed. At that time, it would seem
appropriate for the frequency of publication to align with that of the equivalent Retail
measures (6-monthly for CPI indicators). For ISDN, we would expect the frequency
of publication requirement to be no more onerous than that applied to WA.

Q.4.2 Is there a case for applying different frequency criteria to different

As stated above, we believe that it would be reasonable to apply different frequency
criteria to different products depending upon individual circumstances. We also
believe that differentiation may be appropriate between KPIs applicable to an
individual product. For example, if volumes of new provides were low but volumes
of repairs were significant for the same product, different frequency of reporting
might be proportionate.

5. Criteria for selection

Q. 5.1 Which of the options for the selection of operators to be included do you

In view of the varying nature of the customer bases (and associated volumes) for each
of the proposed products, we consider that a uniform approach would not be
appropriate and a product-specific view needs to be taken:-

       FRIACO:        Given that the number of customers is small it would make
       sense to include all of them in the report, so no threshold is needed. However,
       see “Confidentiality” below.

       WLR:            BT’s preference would be to measure all SPs at an aggregate
       level given that the costs of disaggregation are likely to be prohibitive. WA is
       a new product, so no meaningful volume or market share information will be
       available at an individual level until such time as the market has developed
       and stabilised.

       I/X Links:      Our view is that operators for inclusion should be selected on
       the basis of volumes of circuits in service. Taking this approach, a threshold
       of 1000 circuits means that 16 operators would be measured, which represents
       some 10% of the customer base. This would capture the larger players and
       would, in our view, constitute a sensible and meaningful sample.

BT Group plc                           Septemb er 2003                              Page 8 of 10
       Proposed requirement on dominant providers to publish Key Performance Indicators

                                        BT Response

       ATM:           As Oftel is aware, this product is at a very early stage of
       maturity with a small number of providers taking the product in comparison to
       IPStream and a very small overall end user volume in comparison to
       IPStream. Again, as Oftel is aware via statistics already provided, a single
       non-BT SP generates the vast proportion of volumes. In addition, neither BTR
       nor BTOW utilises this product which means that there is no ‘retail’

       Our view is that it would be proportionate to include the top 10 SPs in terms of
       volumes. It should be borne in mind, however, that those SPs at the lower end
       of this group might not generate volumes which are statistically meaningful
       given that a single operator has more than 90% market share.

Q.5.2 What should be the threshold for your preferred option?

See 5.1 above.

6. Call accounting accuracy

Q.6.1 Are you aware of a KPI that would provide useful additional information on
call accounting accuracy?

BT is not aware of a useful call accounting accuracy KPI. We consider that call
accounting accuracy is a measurement which is best addressed via SLA arrangements
rather than a regulatory obligation to demonstrate non-discriminatory behaviour. It is
in BT’s best interests to bill all operators in a timely and accurate manner, thereby
avoiding time and resource-consuming disputes and attendant impact on revenue
flows. We do not believe that there is scope for discrimination between operators in
terms of billing accuracy in any event. Any KPIs would be extremely difficult to
define and disproportionate given the regulatory purpose behind this consultation.

7. Confidentiality

Q.7.1 Do you have any concerns regarding confidentiality with regard to any of the
proposed indicators or the proposed process for publication?

In general, BT does not have any serious concerns with confidentiality issues
particularly if information in many instances is provided at an aggregate level as with
the option 2 proposal and if anonymity of other providers is preserved where this
aggregation is not the case. There may be a potential issue around publishing
individual information where only a small number of providers are taking the service
in question. For example, with FRIACO (where only five external customers exist) or
ATM (where only a few customers exist and where one SP generates the vast majority
of volumes), publication of individual data, anonymity notwithstanding, could still be
indicative of market activity and could allow competitors to make educated guesses at
the identities of certain players and, by extension, their individual KPIs.

BT Group plc                           Septemb er 2003                              Page 9 of 10
       Proposed requirement on dominant providers to publish Key Performance Indicators

                                        BT Response

8. Costs of implementation

Q.8.1 Do you believe there will be disproportionate costs involved in the collection
of the proposed KPIs? If so, please quantify where possible and indicate how you
would modify the proposals to reduce the costs of implementation and minimise the
number of changes needed to existing systems.

In view of the fact that Oftel would expect BT to absorb any costs associated with
implementation, we consider it essential that any solutions represent the minimum
required to demonstrate non-discriminatory behaviour on BT’s part.

Further to our comments in response to Question 1, there would be significant costs
involved in the development of systems were the KPIs to be implemented uniformly
across all specified products. For example, FRIACO development costs could be in
the region of £500k were the full range of KPIs to be required.. In view of this figure,
and given the declining volumes associated with the product, we consider that it
would be disproportionate to impose the full range of KPIs on FRIACO.

Simple high-level measures for provision and repair would be more appropriate in the
circumstances and, if reported annually as a sub-set of Interconnect links, we estimate
that these could be implemented at an approximate cost of £20k - £40k (see Question
1 for more detail on appropriate measures).

The potential costs of implementing the full set of proposed KPIs for WLR could be
in the order of £400k to report them systematically going forward. This figure
excludes any impact on associated retail measures which might have to be either
developed or modified.

Following the work undertaken with Oftel on the wider Wholesale Broadband KPIs,
we believe an activity to re-engineer the instrumentation to segment the KPIs for the
DataStream product only would be possible for the majority of the metrics that Oftel
has suggested. A detailed evaluation would need to be considered but we estimate that
to ‘industrialise’ the process for this exercise in its proposed totality could potentially
cost approximately £50K to implement with ongoing incremental costs.

NB – all of the above figures are very rough estimates. Full feasibility activity would
need to be undertaken to establish more accurate costs once Oftel’s proposals are

BT Group plc                           Septemb er 2003                            Page 10 of 10

Shared By: