Pension_Funds_Survey_Technical_Article by shimeiyan1




The ONS Pension Funds survey is directed at a sample of UK Pension Funds drawn from the
membership list of the National Association of Pension Funds (NAPF). The list does not
include all Pension Funds and there is a need to compensate for this undercoverage.
This article examines the possible alternative approaches and proposes the use of an uplift
factor of 1.22 which forms the basis of the revised figures published by ONS on 16 May

The relaunched series represent the best estimates from the survey but, given the uncertainty
of the register, they are less reliable than other series derived more conventionally by
grossing sample estimates to register totals. The factor to be used to uplift future years' data
will be kept under review.


This article considers some of the methodological issues surrounding the ONS inquiries to
Pension Funds. The work re-examines and builds upon a methodological review that was
undertaken by ONS in 1999 and concentrates on issues concerned with the creation of a
sampling frame and population. The report "Pension Funds inquiries redesign" (June 1999)
can be found here. It is generally recognised that there are problems in compiling a
comprehensive statistical sampling frame for deriving estimates of pension funds


Estimates of Pension Funds' assets and liabilities, transactions in these assets, and their
income and expenditure are needed by National Accountants to meet European System of
Accounts (ESA) requirements for National Accounts. To meet this need the ONS conducts
three statutory surveys of Pension Funds:

       •    Quarterly Transactions in Financial Assets
       •    Quarterly Income and Expenditure
       •    Annual Balance Sheet

The results of these surveys are published, and are therefore used more widely, for example,
in support of analysis of the pensions industry.

Most ONS business surveys use the Inter Departmental Business Register (IDBR) as the
sampling frame and to provide population information. The IDBR has two main sources - the
Value Added Tax (VAT) and Pay as You Earn (PAYE) systems - and combines these into a
single register. However, the present IDBR is not suitable for surveys of pension funds;
pension funds are not generally registered for VAT and do not always have employees and so
may be missing from the IDBR. Moreover, there would be difficulties surrounding the
possible misclassification of businesses. The IDBR generally contains information relating to
the companies which are running pension schemes for their employees, and not to the
pension schemes themselves.
The unit of interest for the inquiries is the pension fund, which can consist of more than one
pension scheme. The target population is all self-administered funds in the UK, including
those that are partly insured. Funds that are fully insured are excluded from this inquiry as
they are covered by similar surveys addressed to Insurance companies. A self-administered
pension scheme is defined as an occupational pension scheme with units invested in one or
more managed schemes or unit trusts. The trustees of these types of schemes can employ
either an in-house fund manager to make the day-to-day investment decisions or they can opt
to use an external manager to manage the investment. An insured scheme is one where the
scheme's trustees hold, as a sole asset, an insurance policy contract or an annuity contract. All
the scheme's assets are held in one insurance company and have a guaranteed level of return.
A number of sources for the population for the Pension Funds surveys were considered in

       •    Occupational Pensions Regulatory Authority (OPRA)
       •    Pensions Schemes Office (PSO)
       •    National Association of Pension Funds (NAPF)
       •    Government Actuary Department (GAD)

Some of the main requirements for a register source are up to date accurate information; high
coverage of target population; the presence of auxiliary variables to aid in sample design and
improve the accuracy of estimates; markers for self-administered/insured funds and, most
importantly, the register needed to be available to ONS.

Initial work in 1999 suggested that the OPRA list should be the source of the register but,
although the register has complete coverage, it is at scheme level. Not all of the variables
covered by the inquiries are available at scheme level. It was therefore decided to use NAPF
as the register source even though it only has partial coverage of pension funds. In order to
estimate the undercoverage of the NAPF register, ONS carried out an exercise to match the
NAPF members to OPRA schemes.

The two registers were matched to calculate the proportion of the OPRA membership
covered by the NAPF population. This was used to determine a suitable factor to apply to
survey estimates for the NAPF population in order to estimate for the complete population.
Because Local Authorities are generally not members of NAPF, all Local Authority pension
funds were surveyed, and so Local Authority pension funds/schemes were excluded from
both the OPRA and NAPF lists for the purpose of calculating this factor.

The OPRA file had 11.5 million members in self-administered schemes; 9.8 million members
were in schemes that were members of NAPF and 1.8 million members were in non-NAPF
schemes. 139 NAPF pension funds were not matched to self-administered schemes in the
OPRA file. Further inspection of these funds discovered that these were designated as
insured schemes in the OPRA file. It was assumed that these funds were partly insured.
With this assumption, NAPF members covered 84 per cent of the OPRA self-administered
population. In 1999, this was judged to be sufficient and that the estimates from the grossed
non-local authority funds should be further scaled by a factor, which should be 1.18 in the
first instance but recalculated on an annual basis.
Experience on updating the uplift factor

An attempt was made to update the uplift factor for the first time early in 2002. A register of
NAPF members had been regularly kept up to date for the purpose of sample design and so
was readily available. For the purpose of calculating a new update factor, a new OPRA file
was needed. The work was undertaken in the period March and April 2002 and some details
of the matching exercise are shown in annex A. This work has revealed that the method for
producing the 1999 uplift factor relied upon a number of assumptions to do with the
treatment of units which had been misclassified on either the OPRA or NAPF registers.
Discussions were held with NAPF and OPRA to try to understand the structure of the
Pension Funds industry and the content and quality of the present datafiles and those from the
1999 exercise. As a result of these discussions, it was concluded that the OPRA list
underlying the 1998 exercise was of lower quality. The number of schemes in OPRA had
fallen from 165,000 in 1999 to 104,000 in 2001. This was largely due to weeding of "dead"
schemes. In addition, the measure of membership had been changed from active
(contributing) members to total members shortly before the earlier extract was taken: some of
the information in the earlier file would still be based on active members compared to the
total members in the later file. It was therefore impossible to produce a time series based on
an OPRA/NAPF uplift factor. However, the 2001 OPRA register was thought to be more
soundly based but the calculation of an uplift factor for the Pension Funds survey relied on a
number of assumptions regarding the comparability of membership figures between small
and large schemes and on accurate classification between insured and self-administered
funds. The assumption regarding membership is that the discrepancy between NAPF and
OPRA membership figures for NAPF (mainly large) schemes can also be used to adjust the
membership of the non-NAPF (mainly small) schemes.


There were a number of different uplift factors that could be justified and the range of them
gives some indication of the quality of the final survey estimates.

The recent work based on the 2001 registers gives a best estimate of 1.23 (see annex A),
although this estimate is itself sensitive to the treatment of outliers and assumptions about
misclassifications. The simple use of the ratio of total OPRA scheme membership to total
NAPF scheme membership as the uplift factor suffers from the problem of comparability of
membership figures between the two sources for any given scheme. Analysis of the schemes
which could be matched between the two sources shows that membership figures differ and
that on the whole the OPRA membership figures are slightly lower. The uplift factor should
therefore be adjusted for this difference and in doing so it is necessary to assume that the
same adjustment would be appropriate for the schemes which were not matched as for those
that were. Use of an adjustment factor equal to total OPRA matched membership over total
NAPF matched membership gives an adjusted uplift factor of 1.25. A more sophisticated
analysis in which extreme cases(or outliers) are eliminated gives rise to an uplift factor of

Comparisons across time are not possible with this approach since the OPRA system changed
very substantially during the period in question. The method used to derive the uplift factor
was different in 1998 from that in 2001 and it is not possible to rework the 1998 factor on a
comparable basis due to lack of historical OPRA data.
Share Ownership Survey

It is also possible to estimate an uplift factor using information from the Share Ownership
Survey (SOS). The Pension Funds surveys collects information on the full range of assets,
liabilities, financial flows, income and expenditure. For one of these assets (UK Quoted
Ordinary Shares), there is an alternative source; the Share Ownership survey, which provides
a sectoral breakdown of the holdings of UK quoted ordinary shares. For a description of the
methodology see Annex A of The Share Ownership Report.

           Estimates of Pension Funds holdings of UK quoted ordinary shares

              £billion                            1999                          2000
 Pension Fund survey without uplift               290.4                         251.9
 Share Ownership survey                           348.4                         310.0
 Implied uplift                                    1.20                          1.23

The implied uplift factor assumes that the ratio of UK shares held as a proportion of total
assets is the same for NAPF funds as for the pension schemes not covered by NAPF
members. Further validation of the SOS was carried out by comparing its estimates of shares
held by insurance companies with the equivalent estimates from the insurance surveys (which
are similar to the Pension Funds surveys but are more soundly based and do not require an
uplift factor). The estimates were found to be within about 2 per cent of each other. The
uplift factors arrived at using the SOS approach were 1.20 at end-1999 and 1.23 for end-2000
(the last year for which data are currently available).

The choice of uplift factor

There are therefore three possible estimates of the uplift factor relating to different years and
based on two different methodologies. These are in the range 1.20 to 1.23. There is only
weak evidence of a trend from the SOS given that the validity of the underlying assumption
may vary from year to year and that both the SOS and pension funds inquiries are subject to
sampling and non-sampling errors. There is therefore not enough evidence to justify
applying uplift factors with a rising trend over time. This was backed up by the discussions
with NAPF and OPRA, which suggested that the proportion of Funds covered by NAPF
schemes would be broadly constant over that time period. We have assumed that it is
acceptable to average these estimates in some way even though they relate to different time
periods. A simple average of these three estimates gives an uplift factor of 1.22

Revisions to earlier years

The SOS results for 1998 were also compared with those from the pension fund survey,
although the latter were based on the previous methodology which did not use the
NAPF/OPRA registers. There is a difference of about 3 per cent. Data for the pre-1999
period is not in a compatible format, and figures for these earlier years cannot be
recalculated. There will therefore always be a small discontinuity in the series between 1998
and 1999. Users should therefore be particularly careful about drawing conclusions about
changes covering these two periods.
The numbers

The following table summarises the main figures from the Pension Funds survey published
on 16 May and indicates the extent of revisions that have been made to key variables. The
full range of published results from the Pension Funds survey are shown at As well as taking account of the new
methodology, they also reflect late returns to the surveys that were not included in the
previous estimates. Also, because the 22 per cent uplift factor is not applied to Local
Authority pension funds, the revisions reflect the relative importance of Local Authorities to
each variable. In addition the raw survey estimates are subject to alignment adjustments as
part of the process. This ensures that the estimates of income and expenditure, investment
flows and assets and liabilities, which are each subject to statistical errors of different size
and sign, present a consistent picture of the Pension Funds industry. This process, which is
similar to that used in national accounts, is described in greater detail in annex B.
                                        Assets (£ billion)
                    Latest estimate (revisions from previously published)

                                   1998                    1999                   2000
  Total assets                    699 (-)                813 (+36)              768 (+117)
 of which: UK quoted              332 (-)                353 (-42)              296 (+44)
ordinary shares

                                 Net investment (£ billion)
                   Latest estimate (revisions from previously published)

                                    1999                     2000                    2001
 Total net investment            6.45 (+0.46)             8.81 (+2.41)               10.77

                             Income and expenditure (£ billion)
                   Latest estimate (revisions from previously published)

                                     1999                     2000                    2001
Total contributions less         13.69 (-0.26)            15.69 (+2.08)               16.23

The future

The Pension Funds survey is based on a register that is less well founded than for other ONS
business surveys. The scale of the difference between the population used for sampling
(NAPF) and the population used for grossing is relatively large at 22 per cent and therefore
the survey results are consequently less accurate than other ONS business surveys, other
things being equal. The 1.22 factor is the best estimate available given the current
information, but the factor will need to be reassessed when the next SOS/pension funds
survey comparison is made and when the next NAPF/OPRA matching exercise is undertaken.
The factor to be used for uplifting future years' data , and the overall methodology will be
kept under review.

In summary, the basis of the calculation of the Pension Funds estimates and the plan for
further work is as follows:

       •    no revision should be made to 1998 or earlier data, these being based on the old
       •    a constant uplift of 22 per cent should be made to 1999 and subsequent years
            (both annual and quarterly)
       •    the factor to be used for uplifting future years’ data, and the overall methodology
            should be kept under review

Office for National Statistics

16 May 2002
ANNEX A - Possible uplift factors under different assumptions

The following tables categorise the number of members in the various NAPF/OPRA

However, for the 2002 exercise, more information was available from NAPF and OPRA that
enabled alternative assumptions to be considered. These are discussed below.

There are many other ways in which an uplift factor could be calculated. Each method relies
on assumptions about misclassified and unmatched schemes. This annex concentrates on 3
main methods which lead to a cluster of uplift factors in the range 1.21 to 1.25. This is an
area of considerable uncertainty and the closeness of this cluster should not be taken entirely
as a measure of the accuracy of this methodology.

NAPF membership by type of scheme and success of matching to OPRA

                       Not insured on               Insured on NAPF              Partly insured
                       NAPF                                                        on NAPF
                                        Part of a mixed   Part of a solely
                                        NAPF fund         insured NAPF fund
Matched to not          A=9,516,682          B=6,587            C=1,564            D=2,880
insured OPRA
Matched to               E=434,222         F=200,073            G=7,609              H=0
insured OPRA
Unmatched to             I=455,933            J=2               K=5,436               L=0

Population used for grossing the survey results = A+B+D+E+F+H+I+J+L

OPRA occupational pension scheme membership by type of scheme and success of
matching to NAPF

                        Not insured on OPRA                    Insured on OPRA
                                                Solely insured scheme Partly insured scheme
Matched to NAPF              a=9,204,924                          b+c=639,024
Unmatched to                 d=3,610,076                         e+f=2,747,722

Target population = a+d

1.      If it is assumed that no misclassifications on the OPRA file and membership on
        OPRA equals membership on NAPF then the

•    uprating factor     = (a+d)/(A+B+D+E+F+H+I+J+L)
                         =                       (9,204,924 + 3,610,076)
                        (9,516,682 + 6,587 + 2,880 + 434,222 + 200,073 + 0 + 455,933 + 2 + 0)

                         = 1.21
2.      If the membership of OPRA does not equal the membership of NAPF for matched
        schemes (and it does not), an adjustment (a/A in the above table) can be applied to the

•    uprating factor   = (a+d)/(a/A x ( A+B+D+E+F+H+I+J+L))

                       =             (9,204,924 + 3,610,076)
        (9,204,924) x (9,516,682 + 6,587 + 2,880 + 434,222 + 200,073 + 0 + 455,933 + 2 + 0)

                                = 1.25

3.      The method described at 2. above uses an adjustment a/A equal to .967, which is the
        ratio of membership from NAPF to membership from OPRA for matched funds.
        However, for individual funds there are some large discrepancies between NAPF and
        OPRA membership and a more sophisticated estimate eliminates outliers from the
        calculation. If the outlier cut off is defined by (NAPF membership/OPRA
        membership) greater than 1.5 or less than 0.67, the adjustment a/A = 0.979. Under
        this assumption:

•    uprating factor   = 1.23
ANNEX B: Alignment of the results

The major use for the pension funds figures is in the national accounts. For this purpose it is
important that the three inquiries produce a consistent set of data so that the different
accounts into which the figures feed balance. The results from each of the three pension funds
inquiries however contain a degree of error. The errors vary from sampling error (which
depends on the particular sample taken since each sample would result in a slightly different
estimate) to non-sampling errors such as problems with the register and errors in response. In
order to produce a consistent dataset, the results from the different inquiries are "aligned".
This works on the principle that surpluses of income over expenditure (identified from the
quarterly income and expenditure inquiries) are used to fund investment. Effectively the
investment flows figures from the quarterly transactions inquiries are adjusted to bring them
in line with the annual balance sheet data and the quarterly income and expenditure
information. The whole process is similar to procedures used elsewhere in ONS where more
comprehensive annual inquiries (usually based on the audited accounts of companies) are
used to confirm monthly and quarterly inquiries (based on management accounts of lower

Different procedures are used to align figures on short-term and longer-term instruments
collected in the quarterly transactions inquiries. Data on short-term instruments such as
certificates of deposit and commercial paper are collected as closing balances in both the
quarterly transactions and annual balance sheet inquiries. The alignment procedure here
allocates the difference between the fourth quarter and annual closing balances across the
derived investment flows for the four quarters equally.

For longer-term instruments, the total investment flows from the quarterly transactions
inquiries are aggregated across the four quarters. The result is then equated to the surplus of
income over expenditure from the quarterly income and expenditure inquiries adjusted for
investment in the short-term instruments from the annual balance sheet inquiries. Resulting
factors are created for positive and negative components and applied to the investment
figures for individual longer-term instruments to produce aligned series.

To top