PENSION FUNDS SURVEYS – TECHNICAL ARTICLE Summary The ONS Pension Funds survey is directed at a sample of UK Pension Funds drawn from the membership list of the National Association of Pension Funds (NAPF). The list does not include all Pension Funds and there is a need to compensate for this undercoverage. This article examines the possible alternative approaches and proposes the use of an uplift factor of 1.22 which forms the basis of the revised figures published by ONS on 16 May 2002. The relaunched series represent the best estimates from the survey but, given the uncertainty of the register, they are less reliable than other series derived more conventionally by grossing sample estimates to register totals. The factor to be used to uplift future years' data will be kept under review. Introduction This article considers some of the methodological issues surrounding the ONS inquiries to Pension Funds. The work re-examines and builds upon a methodological review that was undertaken by ONS in 1999 and concentrates on issues concerned with the creation of a sampling frame and population. The report "Pension Funds inquiries redesign" (June 1999) can be found here. It is generally recognised that there are problems in compiling a comprehensive statistical sampling frame for deriving estimates of pension funds characteristics. Background Estimates of Pension Funds' assets and liabilities, transactions in these assets, and their income and expenditure are needed by National Accountants to meet European System of Accounts (ESA) requirements for National Accounts. To meet this need the ONS conducts three statutory surveys of Pension Funds: • Quarterly Transactions in Financial Assets • Quarterly Income and Expenditure • Annual Balance Sheet The results of these surveys are published, and are therefore used more widely, for example, in support of analysis of the pensions industry. Most ONS business surveys use the Inter Departmental Business Register (IDBR) as the sampling frame and to provide population information. The IDBR has two main sources - the Value Added Tax (VAT) and Pay as You Earn (PAYE) systems - and combines these into a single register. However, the present IDBR is not suitable for surveys of pension funds; pension funds are not generally registered for VAT and do not always have employees and so may be missing from the IDBR. Moreover, there would be difficulties surrounding the possible misclassification of businesses. The IDBR generally contains information relating to the companies which are running pension schemes for their employees, and not to the pension schemes themselves. The unit of interest for the inquiries is the pension fund, which can consist of more than one pension scheme. The target population is all self-administered funds in the UK, including those that are partly insured. Funds that are fully insured are excluded from this inquiry as they are covered by similar surveys addressed to Insurance companies. A self-administered pension scheme is defined as an occupational pension scheme with units invested in one or more managed schemes or unit trusts. The trustees of these types of schemes can employ either an in-house fund manager to make the day-to-day investment decisions or they can opt to use an external manager to manage the investment. An insured scheme is one where the scheme's trustees hold, as a sole asset, an insurance policy contract or an annuity contract. All the scheme's assets are held in one insurance company and have a guaranteed level of return. A number of sources for the population for the Pension Funds surveys were considered in 1999: • Occupational Pensions Regulatory Authority (OPRA) • Pensions Schemes Office (PSO) • National Association of Pension Funds (NAPF) • Government Actuary Department (GAD) Some of the main requirements for a register source are up to date accurate information; high coverage of target population; the presence of auxiliary variables to aid in sample design and improve the accuracy of estimates; markers for self-administered/insured funds and, most importantly, the register needed to be available to ONS. Initial work in 1999 suggested that the OPRA list should be the source of the register but, although the register has complete coverage, it is at scheme level. Not all of the variables covered by the inquiries are available at scheme level. It was therefore decided to use NAPF as the register source even though it only has partial coverage of pension funds. In order to estimate the undercoverage of the NAPF register, ONS carried out an exercise to match the NAPF members to OPRA schemes. The two registers were matched to calculate the proportion of the OPRA membership covered by the NAPF population. This was used to determine a suitable factor to apply to survey estimates for the NAPF population in order to estimate for the complete population. Because Local Authorities are generally not members of NAPF, all Local Authority pension funds were surveyed, and so Local Authority pension funds/schemes were excluded from both the OPRA and NAPF lists for the purpose of calculating this factor. The OPRA file had 11.5 million members in self-administered schemes; 9.8 million members were in schemes that were members of NAPF and 1.8 million members were in non-NAPF schemes. 139 NAPF pension funds were not matched to self-administered schemes in the OPRA file. Further inspection of these funds discovered that these were designated as insured schemes in the OPRA file. It was assumed that these funds were partly insured. With this assumption, NAPF members covered 84 per cent of the OPRA self-administered population. In 1999, this was judged to be sufficient and that the estimates from the grossed non-local authority funds should be further scaled by a factor, which should be 1.18 in the first instance but recalculated on an annual basis. Experience on updating the uplift factor An attempt was made to update the uplift factor for the first time early in 2002. A register of NAPF members had been regularly kept up to date for the purpose of sample design and so was readily available. For the purpose of calculating a new update factor, a new OPRA file was needed. The work was undertaken in the period March and April 2002 and some details of the matching exercise are shown in annex A. This work has revealed that the method for producing the 1999 uplift factor relied upon a number of assumptions to do with the treatment of units which had been misclassified on either the OPRA or NAPF registers. Discussions were held with NAPF and OPRA to try to understand the structure of the Pension Funds industry and the content and quality of the present datafiles and those from the 1999 exercise. As a result of these discussions, it was concluded that the OPRA list underlying the 1998 exercise was of lower quality. The number of schemes in OPRA had fallen from 165,000 in 1999 to 104,000 in 2001. This was largely due to weeding of "dead" schemes. In addition, the measure of membership had been changed from active (contributing) members to total members shortly before the earlier extract was taken: some of the information in the earlier file would still be based on active members compared to the total members in the later file. It was therefore impossible to produce a time series based on an OPRA/NAPF uplift factor. However, the 2001 OPRA register was thought to be more soundly based but the calculation of an uplift factor for the Pension Funds survey relied on a number of assumptions regarding the comparability of membership figures between small and large schemes and on accurate classification between insured and self-administered funds. The assumption regarding membership is that the discrepancy between NAPF and OPRA membership figures for NAPF (mainly large) schemes can also be used to adjust the membership of the non-NAPF (mainly small) schemes. NAPF/OPRA There were a number of different uplift factors that could be justified and the range of them gives some indication of the quality of the final survey estimates. The recent work based on the 2001 registers gives a best estimate of 1.23 (see annex A), although this estimate is itself sensitive to the treatment of outliers and assumptions about misclassifications. The simple use of the ratio of total OPRA scheme membership to total NAPF scheme membership as the uplift factor suffers from the problem of comparability of membership figures between the two sources for any given scheme. Analysis of the schemes which could be matched between the two sources shows that membership figures differ and that on the whole the OPRA membership figures are slightly lower. The uplift factor should therefore be adjusted for this difference and in doing so it is necessary to assume that the same adjustment would be appropriate for the schemes which were not matched as for those that were. Use of an adjustment factor equal to total OPRA matched membership over total NAPF matched membership gives an adjusted uplift factor of 1.25. A more sophisticated analysis in which extreme cases(or outliers) are eliminated gives rise to an uplift factor of 1.23. Comparisons across time are not possible with this approach since the OPRA system changed very substantially during the period in question. The method used to derive the uplift factor was different in 1998 from that in 2001 and it is not possible to rework the 1998 factor on a comparable basis due to lack of historical OPRA data. Share Ownership Survey It is also possible to estimate an uplift factor using information from the Share Ownership Survey (SOS). The Pension Funds surveys collects information on the full range of assets, liabilities, financial flows, income and expenditure. For one of these assets (UK Quoted Ordinary Shares), there is an alternative source; the Share Ownership survey, which provides a sectoral breakdown of the holdings of UK quoted ordinary shares. For a description of the methodology see Annex A of The Share Ownership Report. Estimates of Pension Funds holdings of UK quoted ordinary shares £billion 1999 2000 Pension Fund survey without uplift 290.4 251.9 Share Ownership survey 348.4 310.0 Implied uplift 1.20 1.23 The implied uplift factor assumes that the ratio of UK shares held as a proportion of total assets is the same for NAPF funds as for the pension schemes not covered by NAPF members. Further validation of the SOS was carried out by comparing its estimates of shares held by insurance companies with the equivalent estimates from the insurance surveys (which are similar to the Pension Funds surveys but are more soundly based and do not require an uplift factor). The estimates were found to be within about 2 per cent of each other. The uplift factors arrived at using the SOS approach were 1.20 at end-1999 and 1.23 for end-2000 (the last year for which data are currently available). The choice of uplift factor There are therefore three possible estimates of the uplift factor relating to different years and based on two different methodologies. These are in the range 1.20 to 1.23. There is only weak evidence of a trend from the SOS given that the validity of the underlying assumption may vary from year to year and that both the SOS and pension funds inquiries are subject to sampling and non-sampling errors. There is therefore not enough evidence to justify applying uplift factors with a rising trend over time. This was backed up by the discussions with NAPF and OPRA, which suggested that the proportion of Funds covered by NAPF schemes would be broadly constant over that time period. We have assumed that it is acceptable to average these estimates in some way even though they relate to different time periods. A simple average of these three estimates gives an uplift factor of 1.22 Revisions to earlier years The SOS results for 1998 were also compared with those from the pension fund survey, although the latter were based on the previous methodology which did not use the NAPF/OPRA registers. There is a difference of about 3 per cent. Data for the pre-1999 period is not in a compatible format, and figures for these earlier years cannot be recalculated. There will therefore always be a small discontinuity in the series between 1998 and 1999. Users should therefore be particularly careful about drawing conclusions about changes covering these two periods. The numbers The following table summarises the main figures from the Pension Funds survey published on 16 May and indicates the extent of revisions that have been made to key variables. The full range of published results from the Pension Funds survey are shown at http://www.statistics.gov.uk/products/p502.asp As well as taking account of the new methodology, they also reflect late returns to the surveys that were not included in the previous estimates. Also, because the 22 per cent uplift factor is not applied to Local Authority pension funds, the revisions reflect the relative importance of Local Authorities to each variable. In addition the raw survey estimates are subject to alignment adjustments as part of the process. This ensures that the estimates of income and expenditure, investment flows and assets and liabilities, which are each subject to statistical errors of different size and sign, present a consistent picture of the Pension Funds industry. This process, which is similar to that used in national accounts, is described in greater detail in annex B. Assets (£ billion) Latest estimate (revisions from previously published) 1998 1999 2000 Total assets 699 (-) 813 (+36) 768 (+117) of which: UK quoted 332 (-) 353 (-42) 296 (+44) ordinary shares Net investment (£ billion) Latest estimate (revisions from previously published) 1999 2000 2001 Total net investment 6.45 (+0.46) 8.81 (+2.41) 10.77 Income and expenditure (£ billion) Latest estimate (revisions from previously published) 1999 2000 2001 Total contributions less 13.69 (-0.26) 15.69 (+2.08) 16.23 refunds The future The Pension Funds survey is based on a register that is less well founded than for other ONS business surveys. The scale of the difference between the population used for sampling (NAPF) and the population used for grossing is relatively large at 22 per cent and therefore the survey results are consequently less accurate than other ONS business surveys, other things being equal. The 1.22 factor is the best estimate available given the current information, but the factor will need to be reassessed when the next SOS/pension funds survey comparison is made and when the next NAPF/OPRA matching exercise is undertaken. The factor to be used for uplifting future years' data , and the overall methodology will be kept under review. Conclusion In summary, the basis of the calculation of the Pension Funds estimates and the plan for further work is as follows: • no revision should be made to 1998 or earlier data, these being based on the old methodology • a constant uplift of 22 per cent should be made to 1999 and subsequent years (both annual and quarterly) • the factor to be used for uplifting future years’ data, and the overall methodology should be kept under review Office for National Statistics 16 May 2002 ANNEX A - Possible uplift factors under different assumptions The following tables categorise the number of members in the various NAPF/OPRA schemes/funds. However, for the 2002 exercise, more information was available from NAPF and OPRA that enabled alternative assumptions to be considered. These are discussed below. There are many other ways in which an uplift factor could be calculated. Each method relies on assumptions about misclassified and unmatched schemes. This annex concentrates on 3 main methods which lead to a cluster of uplift factors in the range 1.21 to 1.25. This is an area of considerable uncertainty and the closeness of this cluster should not be taken entirely as a measure of the accuracy of this methodology. NAPF membership by type of scheme and success of matching to OPRA Not insured on Insured on NAPF Partly insured NAPF on NAPF Part of a mixed Part of a solely NAPF fund insured NAPF fund Matched to not A=9,516,682 B=6,587 C=1,564 D=2,880 insured OPRA schemes Matched to E=434,222 F=200,073 G=7,609 H=0 insured OPRA schemes Unmatched to I=455,933 J=2 K=5,436 L=0 OPRA Population used for grossing the survey results = A+B+D+E+F+H+I+J+L OPRA occupational pension scheme membership by type of scheme and success of matching to NAPF Not insured on OPRA Insured on OPRA Solely insured scheme Partly insured scheme Matched to NAPF a=9,204,924 b+c=639,024 Unmatched to d=3,610,076 e+f=2,747,722 NAPF Target population = a+d 1. If it is assumed that no misclassifications on the OPRA file and membership on OPRA equals membership on NAPF then the • uprating factor = (a+d)/(A+B+D+E+F+H+I+J+L) = (9,204,924 + 3,610,076) (9,516,682 + 6,587 + 2,880 + 434,222 + 200,073 + 0 + 455,933 + 2 + 0) = 1.21 2. If the membership of OPRA does not equal the membership of NAPF for matched schemes (and it does not), an adjustment (a/A in the above table) can be applied to the denominator: • uprating factor = (a+d)/(a/A x ( A+B+D+E+F+H+I+J+L)) = (9,204,924 + 3,610,076) (9,204,924) x (9,516,682 + 6,587 + 2,880 + 434,222 + 200,073 + 0 + 455,933 + 2 + 0) (9,516,682) = 1.25 3. The method described at 2. above uses an adjustment a/A equal to .967, which is the ratio of membership from NAPF to membership from OPRA for matched funds. However, for individual funds there are some large discrepancies between NAPF and OPRA membership and a more sophisticated estimate eliminates outliers from the calculation. If the outlier cut off is defined by (NAPF membership/OPRA membership) greater than 1.5 or less than 0.67, the adjustment a/A = 0.979. Under this assumption: • uprating factor = 1.23 ANNEX B: Alignment of the results The major use for the pension funds figures is in the national accounts. For this purpose it is important that the three inquiries produce a consistent set of data so that the different accounts into which the figures feed balance. The results from each of the three pension funds inquiries however contain a degree of error. The errors vary from sampling error (which depends on the particular sample taken since each sample would result in a slightly different estimate) to non-sampling errors such as problems with the register and errors in response. In order to produce a consistent dataset, the results from the different inquiries are "aligned". This works on the principle that surpluses of income over expenditure (identified from the quarterly income and expenditure inquiries) are used to fund investment. Effectively the investment flows figures from the quarterly transactions inquiries are adjusted to bring them in line with the annual balance sheet data and the quarterly income and expenditure information. The whole process is similar to procedures used elsewhere in ONS where more comprehensive annual inquiries (usually based on the audited accounts of companies) are used to confirm monthly and quarterly inquiries (based on management accounts of lower quality). Different procedures are used to align figures on short-term and longer-term instruments collected in the quarterly transactions inquiries. Data on short-term instruments such as certificates of deposit and commercial paper are collected as closing balances in both the quarterly transactions and annual balance sheet inquiries. The alignment procedure here allocates the difference between the fourth quarter and annual closing balances across the derived investment flows for the four quarters equally. For longer-term instruments, the total investment flows from the quarterly transactions inquiries are aggregated across the four quarters. The result is then equated to the surplus of income over expenditure from the quarterly income and expenditure inquiries adjusted for investment in the short-term instruments from the annual balance sheet inquiries. Resulting factors are created for positive and negative components and applied to the investment figures for individual longer-term instruments to produce aligned series.
Pages to are hidden for
"Pension_Funds_Survey_Technical_Article"Please download to view full document