3. Challenges to inclusive analysis - Lack of harmonised standards

Potentially important problems for inclusive analysis are that definitions of some protected characteristics differ from one administrative dataset to another and that data collected for administrative purposes may focus on policy issues rather than the concepts embodied in the harmonised standards used by the census and ONS surveys.

3(a): Disability

For example, Department for Work and Pensions (DWP) administrative data focusses on whether one’s disability limits one’s ability to undertake paid employment, a much narrower concept than that of the harmonised standard. Similarly, Department for Transport (DfT) data on disability will relate to mobility issues. Both sources may therefore involve major undercounts when compared with measures which are consistent with the 2010 Equality Act definition of disability. Another example would be around NHS or wider healthcare data which does not, and cannot, capture disability or long-term conditions in ways which the census does; this creates major analytical challenges.

While in the case of ethnicity administrative datasets have tended to use the harmonised standard developed by ONS, this does not appear to be the case for disability. Given the very specific administrative concerns of departments administering programmes for disabled people, it is unlikely that the current harmonised standard for disability could be rolled out across administrative authorities in its current form.

Because of the very early stage of ONS’s research progress on protected characteristics such as disability, it is very hard to be sure how tractable the problems will turn out to be: we note that the most recent documentation on the harmonised standard for disability refers only to survey datasets. Inconsistency of concepts between the different datasets is likely to create serious challenges.

3(b): Intersectional analyses

The Consultation Document indicates that occupation and the National Statistics Socio-economic Classification (NS-SeC99) are at an exploratory stage of research and have limited coverage at present. These measures are likely to be important for intersectional analysis. The ONS’s use of linked census/Admin data to understand whether the impact of Covid on ethnic minorities could be explained by the occupations undertaken by minorities in this methodology report is a good example of the need for this kind of data. To the best of our knowledge, occupational data is currently available in very few if any administrative datasets. Its absence is likely to severely limit our ability to understand disparities in the outcomes obtained by different groups with protected characteristics.

It also appears likely that some key outcome measures will also be missing from the linked administrative datasets. While the Consultation Document’s Figure 9 as referred to above suggests that labour market status is at a relatively advanced stage with full coverage and partial development, it appears that this is only true for income and for receipt of benefits (from His Majesties Revenue and Customs (HMRC) and DWP administrative data respectively). It is unclear whether there will be any measure of unemployment, for example. According to the Social Mobility Commission’s State of the Nation Report, 2023 (PDF, 25.5MB), unemployment is one of the most important indicators of disadvantage with recent research demonstrating elevated risks of unemployment among people with a disability and among some ethnic minorities. As discussed below, there are considerable problems with administrative data on unemployment based on the claimant count. Our understanding is that this key outcome measure of disadvantage will therefore not be included.

3(c): UK wide variation

A further issue that may become relevant is harmonisation across the four nations of the UK. We appreciate that the ONS is responsible for the census in England and Wales while Northern Ireland Statistics and Research agency (NISRA) is the responsible body in Northern Ireland and National Records of Scotland (NRS) in Scotland. Our understanding is that both NISRA and NRS will be making recommendations to the devolved administrations about the future of their population statistics. While there are some differences in the content of the census in Scotland and Northern Ireland from that in England and Wales, in general there is a high degree of comparability in the census across the four nations with respect to most protected characteristics, reflecting the close working arrangements and agreements between the national statisticians.

In contrast, there are some major divergences between the administrative categories in use. For example, in the case of school statistics the DfE in England has developed measures derived from the administrative category of eligibility for Free School Meals whereas Scotland uses an area-based measure – the Scottish Index of Multiple Deprivation (SIMD). As they stand at present, these measures are incommensurable.

In addition to the existing differences in the policy environments between Northern Ireland, Scotland and England and Wales, there must be a risk that future policy developments might entail greater divergence between their respective linked administrative datasets than there currently are with the census. The loss of UK-wide data sources would hinder deeper understanding of inclusion and its relationship with policies and geographical context and would limit opportunities for analysis and learning.

3(d): Continuity over time

We are also concerned about the risks of lower continuity over time in the absence of a regular census. An example concerns the changes to job-seekers’ allowances in the 2010s which meant that the claimant count figure – an administrative data source – which in the 2000s gave a consistent (albeit incomplete) estimate of the number of people out-of-work/unemployed. However, as administrative definitions changed and people were moved onto Universal Credit, this became an unsuitable proxy or rapid indicator of the changing level of unemployment. Between November 2015 and November 2019, the number of claimants across the whole of the UK increased by around 420,000, an increase of over 70%. For many areas or constituencies, the claimant count more than doubled during this period: this was in part because Universal Credit required a broader span of people to look for work than was the case for legacy benefits.

While the DWP has now developed and published an alternative time series which is intended to be consistent over time with the new Universal Credit requirements, the ONS regards this as ‘experimental’ and is not planning to include it in the proposed administrative database. One also needs to factor in the continuing need to undertake such exercises and the lack of control that ONS will have over these administrative changes.

It is also surprising that the DWP does not supply any measure of the precision of the estimates for local authorities or their sensitivity to different modelling assumptions. A dynamic policy environment across the UK therefore has the potential to lead to lack of continuity over time in administrative-based datasets, and hence a reduction in our ability to understand the changes taking place.

3(e): Surveys: limitations and resourcing

Major UK-wide surveys using harmonised standards, such as the Labour Force Survey/Annual Population Survey (LFS/APS), are planned to continue under the FPMS proposals. Our expectation is that such surveys will become of even greater importance in the new statistical environment.

While, as noted earlier, we are concerned at the declining response rates to the LFS/APS, another important limitation of the LFS is that the sample is not sufficiently large to enable annual measures at a Local Authority level. Given the importance to stakeholders of geographically disaggregated measures of at-risk groups, and given the large geographical variations noted above in the coverage of administrative data on ethnicity – and potentially of other at-risk groups too – there will be a major challenge when analysing geographical variation in the disadvantages experienced by at-risk groups in the absence of the census. If the proposed administrative datasets fail to deliver, ONS could find itself obliged to expand the APS greatly, perhaps up to the scale of the German Mikrocensus, which is approximately 1% of Germany’s population annually. We recommend that this should form part of ONS’s contingency plans in the light of potential optimism bias in current development programmes. We note that in 2013 ONS published a paper ‘Beyond 2011’ exactly on these lines. It proposed a mandatory annual survey of 4% of households, with a target response of 900,000 households each year, covering those characteristics that could not be covered adequately from administrative sources. This would permit univariate statistics to be produced each year for local authorities, and bivariate (and more complex statistics) if a number of years were pooled. This would meet most of NSIDAC’s concerns, although such a large survey might well cancel out the cost savings from discontinuing the census (declining response rates mean that an ever larger and therefore more expensive sample might now need to be drawn each year).