Minutes and papers

Meeting of 19 January 2024

Time Item Presenter and Paper Description
10:25 Join call
10:30 Introductions, apologies, and actions Grant Fitzner Make introductions if necessary.
Inform Panel members of any apologies.
Update on outstanding actions.
10:35 Inflation measurement with high frequency data Peter Levell K, Fox and P, Levell and M, O'Connell. (2023). Inflation measurement with high frequency data. 23/29. London: Institute for Fiscal Studies. Available at: https://ifs.org.uk/publications/inflation-measurement-high-frequency-data (accessed: 9 January 2024). The availability of large transaction level datasets, such as retail scanner data, provides a wealth of information on prices and quantities that national statistical institutes can use to produce more accurate, timely, measures of inflation. However, there is no universally agreed upon method for calculating price indexes with such high frequency data, reflecting a lack of systematic evidence on the performance of different approaches. We use a dataset that covers 178 product categories comprising all fast-moving consumer goods over 8 years to provide a systematic comparison of the leading bilateral and multilateral index number methods for computing month-to-month inflation."
11:25 Communicating the impact of GEKS-Törnqvist on our Consumer Price Statistics Liam Greenhough
Verbal
This short presentation will aim to address some of the commonly raised questions about the GEKS-Törnqvist multilateral index methodology; particularly in relation to the complexity of the method, whether this means there is no longer a “fixed basket” and whether it affects what the index is measuring. Panel members views will be sought on whether the proposed answers to such questions are accurate and clear to allow a better understanding of the method amongst users.
12:05 Using HMRC Unit Value Indices for measuring inflation in homogenous trade commodities Chris Bloomer
APCP-T(24)01
We investigate the use of Unit Value Indices (UVIs) derived from HMRC admin microdata for measuring inflation in homogeneous UK imports and exports, potentially reducing public spending on ONS resources. The aggregation of UVIs adopts a multilateral index methodology, adapting to frequent changes in weight structure and addressing missing price data in component time series. This is particularly relevant when no price is recorded due to annual changes in the CN8 trade-focused classification used by HMRC. Considered methods include Multi GEKS-T (annual and monthly weighting) and Multi Time Product Dummy (annual and monthly weighting).
12:45 Publication status of papers Grant Fitzner
Verbal
Publication status of papers
12:50 Any other business Grant Fitzner Next meeting Friday 12 April
12:55 Meeting close

Download agenda

Inflation measurement with high frequency data*1

APCP-T(24)01 Using HMRC Unit Value Indices for measuring inflation in homogenous trade commodities*2

*1 For this paper citation please refer to the published agenda.

*2 This paper has not been published at this time, for more information please refer to the published minutes.

Meeting of 27 October 2023

Time Item Presenter and Paper Description
10:25 Join call
10:30 Introductions, apologies, and actions Grant Fitzner Make introductions if necessary.
Inform Panel members of any apologies.
Update on outstanding actions.
10:35 Clothing classification grouping Laura Christen / Ahmet Aydin
APCP-T(23)10
Update on the research on the classification and product grouping pipelines and discussion on ways to address some potential challenges and solutions associated with the methods explored.
11:15 Grocery data cleaning Mario Spina
APCP-T(23)11
Update on the research into data cleaning methodologies and treatment of dump prices into grocery scanner data.
11:50 Publication status of papers Grant Fitzner - Verbal
11:55 Any other business Grant Fitzner Proposed next meetings:
Friday 19 January 2024
Friday 12 April 2024
Friday 5 July 2024
Friday 11 October 2024
12:00 APCP-S arrive / lunch
12:25 Join call
12:30 Introductions,
apologies, and actions
Dame Kate Barker Make introductions if necessary.
Inform Panel members of any
apologies.
12:35 Impact analysis of
private rental prices
and second-hand cars
analysis
Natalie Jones – Private
rental prices (12:35 –
12:55)
Emma Halshaw – secondhand
cars (12:55 – 13:10)
A reminder of the data and
methods used for the private
rents and second-hand cars
categories with indicative
impacts on measures of
consumer price inflation.
13:10 Readiness assessment
of private rental prices
analysis and secondhand
cars
Michael Hardie Members of the Panels are
invited to advise on the
acceptability of introducing
these new data, methods,
systems and processes for rents
and second-hand cars into the
live production of consumer
price statistics in 2024, with a
particular focus on user
readiness (APCP-S) and methods
readiness (APCP-T).
13:45 Any other business Dame Kate Barker
13:50 Meeting close

Download agenda

Redacted APCP-T(23)10 Clothing Classification and Product Grouping

APCP-T(23)11 Scanner data research – data cleaning*

*This paper has not been published at this time, for more information please refer to the published minutes.

Meeting of 7 July 2023

Time Item Presenter and Paper Description
10:25 Join call
10:30 Introductions, apologies, and actions Grant Fitzner Make introductions if necessary.
Inform Panel members of any apologies.
Update on outstanding actions.
10:35 Potential future data source improvements Andrew Yeap / Matthew Corder
APCP-T(23)05
11:05 Implementation of private rental controls in the UK Aimee North
APCP-T(23)06
Paper outlining the risk from rental controls (in Scotland, consulting in Wales) to measuring OOH using the rental equivalence approach. Also presenting Scottish Government’s concerns with IPHRP arising from IPHRP continuing to report rising annual rental price inflation despite introduction of rental controls for existing lets, including impact analysis of Scottish Government’s proposed methodology change.
11:30 Scanner data research – date trimming Laura Christen / Liam Greenhough
APCP-T(23)07
Updated research into date trimming for the use in grocery scanner data.
11:55 Scanner data research – data cleaning Mario Spina / Liam Greenhough
APCP-T(23)08
Updated research into data cleaning for the use in grocery scanner data.
12:20 Scanner data research – Handling outlets and store-types Joanna Corless
Verbal
Updated research on the handling of different outlets and store types within groceries scanner data.
12:35 Update on timelines Helen Sands
Verbal update
12:45 Publication status of papers Grant Fitzner
Verbal
Publication status of papers
12:50 AOB Grant Fitzner Next meeting Friday 27 October
12:55 Meeting close

Download agenda

APCP-T(23)05 Potential future data source improvements*

APCP-T(23)06 Implementation of private rental price controls in the UK

APCP-T(23)07 Scanner data research, date trimming

APCP-T(23)08 Scanner data research, data cleaning

Scanner data research – store types presentation

*This paper has not been published at this time, for more information please refer to the published minutes.

Meeting of 14 April 2023

Time Item Presenter and Paper Description
10:25 Join call
10:30 Introductions, apologies, and actions Grant Fitzner Make introductions if necessary.
Inform Panel members of any apologies.
Update on outstanding actions.
10:35 Second hand cars Liam Greenhough
APCP-T(23)01
An update on the introduction of new data sources for second hand cars in CPI & CPIH
11:10 Timelines for introduction of alternative data sources Sofia Poni
Verbal update
11:25 Standard Errors Paul Smith
APCP-T(23)02
Updated research into variance estimation in UK consumer prices indices
12:00 HPI replatforming Aimee North
APCP-T(23)03
Proposed methodological updates to the HPI
12:35 Annual report Helen Sands
Verbal update
12:45 Publication status of papers Grant Fitzner
Verbal
Publication status of papers
12:50 Any other business Grant Fitzner Next meeting Friday 7 July
12:55 Meeting close

Download agenda

Meeting of 21 October 2022

Time Item Presenter and Paper Description
10:15
10:25
Arrival at venue
Join call
10:30 Introductions and apologies. Mr Michael Hardie Make introductions if necessary.
Inform panel members of any apologies.
10:45 Index methods: framework for using multilaterals Mr Liam Greenhough We present a range of lower-level index methods choices that we need to make in preparation for introducing alternative data sources for second-hand cars and rail fares in 2023.
11:45 Data cleaning within alternative data sources Dr Mario Spina We outline the data cleaning methods (primarily outlier detection and junk filtering) that we are exploring the use of with alternative data sources
12:15 AOB and date of next meeting Mr Michael Hardie Next meetings: 13 January 2023
Proposed: 14th April & 7th July 2023
12:30 Meeting close

Download agenda

Meeting of 8 July 2022

Time Item Presenter and Paper Description
10:25 Join call
10:30 Introductions and apologies. Grant Fitzner Make introductions if necessary.

Inform panel members of any apologies.

Update on outstanding actions.

10:45 Terms of reference for the APCP-T Liam Greenhough
APCP-T(22)07 Terms of
reference for the APCP-T
Publication status: draft
for future publication
The terms of reference for the
operation of the Advisory Panel
on Consumer Prices – Technical
(APCP-T) are presented for
review. Changes have been
suggested to align more closely
with the terms of reference for
the APCP-Stakeholder.
11:05 Multilateral index
methods: introducing
the GEKS-Törnqvist
Liam Greenhough
APCP-T(22)09 Multilateral
index methods:
introducing the GEKSTörnqvist
Publication status: draft
for future publication
ONS are developing materials to
provide step-by-step guidance
on using the GEKS-Törnqvist and
splicing techniques to improve
the accessibility of these
methods to wider audiences,
which we intend to publish later
in the year. We have developed
an early draft of this work and
are looking for the panel to
provide feedback (either at the
panel or in writing) on what
they think works well or less
well in the materials, and to also
provide quality assurance of
what has been written.
11:25 Research indices using
scanner data: progress
report
Jo Corless
APCP-T(22)10 Research
indices using scanner
data
Publication status:
SENSITIVE - not for
publication
An update on the latest
research undertaken into the
use of scanner data for
calculating inflation of groceries
in our consumer price statistics.
Includes research questions
around the methods for
calculating average prices across
outlets.
12:35 Publication status of
papers
Grant Fitzner
Verbal update
Publication status of papers
12:40 AOB Grant Fitzner Next meeting 8 July 2022
12:45 Meeting close

Download agenda

Meeting of 8 April 2022

Time Item Presenter and Paper Description
10:25 Join call
10:30 Introductions and apologies. Grant Fitzner Make introductions if necessary.

Inform panel members of any apologies.

Update on outstanding actions.

10:40 Finalising a multilateral
index method for using
scanner data to its
potential
Liam Greenhough
Presentation
ONS lays out the groundwork
for finalising its decision on
which index methodology to use
with scanner data following an external review of its index methods, in preparation for use in its upcoming alternative data
sources impact analyses
12:10 Update on Prices work to support
understanding of the impact of inflation on
UK households
Andy King
Presentation
An overview of the work being done in response to the increased focus on the impact of
high inflation on UK households, including the ONS’s
development of a personal inflation calculator, and a ‘least
cost’ index.
12:30 Future plans for
population subgroup
indices
Natalie Jones
Verbal update
An opportunity to provide
suggestions on the next stages
of our continued development
of population subgroup indices.
12:55 AOB and date of next meeting Grant Fitzner Outcome of the Council Tax rebate paper

Publication status of papers

Proposed joint meeting with APCP-S

Next meeting 8 July 2022

13:00 Meeting close

Download agenda

Meeting of 14 January 2022

Time Item Presenter and Paper Description
10:25 Join call
10:30 Introductions and apologies. Grant Fitzner Make introductions if necessary.

Inform panel members of any apologies.

10:40 Transformation of UK Consumer Price Statistics Andy King

APCP-T(22)01 Transformation of UK consumer price indices: Impact Analysis, 2022

An outline of the updated consumer price transformation timeline and a summary of the impact analysis of the inclusion of Alternative Data Sources in the measures of consumer
price inflation
11:10 New Data Sources for UK Consumer Price Indices (1) Joe Barker

APCP-T(22)02 Transformation of UK consumer price indices for rail fares

Using new transaction level data to produce price indices for rail fares, and the impact of these on UK consumer price statistics.
12:00 New Data Sources for UK Consumer Price Indices (2) David Moran

APCP-T(22)03 Transformation of consumer price indices for second-hand cars

Proposed methods for the calculation of second-hand car indices in the UK using online car listings data
12:50 AOB and date of next meeting Grant Fitzner Publication status of presented papers.
13:00 Meeting close

Download agenda

Paper APCP-T(22)01 has not been published at this time. For more information please refer to the published minutes.

Transformation of UK consumer price indices: rail fares

Transformation of UK consumer price indices: second-hand cars

Meeting of 8 October 2021

Time Item Presenter and Paper Description
10:25 Join call
10:30 Introductions and
apologies
Mike Hardie Make introductions if necessary.
Inform panel members of any
apologies.
10:35 New index number
methods in Consumer
Price Statistics
Kevin Fox & Peter Levell
Presentation
ONS have developed a quality framework to assess index number methods against, to ensure any methods of calculating price indices are appropriate for the data source,
as part of introducing new alternative data sources in consumer price statistics. Through the Economic
Statistics Centre of Excellence, the ONS has commissioned a review of the framework, which is being led by Kevin Fox and Peter Level. Panel members are asked to feedback on the findings of the review.
11:35 Rents Development Natalie Jones
APCP-T(21)14
Redeveloping private rental
market statistics
We present our proposed
method to measuring rental
price statistics. The aim of the
session is for the Panel to advise
ONS on the suitability of using its
preferred method for live
statistical production, and
provide any feedback only if
there are fundamental problems
with the proposed method.
11:55 Weights 2022 Chris Payne
APCP-T(21)15 Weights
update, 2022
This paper proposes an approach
for calculation of weights for the
Consumer Prices Index including
owner occupiers’ Housing costs
and the Consumer Prices Index
for 2022, taking into account
international guidelines, data
issues and the relevance of our
consumer prices weights
12:10 HCIs Mortgage Interest Domenica Rasulo and Yasin
Auckbur
APCP-T(21)16 Measuring
mortgage interest
payments in the Household
Costs Indices
We present an update to our
research on the methodology for
mortgage interest payments. We
consider three approaches:
1. The current MIPs method used
in the RPI
2. Simple revaluation
3. Lenders' formula method
12:55 AOB and date of next meeting Mike Hardie Publication status of presented
papers.
13:00 Meeting close

Download agenda

Item 2 is a draft of a paper that will be published at a future date on the ESCoE website

Item 3 has not been published at this time, for more information please refer to the published minutes.

Mortgage interest payments

Weights update 2022

Meeting of 9 July 2021

Time Item Presenter and Paper Description
10:25 Join call
10:30 Introductions and
apologies
Grant Fitzner Make introductions if necessary.
Inform panel members of any apologies.
10:40 Update on Rental Indices Development Chris Jenkins Verbal update
10.55 RPI Revisions Policy Mike Hardie Presentation of the proposed changes to the RPI Revisions Policy followed by discussion.
11:25 Alternative data sources in the CPI and CPIH. Helen Sands
Integration of scanner and web-scraped data into consumer price statistics: aggregation and weights
Proposals and remaining challenges for the integration of scanner and web-scraped data into consumer price statistics, with a particular focus on aggregation of different data sources and their respective weights, as well as imputation for missing strata and consumption segments.
12:50 Publication status of presented papers. Grant Fitzner Confirmation of the publication status of the papers presented at the session.
12:55 AOB and date of next meeting Grant Fitzner Next meeting:
8th October 2021
13:00 Meeting close

Download agenda

Meeting of 16 April 2021

Time Item Presenter and Paper Description
10:25 Join call
10:30 Introductions and
apologies
Grant Fitzner Make introductions if necessary.
Inform panel members of any apologies.
10:40 Progress update on private rental development Mike Hardie Verbal update
10.50 Household Costs Indices: mortgage interest payments. Domenica Rasulo
APCP-T(21)07 Lenders’ formula method for mortgage interest repayments
APCP-T(21)08 Current mortgage interest payments methodology: a review
These papers present work to review the method used to calculate mortgage interest payments in the current Household Costs Indices estimates. The work focusses on two workstreams: the development of a new methodology which incorporates the lenders’ formulae used to derive households’ interest payments, and a sensitivity analysis of the parameters used in the current model.
11:25 Web scraping: Product grouping methods. Hazel Martindale & Matt Eddolls
APCP-T(21)09 Dealing with product churn in web-scraped clothing data: product grouping methods
This paper discusses the methods and metrics under consideration for forming homogenous groups of products in web-scraped data, to reduce the effect of product churn. Two different methods of forming groups are presented along with discussion of how to assess the group properties. This paper
discusses the options currently under consideration and our plans for further work.
12:00 Break
12:05 Outlier detection and filtering methods for UK web-scraped and scanner data Loes Charlton
Presentation
This presentation reviews potential methods for anomaly detection to apply to scanner and web-scraped CPI data. Following a literature review, the suitability of short-listed methods is assessed by comparing their computation cost, the number and distribution of outliers they produce and their effect on price indices.
12:40 Web scraping: Expenditure proxies Alex Rose & Matt Love
APCP-T(21)10 Approximating Sales Quantities for Web Scraped UK Grocery Data
This paper assesses methods for producing sales quantity approximates for products at the elementary aggregate level. The considered methods focus on using product characteristics, such as price and page location, collected from web scraped data, along with item level sales quantities, provided by the retailer.
13:15 Transparency: review of paper publication classifications Grant Fitzner A review of the current paper classifications used to determine publication status.
13:20 Revisions policy Mike Hardie Verbal update
13:25 AOB and date of next meeting Grant Fitzner Next meeting:
9th July 2021
13:30 Meeting close

Download agenda

Meeting of 15 January 2021

Time Item Presenter and Paper Description
10:25 Join call
10:30 Introductions and
apologies
Grant Fitzner Make introductions if necessary.
Inform panel members of any apologies.
10:35 Update on price collection Mike Hardie Trends in unavailable items and the expected impact on local collection of the new lockdown.
10.45 Rental Indices Development Plans and Progress Chris Jenkins / Natalie Jones An overview of our plan and progress of developing our private rental price statistics.
11:10 Weekly online price changes for food and drink items methodology Helen Sands
APCP-T(21)04 Weekly online price changes for food and drink items methodology
Challenges presented by producing weekly price indices, as well as our plans for improving the indices in the coming months
11:35 Research indices using scanner data Jo Corless
APCP-T(21)01 Research indices using scanner data
The latest research undertaken into the use of scanner data in our consumer price indices
12:00 Refunds in scanner data George Bettsworth
APCP-T(21)02 Refunds in scanner data
Preliminary advice on how to consider refunds in scanner data
12:25 Classification of Alternative Data Sources Liam Greenhough
APCP-T(21)03 Classification of Alternative Data Sources
An update on clothing classification and research for grocery classification
12:50 AOB and date of next meeting Grant Fitzner Next meeting:
16 April 2021
13:00 Meeting close

Download agenda

Download minutes

The January APCP-T meeting was held over two dates to allow further work and discussion of the item on rental indices development plans and progress. Items 3 and 4 were discussed on 15th January and 19th February, and the remaining items were discussed on 15th January.

Part 1:

  • 15th January 2021
  • Teleconference
  • 10:30 – 12:00

Part 2:

  • 19th February 2021
  • Teleconference
  • 13.00 – 14.00

Members in attendance

  • Mr Grant Fitzner (ONS, chair)
  • Mr Mike Hardie (ONS)
  • Mr John Astin
  • Mr Peter Levell
  • Dr Jens Mehrhoff
  • Prof. Paul Smith
  • Dr Martin Weale
  • Dr Antonio Chessa

Secretariat

  • Mr Huw Pierce (ONS)
  • Mr Chris Payne (ONS)

Presenters

  • Ms Aimee North (ONS)
  • Mr Chris Jenkins (ONS)
  • Ms Natalie Jones (ONS)
  • Mr Liam Greenhough (ONS)
  • Ms Jo Corless (ONS)
  • Ms Helen Sands (ONS)
  • Mr George Bettsworth (ONS)

Observers

  • Ms Sofia Poni (ONS)
  • Ms Tanya Flower (ONS)

Apologies

  • Dr Gareth Clews (Methodology, ONS)
  • Mr Rupert de Vincent-Humphreys
  • Prof. Bert Balk
  • Prof. Ian Crawford
  • Mr Jonathan Athow (ONS)
  1. Introduction and apologies
    • Mr Fitzner opened the meeting and passed on apologies from members unable to attend.
    • Two actions were carried forward from the previous meeting. Establishing whether rateable values were used in the CPIH historic series remains ongoing. This will feed into the second action carried forward: the methodological paper covering the data and modelling used for the CPIH historic series is expected to be published in Q1 2021.
  2. Update on price collection
    • Mr Hardie gave an update on changes to price collection due to the current level of COVID restrictions. Since November 2020, price collection reverted to online only with price collectors working from home. This will also be the case in the base period, January 2021, and will be reviewed on a monthly basis. The list of unavailable items is reviewed monthly and is referred to in the key points of the CPI publication.
    • An article explaining changes to price collection procedures and the considerations for calculating the 2021 weights, discussed in the December 2020 APCP-T meeting, is being drafted by Mr Payne for publication on 11th
  3. Rental Indices Development Plans and Progress (15th January)
    • Ms North presented on developments to private rental price statistics, covering proposed methodological changes and a review of preliminary results. The new methodology utilises Valuation Office Agency transaction level data allowing for more granular outputs by region, property type and bedroom category. A hedonic regression model is used to predict prices for rental properties based on a selection of property characteristics. Two forms of regression model are under consideration: a general linear model (GLM) with and without interaction terms, and a random forest.
    • Mr Fitzner highlighted that an update to the private rental prices development plan would be published on the 19th January 2021.
    • Dr Chessa noted from his own experience of hedonic modelling the importance of accounting for variable interactions and was pleased to see this was a consideration in the current work.
    • Dr Weale suggested that some of the anomalous results may be a result of overfitting the model and that removing some of the independent variables might improve performance. He enquired about the functional form of the model, as there were additional considerations for converting a log normal distribution back to a price. Finally, he queried the sample sizes at the most granular level of analysis, as certain local authorities may have very small numbers of certain property types. Ms North acknowledged this concern and described ongoing analysis to determine appropriate combinations of geography and property characteristics that would allow robust estimates.
    • Dr Mehrhoff highlighted the paradigm change involved in using a machine learning approach (i.e. random forest) for hedonic modelling, asking if it was sufficient for the model to make reliable predictions or if there was also a requirement for the model to be explainable in economic terms. He also highlighted that random forest models perform poorly with time dummies as the fitting process generally discards them, and that this can make the models unstable. He suggested using a shrinkage model (such as lasso) with explicit interactions as a compromise between the control of GLM and the flexibility of random forest. Mr Levell noted that random forest models have an inherent opacity that makes comparisons difficult.
    • Prof Smith queried the choice of 14 months as the rolling period for the dataset, when 10 months is the average contract length. Ms North explained that 10 months would be ideal, however there are two practical limitations. The first is that relatively few contracts are exactly 10 months long, the average being a balance between six or 12 months which are more common. The second is that data is collected by rent officers who are only prompted to visit properties annually. With a ten-month window this leads to far fewer data points and therefore a more volatile index.
    • Mr Levell highlighted that changes in the composition of the rental market in recent years makes the hedonic adjustment especially important. The observation that the random forest model gives a different index value to GLM may reveal something about how rental patterns at the local authority level are treated. Comparing GLM with other models that explicitly incorporate local authority interactions could illuminate this question.
    • Several panel members were interested in more details of the model fitting and asked if a technical paper was in preparation. Mr Jenkins confirmed that there is.

ACTION 1: Ms Jones to provide more detail on the model specifications

ACTION 2: Ms Jones to share a draft of the technical paper with Panel members

  1. Update on Private Rental Statistics Development (19th February)
    • Mr Fitzner emphasised the experimental nature of the results being presented and the rigorous quality assessment they will undergo before being introduced to the headline indices.
    • Ms North presented an update on progress since the APCP-T meeting on January 15th. Following feedback from that session ONS have tested four further models in addition to the two models presented in January. These are: weighted least squares (WLS) with the interactions that could be implemented and with and without shrinkage applied, WLS using longitude and latitude so that all interactions could be used, and a random forest model with constraints on tree size (“pruning”).
    • Three of these models have already been discarded by ONS. WLS using longitude and latitude with interactions modelled geographically adjacent local authorities in a similar way, which was felt to be unrealistic. The random forest model using longitude and latitude without pruning applied produced unacceptably volatile results. The WLS model using local authorities with the interactions that were able to be applied and shrinkage applied gave almost identical results to the same model without shrinkage, however it does not provide the statistical metrics (e.g.: R2, mean squared error) required to produce observation weights.
    • The three remaining models for consideration are: WLS with no interactions (hereafter “model 1”), a pruned random forest (“model 2”), and a WLS model with selected interaction terms (“model 3”). The model preferred by ONS is WLS with no interactions, as it is more transparent and easier to quality assure than a random forest and the results from WLS with interactions diverge markedly from the other two.
    • Mr Fitzner queried an anomaly in the chart of England annual percent growth. Ms North confirmed that the upward trend at the end of the WLS with interactions series (contrasting with the downward trend of the other two models) was most likely explained by the interactions in that model.
    • Mr de Vincent-Humphreys noted the consistency between models 1 and 2 and asked if this was reflected at smaller geographies and when comparing rental levels instead of index values. Ms North confirmed that all three models were in close agreement at lower geographies, with models 1 and 3 being almost identical. In respect of rental prices, models 1 and 3 were in close agreement while information on model 2 was not available at the meeting.

ACTION 3: Ms North to provide details of the price level estimates for all models at the regional level.

  • Dr Weale commented that the models predicted a geometric mean of rents (as expected from a logarithmic model) and queried if this was consistent with other ONS price statistics. Ms North confirmed it was consistent with the HPI, had been peer reviewed internally and is consistent with international best practice for housing market indices.
  • Crawford asked if confidence intervals could be provided for the models. Ms North confirmed that these were available for the WLS models. Prof Smith recommended bootstrapping as a way of establishing confidence intervals for the random forest model.

ACTION 4: Ms North to provide details of the confidence level estimates for the WLS models.

  • Prof Smith suggested using cross-validation and analysing prediction errors to give a more objective basis for recommending one model over the others. Prof Smith also raised concerns that the similarity between models 1 and 2 may arise because the pruning mechanism in model 2 has removed all the interactions. Mr Fitzner requested a sensitivity analysis be performed on the models to address these concerns.

ACTION 5: Ms North to perform the sensitivity analysis on the models and share the results with the panel.

  • Mr Levell asked for greater detail on which interactions were driving the differences between the two WLS models and proposed progressively adding interaction terms to understand the sources of difference. Mr Levell also suggested that controlling for the socioeconomic status of an area (via the ACORN variable) might suppress some of the modelled rent increase.

ACTION 6: Ms North to investigate the use of ACORN in relation to endogeneity

  • Mr de Vincent-Humphreys enquired about the likelihood of revising the back series of CPI and CPIH in light of these new methods, whether the methods were consistent with HPI in its use of hedonic regression, and for confirmation of the timeline for introducing these new methods into production. Mr Hardie confirmed that further analysis will be carried out during 2021, with a parallel run phase under consideration. No revisions will be made to historic CPI and CPIH values, consistent with ONS policy. Ms North described the parallels and divergences between the HPI and Rents Development processes.
  • Balk averred that the opacity of the random forest model was disqualifying, and without understanding the economic meaning of the interaction terms their use was unsafe. This leaves model 1 as the preferred option.
  • Dr Mehrhoff framed the decision as a balance between the better predictions of a random forest vs the transparency of the WLS model, but only if the random forest’s predictions really were better as measured by mean squared errors or R2. The preferred model (model 1) sits well with international recommendations and practices.
  1. Weekly online price changes for food and drink items methodology
    • Ms Sands described the process for compiling the weekly price indices that have been produced during the COVID-19 pandemic, highlighting that this is the first time alternative data sources have been used in a production situation. As many of the decisions taken in the development process were taken at short notice this was an opportunity to review these choices and consider how to take the indices forward. Areas of focus include product identification, outlier detection, and choice of index formula (currently GEKS-J with a five-week movement splice).
    • Dr Weale queried why the weekly index was using different samples of products to CPI. Ms Sands confirmed that this was partly due to the difficulties in product identification given that local collectors do not collect product IDs, and partly to leverage the greater coverage that the web dataset offers.
    • Dr Chessa complimented the work of ONS on alternative data sources and highlighted his own research on the topics discussed in the paper, offering to circulate a report on this to panel members. His analysis showed that indices with movement splices are prone to drift and that short windows can create problems with seasonal products. A potential solution could be to use an expanding window instead. Dr Chessa described another approach using a 25-month window with the lack of back data being addressed through imputation, however it was not appropriate for this situation. Dr Mehrhoff observed that 25-months was the emerging international consensus for treating seasonal products. There is a risk however, that if price trends within the basket diverge then the characteristicity of the index may be compromised.

ACTION 7: Dr. Chessa to provide a copy of his multilateral indices research report for circulation to panel members.

  • Ms Sands described the level of interest from other government departments in the weekly statistics. Mr Fitzner said it was likely that the weekly index will continue in some form after the COVID-19 pandemic due to this level of interest. Further analysis and stakeholder engagement are required to understand how the index can provide the most value in the longer term.
  • Dr Mehrhoff raised the opportunities that big data provided for identifying emerging trends and enquired about a comparison with conventional headline indices. If the weekly index is to become a permanent publication, then users will require greater detail on its methodology. Ms Sands offered to circulate comparisons with CPIH to panel members.

ACTION 8: Ms Sands to provide comparisons of the weekly index with CPIH.

  • Mr Astin praised the volume and quality of work by ONS on alternative data sources and enquired if these sources could lead to the end of in-person price collecting. Ms Sands averred that while meaningful volumes of expenditure continued to take place in independent shops there would be a requirement for local collection. Mr Hardie added that the RPI requires local collection, and therefore it would be required to continue until at least 2030. Beyond that it may be possible to refocus local collection efforts away from supermarkets and more towards independent stores, perhaps in support of regional price statistics.
  • Prof Smith expressed interest in the distributions of price levels, suggesting that if the distribution was not normal then using a multiple of standard deviation as an outlier detection method was unsuitable. He also questioned the implicit weighting towards stores with larger product ranges. While this might be appropriate for supermarkets vs independent retailers in groceries, in other markets smaller specialist retailers may have larger ranges than larger, more general outlets. Ms Sands confirmed that the price distributions were not normal and improved outlier detection methods were an ongoing topic of research. The choice of product range as a basis for weighting was made due to the difficulties encountered using market share directly. For the retailers being used currently analysis indicates that product range mirrors market share fairly well, however if the number of retailers expands this may need to be revisited.
  1. Research indices using scanner data
    • Ms Corless presented findings from research undertaken using scanner data. Retailers have provided data covering a range of goods, including groceries, homeware, clothing, electronics and fuel. Advice was sought from the panel on three topics:
      • Time coverage. Scanner data is provided by retailers aggregated by week; however, Consumer Price Statistics require prices to be attributable to a single month and some weeks span month boundaries. Proposed solutions are to exclude weeks that straddle months or only use the first three weeks of the month (to align with production timescales), but there is concern that this could introduce bias.
      • Product relaunches. Products are sometimes relaunched with different identification codes creating difficulty in maintaining continuity of data. An approach based on text matching product descriptions is under development, with an intention to test this using a dataset with linked product IDs. However, other NSIs apply alternative approaches such as expenditure monitoring to identify relaunched products.
      • Handling discounts: Discounts can take several forms, requiring a decision on which types to include and exclude from the index. Simple price reductions are straightforward to capture, but multibuy discounts and discriminatory discounts (e.g. loyalty cards) are more challenging. There is also a consideration on whether consistency is required between scanner data and local collection.
    • On time coverage, Prof Smith suggested constructing a weekly unit value index (UVI) and modelling values for those weeks that are split between months. The weekly index can then be averaged over a calendar month. Alternatively, it may be possible for index construction purposes to redefine a month as a collection of weekly periods. On handling discounts, Prof Smith expressed a preference for capturing as much information as possible, noting that significant proportions of expenditure are covered by loyalty card discounts. One way of addressing this is to treat items bought with and without a loyalty card as separate products.
    • Mr Astin asked if concerns over introducing bias by discarding weeks could be addressed by testing. Ms Corless agreed that it could. Dr Weale postulated that systematic bias would only arise from discarding weeks if retailers were engaging in behaviour that he found unlikely. While anomalies may occur in individual months, over the long term these were likely to cancel out.
    • Dr Chessa shared his experience of testing for bias when choosing which weeks to include. For supermarket and fuel data large improvements in accuracy were found when adding second and third weeks to the calculations. In the case of air fares adding a fourth week was necessary for months with moveable holidays (e.g. Easter).
    • Dr Mehrhoff offered to share the results of his own testing surrounding the time coverage question and encouraged ONS to carry out their own tests. He then described the difficulties that Eurostat encountered using text matching to identify product launches in the absence of other metadata. One approach to mitigating the effect of product relaunches is to use broader product categories. This introduces a risk of unit value bias but give the benefit of greater continuity. For discounts and particularly loyalty cards, Dr Mehrhoff advocated considering time consistency in the index when deciding what to include.

ACTION 9: Dr Mehrhoff to provide details of his scanner data research.

  1. Refunds in scanner data
    • Mr Bettsworth described research into the treatment of refunds in scanner data. Refunds are recorded differently in the datasets provided by different retailers; some log them as individual items while others include them in aggregate statistics. Other NSIs do not generally regard refunds as a problem, especially not for perishable items, although many had not fully considered the issue.
    • Where refunds are separately logged it is possible to perform analysis on their frequency, and more importantly remove them when obtaining price information. Refunds are far more common in some product categories (e.g.: clothing, jewellery) than others (e.g.: food). Where refunds are included in aggregate figures there are editing techniques that can be applied to extract price information. Current areas of investigation are whether refunds affect the final index and the consequences of choosing one editing method over another.
    • Mr Astin enquired about the treatment of vouchers, for example discounts offered above an overall spending threshold. Ms Sands confirmed that discounts that were applied to an entire basket rather than a single product are not usefully reported in the datasets. Dr Mehrhoff shared his experience that voucher transactions were recorded as positive quantities and negative expenditures and were likely to be removed by data cleaning processes. Responding to Prof. Smith, Mr Bettsworth confirmed that refunded quantities are reported as negative amounts in the datasets. Prof. Smith asked if this mirrors the treatment of refunds in the Living Costs and Food survey, and how refunds are treated in production currently.
    • Dr Mehrhoff outlined concerns about timing the processing of refunds. If a purchase is included in an index calculation in one month but refunded after publication, this is difficult to handle in an environment where indices are not revised. The problem is complicated further when using multilateral indices with splices, and there is no consensus on the appropriate way to handle this. Dr Mehrhoff suggested running case studies on various refund scenarios to understand the possible outcomes, and Dr Weale concurred. Mr Bettsworth highlighted the difficulties of linking refunds to the original sale, noting that other NSIs had also encountered this problem. Mr Levell suggested using information from the retailers that provide disaggregated refund data to model the volume of refunds across all datasets.
    • Dr Chessa described findings from his own research, confirming that some categories experience a high enough volume of refunds to distort prices. He suggested that if price lists were available it may be possible to build a model that would allow adjustments to be made to the dataset to compensate for refunds.
  2. Classification of Alternative Data Sources
    • Mr Greenhough gave a presentation summarising progress on classifying clothing and groceries in scanner data and outlining further plans for development.
    • Good progress has been made applying a machine learning approach to clothing, with focus turning to achieving performance gains, understanding the level of bias due to misclassification and preparing the pipeline for production use.
    • A machine learning approach is thought to be less suitable for groceries however, as products are more likely to have explicit identifiers that can be used instead. Moreover, smaller training datasets and the more granular COICOP structure for groceries increases the risk of misclassification, and the consequences of misclassification for the overall index are more severe for some products due to the higher weight that some groceries carry. Mr Greenhough proposed potentially using machine learning as a method of machine-assistance for improving the efficiency of manual classification, with products still receiving manual scrutiny as assurance. Also potentially exploring whether very low-expenditure products can be excluded without impacting elementary aggregates.
    • Dr Mehrhoff asked if further information could be provided on the precision/recall performance of the classifiers. A scatter plot of product category weight and precision/recall would indicate if the most important categories for the index were achieving sufficient predictive success. Mr Greenhough confirmed these could be made available.

ACTION 10: Mr Greenhough to provide precision/recall scatter plots.

  • Dr Mehrhoff noted from his experience firstly that manually reviewing higher weight categories is often valuable, and secondly that classification performance is usually more dependent on the quality of input data than on the choice of classification algorithm.
  • Prof Smith expressed concern over the risk of propagating incorrect classifications and suggested carrying out further research into the topic.
  1. AOB and date of next meeting

 

No. Action Person Responsible
1 Provide more details on the private rentals model specification. Ms Jones
2 Provide a draft of the private rentals technical paper for circulation to panel members. Ms Jones
3 Provide details of the price level estimates for all models at the regional level. Ms North
4 Provide details of the confidence level estimates for the WLS models. Ms North
5 Perform the sensitivity analysis on the models and share the results with the panel. Ms North
6 Investigate the use of ACORN in relation to endogeneity Ms North
7 Provide a copy Dr Chessa’s multilateral indices research report for circulation to panel members. Dr Chessa
8 Provide comparisons of the weekly index with CPIH. Ms Sands
9 Provide details of Dr Mehrhoff’s scanner data research. Dr Mehrhoff
10 Provide precision/recall scatter plots for the product classifiers. Mr Greenhough

Meeting of 18 December 2020

Time Item Presenter and Paper Description
10:25 Join call
10:30 Introductions and
apologies
Mike Hardie Make introductions if necessary.
Inform panel members of any apologies.
10:35 Approach to Consumer Price Indices Weights 2021 Chris Payne Presentation on the proposed approach to calculating CPI and CPIH weights for 2021
10:50 Round table discussion Panel members Comments and questions raised by the presentation.
12:00 Meeting close

Download agenda

18th December 2020 – Teleconference -11:00 – 12:00

Members in attendance

  • Mr John Astin
  • Mr Mike Hardie (ONS, chair)
  • Mr Peter Levell
  • Dr Jens Mehrhoff
  • Prof. Paul Smith
  • Dr Martin Weale
  • Dr Gareth Clews (Methodology, ONS)
  • Prof. Ian Crawford
  • Mr Rupert de Vincent-Humphreys

Secretariat

  • Mr Huw Pierce (ONS)
  • Mr Chris Payne (ONS)

Apologies

  • Dr Antonio Chessa
  • Prof. Bert Balk
  • Mr Grant Fitzner (ONS)

1. Introduction and apologies

  • 1.1 Mr Hardie welcomed attendees to the Advisory Panel on Consumer Prices-Technical (APCPT) meeting and introduced Mr Pierce who is assuming the secretary role going forward.
  • 1.2. As an interim meeting there were no previous actions to discuss.

2. Approach to Consumer Price Indices Weights 2021

  • 2.1. Mr Payne gave a presentation on the proposed approach to calculating Consumer Prices Index (CPI) and Consumer Prices Index including Owner Occupiers’ Housing costs (CPIH) weights for 2021 and summary analysis of the calculated weights. Panel members were invited to comment on 1) the process for arriving at the weights and 2) the indicative weights themselves.
  • 2.2. The proposed approach adopted for Retail Prices Index (RPI) weights in 2021 is to use Living
    Costs and Food Survey (LCF) data for the period from July 2019 to June 2020. For CPI and CPIH the proposed approach is to align with international statistical guidance, along the lines set out in recent Eurostat guidance. This has the benefit of retaining international comparability and best practice.
  • 2.3. Adjustments will be limited to categories that are most heavily affected and/or show sustained changes in spending.
  • 2.4. Estimates of the new weights were presented, acknowledging some limitations.
  • 2.5. The new 2021 weights were compared with weights calculated under the conventional process (using 2019 spending data instead of 2020) and the weights used in 2020. While most classes experienced relatively small changes due to the altered process, there were some that experienced larger movements.

3. Round-table discussion

  • 3.1. Panel members were invited in turn to offer feedback and raise questions on the presentation.
  • 3.2. Dr. Mehrhoff commented on the general uncertainty surrounding 2021 and how reflective our estimates would be (as a result of Eurostat guidelines) should circumstances change. Weights should account for 2020, but also anticipate behaviour in 2021 dependent on how far the economy reverts to normal. Dr. Mehrhoff and Prof. Crawford were supportive of the idea of only updating categories that showed a sustained change in consumer behaviour.
  • 3.3. In response to questions from Mr Astin, Mr Payne confirmed that the procedure for updating the RPI weights uses LCF from July 2019 to June 2020 and therefore follows a different schedule to CPI and CPIH, such that the impact of changes to spending patterns is less and therefore there will be no changes to the RPI process. Mr Astin also asked about what the ideal index would be in these circumstances. Conventional approaches to building price indices have little to say about the current situation we face and therefore it is difficult to make a case for any particular index to be the ideal one for benchmarking.
  • 3.4. Mr Hardie emphasised the value users place on continuity across time and international comparability in price indices. For these reasons it made sense to follow Eurostat guidance. Mr. de Vincent Humphreys raised the importance of how we explain our approach to users and the public.
  • 3.5. Prof. Smith and Mr Levell both raised the topic of smoothing the weights, acknowledging the complexity involved in doing this. Mr Levell asked if there was a statistical test being applied as to what constituted a sustained change in spending behaviour, as this would act as a threshold for identifying weights that need adjustment. Mr Payne referred to work done by the Australian Bureau of Statistics on this point and suggested that ONS may follow a similar approach to them.
  • 3.6. Dr. Weale suggested thinking about what price change we wanted to show and using that to guide the approach. If the change between 2021 and 2020 is the key comparison then the approach is probably the right one.
  • 3.7. Panel members queried the process for creating weights for 2022 and the “exit strategy” for this interim process, i.e. how we return to normal procedures after the pandemic.
  • 3.8. Following a comment on the European situation, a panel member asked if the USA or Canada had declared their approach to calculating 2021 weights. Mr Payne confirmed he would be discussing with both countries.
    • Action: Mr Payne to report back on the approach taken by the USA and Canada

  • 3.9. Panel members gave broad approval of the approach taken by ONS and recognised the  complex nature of the problem at hand.

Actions

  1. Update the group on the approach taken by the USA and Canada – Mr Payne
  2. Provide the presentation slide pack to Bert Balk Mr Payne

Download minutes


Meeting of 9 October 2020

Agenda

Minutes

Draft consumer prices development plan 2021*

CPIH: Producing a historical series for the period 1950 to 1987*

On Household Costs Indices*

*This paper has not been published at this time, for more information please refer to the published minutes.

Meeting of 10 July 2020

Agenda

Minutes


Meeting of 17 April 2020

Agenda

Minutes

Contingency plan for ongoing price collection and compilation during the Covid-19 pandemic*

CPIH: Producing a historical series for the period 1947 to 1987**

Progress on automated classification of web-scraped clothing data**

Estimation and presentation of information on the accuracy of the Consumer Price Index**

*This is a draft of an article that has now been published and is available on the ONS website.

**This paper has not been published at this time, for more information please refer to the published minutes.


Meeting of 17 January 2020

Agenda

Minutes

Terms of reference*

Refining the higher education component of the Household Costs Indices

Expanding the measurement of interest payments on financial debt**

The winning formula? A framework for choosing an appropriate index method for use on web scraped and scanner data

* This paper is a draft update of the Terms of Reference, which has now been published on the UK Statistics Authority website.

** This paper is work in progress and has not been published at this time


Meeting of 8 November 2019

Agenda

Minutes

Proposed text for the consultation on the future of the Retail Prices Index*

* This paper is a partial draft of the consultation which has now been published on Citizen Space

Meeting of 6 September 2019

Agenda

Minutes

Alternative methods and data sources for package holidays – update*

Price collection in a no-deal scenario*

Updating the Data Source for Shop Type Weights*

Collecting Discounted Prices

Discussion on unweighted index number formulae**

* This paper has not been published at this time, for more information please refer to the published minutes.

** This paper is a draft and has not been published at this time.


Meeting of 17 May 2019

Agenda

Minutes

The conceptual foundations of the Household Costs Indices (ESCOE workshop)

The conceptual foundations of the Household Costs Indices: Annex A

Alternative methods and data sources for package holidays*

Shortlisting Appropriate Index Methods

Classification metrics guidelines

* This paper has not been published at this time, for more information please refer to the published minutes.


Meeting of 11 January 2019

Agenda

Minutes

The use of unweighted indices in ONS’s consumer price statistics

Assessment of small area estimation for regional CPI expenditure weights*

The use of approximate expenditure weights for web scraped data

Quality adjustment review

Note from Jens Mehrhoff in response to APCP-T(19)01: The use of unweighted indices in ONS’ consumer price statistics

* This paper is a draft of an article that has now been published and is available on the ONS website.


Meeting of 14 December 2018

Agenda

Minutes

Calculating a price index for capital mortgage repayments*

Calculating a price index for student loan repayments*

* This paper has not been published at this time, for more information please refer to the published minutes.


Meeting of 7 September 2018

Agenda

Minutes*

Updating the data source for shop-type weights*

Variance estimates for the Consumer Prices Index including owner occupiers’ housing costs*

Proposed pipeline for processing alternative data sources

The use approximate expenditure weights for web scraped data in consumer price indices*

* Update 04/02/19: Minutes updated to correct a minor error.


Meeting of 11 May 2018

Agenda

Minutes

Calculating a price index for student loan repayments*

Draft work programme for consumer price statistics*

Investigating the use of web scraped data to improve clothing measurement*

Extending the classification structure

* This paper has not been published at this time, for more information please refer to the published minutes.


Meeting of 15 January 2018

Agenda

Minutes

Producing standard errors for CPIH*

Feasibility study into producing CPIH-consistent inflation rates for UK regions

Producing a historical series for CPIH*

Measuring changes in used car prices

* This paper has not been published at this time, for more information please refer to the published minutes.


Meeting of 15 September 2017

Agenda

Minutes (updated February 2018)

A theoretical framework for the Household Costs Indices*

Review of quality adjustment in consumer price statistics

Assessing the feasibility of web scraped data within current collection methods

Smoothing volatile weights

AOB – What is the likely scale of the effect of using the current chain linking algorithm

* This paper has not been published at this time, for more information please refer to the published minutes.


Meeting of 19 May 2017

Agenda

Minutes

The measurement of real household incomes in the UK: Options for a matched approach*

Including the cost of finance in CPIH and HCIs

Inflation for household groups: calculation of weights

Consumer prices work programme*

AOB – Chain linking algorithm question

* This paper has not been published at this time, for more information please refer to the published minutes.


Meeting of 20 January 2017

Agenda

Minutes

Terms of Reference

RPI and CPI: a tale of two formulae

Re-addressing the formula effect

Review of web-scraped price indices

Additional correspondence – OOH weight for imputed rentals


Meeting of 1 September 2016

Agenda

Minutes – Amended 14 February 2017

Inclusion of Council Tax in CPIH


Meeting of 10 May 2016

Agenda

Minutes

Written Submission


Meeting of 22 January 2016

Agenda

Draft Terms of Reference

Minutes

Consumer Price Indices in the UK

Dr Courtney’s notes on the minutes