Mr Fitzner opened the meeting and passed on apologies from members unable to attend.
Mr Fitzner introduced Professor Killick, who has joined the panel as the representative from the Royal Statistical Society.
Ms Sands confirmed the position of any outstanding actions.
2. Second-hand Cars
Mr Greenhough introduced a paper describing two small potential sources of bias in the proposed method for calculating inflation in second-hand car prices using new data sources. The first of these is a small negative bias due to uncaptured age depreciation, and the second is a small positive bias due to uncaptured technological improvements. The panel were asked for input under four headings:
Whether the evidence in the paper pointed to the existence of these sources of bias.
If so, whether the proposed adjustments in the paper would provide adequate mitigation
Whether the unadjusted proposed method represented an improvement over the current method in the CPI and CPIH, such that there is benefit in introducing the new method into the indices while the adjustment approaches are refined.
Whether the panel recommends switching to a time-dummy hedonic (TDH) approach and if so, offer recommendations as to how this could be developed.
Panel members broadly agreed that more complex methods are only justified if the additional complexity adds value, and in this case more complex methods were not likely to be justified. The Panel members agreed that the unadjusted proposed method should be used in the first instance as it offers an improvement over the current method, and that work should continue as necessary to address the biases within this proposed method.
There was interest from the panel in seeing the model building process and examining metrics such as the Akaike Information Criterion (AIC) and Variance Inflation Factors to determine this. However, one panel member argued that the principal consideration was the accuracy of predictions rather than causal inference and therefore a machine learning approach may give better results.
The panel discussed the merits of applying a variance correction to the model and the interpretation of the variance as a quality characteristic.
Responding to a question about consumer preferences, Mr Greenhough noted that MARS analysis can highlight which variables in the model lead to price change. A panel member noted that the MARS approach is not universally accepted, and signposted an alternative method based on time-dummy adjustments within strata.
The panel discussed the problem of sparse data, as matched pairs approaches require items to be present in the base and current period. This is a trade-off between availability of data and homogeneity of strata. The panel expressed interest in seeing analysis of this.
3. Timelines for the Introduction of Alternative Data Sources
Ms Poni gave a verbal update on the timelines for introducing alternative data sources into the CPI and CPIH and that these updated timelines will be published on 31 May 2023.
The panel noted the updated timelines. One member recalled the experience of other NSIs having to revise published measures due to rushing the introduction of new data sources, noting this would be best avoided.
A panel member enquired if the additional work related to implementation and testing or whether there was further methodology work required. Ms Poni emphasised that the work principally related to systems, however noted that there were some limited areas of methodological work remaining.
Mr Fitzner added context to the update, referring to the interests of multiple stakeholders in the transformation programme, as well as regulatory requirements from OSR and surrounding impacts on the RPI.
Ms Poni outlined candidate areas of the basket for development after the delivery of groceries data, though these are not yet confirmed. Of greatest interest was energy prices due to their weight in the indices and the requirement for granularity. There are also several household goods included in current scanner data deliveries that do not fall within the groceries category that could represent a ‘quick win’ for inclusion in future.
4. Standard Errors
Professor Smith presented a paper outlining progress on estimating sampling variance in the CPIH, this work being requested by the Office for Statistics Regulation following the designation of CPIH as a National Statistic. The paper also contained a proposed work plan for further development. The panel were invited to comment under four headings:
The suitability of a jack-knife variance estimator for inclusion in the sample design.
A principled approach for assessing which representative items could be taken to be self-representing.
The implications of component variance estimation for the design of CPIH data collection.
The proposed direction of future work.
Panel members warmly received the paper, noting the complexity of the topic and the historic difficulty of the task. The jack-knife approach was regarded as a sensible approach to variance estimation, with the caveat that for items with low sample sizes the method may introduce bias.
When discussing representative items, panel members suggested choosing items based on year-round availability, market share or identifying items whose prices tend to move in parallel. The panel accepted that a measure of the variance within a representative item would still be of interest even if the items themselves were selected ideally.
The panel discussed the implications for the design of the CPIH sample and the use of a trimmed mean. Prof Smith cautioned that while this would most likely lower the variance it did not necessarily follow that it would lead to a better index.
Panel members were surprised at the relatively low variance shown in the clothing division. Prof. Smith agreed this warranted further examination.
Mr Corder asked whether users could be given any indicative results at this stage, or whether further work was needed. Prof Smith highlighted that a preliminary paper was presented at the ESCoE conference in 2022, so some results are already in the public domain. There is work outstanding to incorporate weights variance and determine the appropriate way to present the outcomes, but a public release should be achievable later in 2023.
5. House Price Index Replatforming
Ms North presented a paper outlining recommendations from ONS Methodology for improvements to the House Price Index (HPI), in the context of moving the calculation process from a legacy software platform to one aligned with wider ONS IT strategy. In brief, the recommendations were to use a simpler regression model, amend the floor area variable fed into the model, and adapt the approaches taken to outlier detection and imputation. The panel were invited to comment.
Using the log of floor area was preferred, subject to the distribution of the data. This would make the estimate an elasticity, showing the percentage change in house price associated with a percentage change in floor area.
A panel member endorsed using early outlier detection to remove gross outliers, but recommended retaining outlier detection during the regression phase.
The panel recommended seeking clarity on the purpose of the imputation, as this would guide the choice of a random or deterministic approach. It would also influence the decision of whether observation weights should be retained in the process.
Several panel members noted that the best choice may become clearer during the model building process and reserved judgement until further analysis had been presented.
The timeline for the replatforming is dependent on the delivery of rents transformation, however a further update on this methodological work is anticipated in Q3 2023.
6. Annual Report
Ms Sands presented a summary of the draft combined advisory panel annual report and its recommendations. The draft report will also be presented to the APCP-S before publication.
A panel member proposed re-emphasising the topic of non-hedonic quality adjustment in the future work plan.
7. Publication Status of Papers
The papers on second-hand cars and HPI replatforming will be published alongside the minutes. The paper on standard errors in CPIH remains a work in progress and will be published at a later date. The update to alternative data sources timelines will be published as an article on the ONS website.
8. Any other business and date of next meeting
The next meeting will be held on Friday 7th July 2023.
