Use of non-official sources for SDG reporting
- Data sources
In July 2021 we published a detailed outline of our approach to assessing the quality and suitability of statistics from non-official sources for SDG reporting. We produced a transparent protocol with a simple scoring system based on the pillars of the UKSA Code of Practice for Statistics and critical SDG characteristics. Putting this protocol in place was the result of collaborative work and feedback from across the GSS. It allows us to expand our breadth of sources beyond official statistics without compromising on quality. As a result, more vulnerable populations and environments could be highlighted as we fill outstanding headline and disaggregation gaps, making sure no-one and no-where is left behind.
In brief, the structure of the protocol includes an initial pass or fail gateway with the following essential criteria:
- ethics and privacy
- transparency and accountability
- need
If the source passes on all of these elements, it proceeds to be scored on a scale of 0 (not acceptable) to 3 (high score) on the criteria:
- relevance
- methods
- coverage
- timeliness and ongoing availability
- data quality
A statistical output from a non-official source with an average score of over 1.5, and no scores of 0, is considered appropriate for SDG reporting. Using this system, we can see where strengths and weaknesses lie for each potential data source.
The scoring criteria listed above represent the latest version as of March 2022. This version differs slightly from the one published in the original publication from July 2021. We have refined the original scoring system following practical use within the SDG team, and dissemination at various conferences and workshops. The change affects the second stage of the scoring, where we have removed the original criteria of “data journey awareness” and “quality assurance”. Instead, we merged these into a single dimension called “data quality”. This change fits better with the details available for most non-official statistical sources. It still captures important and critical aspects of data quality, such as reliability, validity and quality assurance.
Using this protocol, we have assessed the non-official sources already used in SDG reporting, and confirmed they are fit for purpose. We also have plans to use further non-official sources to fill current gaps. For example, we are looking to fill sub-indicator headline gaps for indicator 6.6.1 using National River Flow Archive data. Another sub-indicator has already been filled with non-official statistics from the British Geological Survey. Also, headline gap for indicator 14.1.1 is soon going to be filled with non-official statistics based on Marine Conservation Society beach litter data.
In the original publication of the protocol, we emphasised that we would prioritise headline gaps, but we committed to outline a set of priorities for filling disaggregation gaps. Here we state that if we need to prioritise due to potential multiple non-official sources, this will be done based on disaggregations explicitly mentioned in the indicator’s title (we call these required disaggregations). For example, if the indicator has a required disaggregation by age or sex, we will prioritise non-official sources that help to fill those gaps. On the other hand, if a non-official source provides age or sex (or another disaggregation) for an indicator that does not explicitly require this breakdown in its title, the source will be classed as a lower priority. However, if we are in a position where resource is not an issue, we will aim to fill any available disaggregation or headline gap.
This protocol for assessment of non-official sources has gathered much interest internationally, and we have shared lessons learnt and appropriate resources with other National Statistical Institutes and through dissemination across various UN forums. This work continues to evolve, so we will update and refine in the future as necessary as we implement it. We also plan to publish a version for assessment of non-official qualitative sources relevant to SDGs. Some user research has already been done in this area, and we know there is an appetite for contextual information around indicators. As always, we welcome any feedback and suggestions.