Review of the Quality of Recorded Crime Statistics 2020

The Central Statistics Office (CSO) compiles and publishes Recorded Crime statistics for Ireland. Recorded Crime statistics are intended to inform users about the set of criminal offences which are reported to, or become known to, An Garda Síochána (AGS), Ireland’s national police and security service. The statistics are compiled using administrative data recorded by AGS on its PULSE (Police Using Leading Systems Effectively) crime incident database.

Recorded Crime statistics are a vital information source on levels of crime reported in Ireland, and trust and public confidence in the statistics is critical. As part of its commitment to quality, CSO has an ongoing role in monitoring the quality of the data source used to compile the statistics and in communicating information about quality to users.

This report outlines the findings of a fourth review of the quality of PULSE data for statistical purposes and has been carried out by the CSO in 2021.

Recorded Crime statistics compiled by CSO are intended to inform users about the set of criminal offences which are reported to, or become known to, An Garda Síochána (AGS).

The CSO’s Recorded Crime series, published quarterly, provides statistical information on the number of crime incidents recorded by AGS on PULSE, including analysis by the type of criminal offence, the reference period when the offence was reported (i.e. year, quarter), and the location of the offence by Garda administrative areas (e.g. Garda regions, Garda divisions).

The CSO publishes annual statistics on Recorded Crime incidents which have been marked as detected by AGS. Crime incidents are considered to have been detected when at least one person has been identified and sanctioned by AGS for having committed the offence, see Recorded Crime Detection.

The CSO publishes annual statistics in respect of persons who have been recorded as the victims or suspected offenders of serious crimes against the person such as homicide, sexual violence or assault and related offences, see Recorded Crime Victims and Suspected Offenders.

Since March 2018, the CSO has applied a categorisation of Statistics Under Reservation to Recorded Crime statistics to inform users that:

• the CSO has concerns about the quality of the data source
• statistics compiled from the source may understate or overstate the actual occurrence of reported crime
• statistics may be subject to revision in subsequent publications

Recorded Crime statistics are made available to users in CSO Recorded Crime publications, via PxStat the CSO’s new Open Data Portal, or through direct contact with the CSO.

The CSO also uses PULSE crime incident data as a source when compiling other statistical series, such as Prison Re-offending Statistics and Probation Re-offending Statistics.

The Garda Inspectorate report Crime Investigation, published in 2014, raised concerns about the quality of data recorded by AGS on the PULSE database. The findings of the report led the CSO to suspend publication of Recorded Crime statistics in 2014 in order to conduct a review of the impact of the Inspectorate’s findings on Recorded Crime statistics.

The CSO review on PULSE data quality related to crimes reported in 2011, and findings were published in the Review of the Quality of Crime Statistics 2015 (PDF 294KB) Following the review, the CSO recommenced publication of Recorded Crime statistics in 2015 but began to advise users to consider the quality implications of the review when interpreting the statistics. A follow-up review, based on data relating to crimes reported in 2015, was carried out during 2016, and the findings were published in Review of the Quality of Crime Statistics 2016 (PDF 259KB)

In June 2017, the Recorded Crime Q1 2017 statistical release was postponed by the CSO pending the completion of an internal review of homicide incidents by AGS and the investigation of concerns raised separately by the CSO in respect of homicide incident records on PULSE (such as the correct application of crime counting rules). The CSO again deferred publication in September 2017 because the remedial work needed to address the CSO’s observations had not been completed by AGS, and because the scope of the AGS internal review had been extended.

To address the absence of independently produced Recorded Crime statistics and the resulting information vacuum for policy makers and citizens, the CSO decided in March 2018 to recommence publication of Recorded Crime statistics using a new categorisation of Statistics Under Reservation. The categorisation is used to alert users to the CSO’s concerns in respect of the quality of the data source.

In July 2018, the CSO shared a Quality Improvement Proposal with AGS to outline a set of recommendations which the CSO saw as critical to improving the quality of PULSE data for statistical purposes, including three main areas for development:

• appointment of a senior manager with overall responsibility for data quality
• provision of a comprehensive and publicly available document explaining how crime is recorded
• development and implementation of a formal data quality management system for PULSE that ensures fit-for-purpose crime data.

The CSO carried out a third review of PULSE data in 2018, relating to crimes reported in 2017, and the findings were published in Review of the Quality of Recorded Crime Statistics 2017 (PDF 539KB)  The review found appreciable development in PULSE data quality since the previous review and acknowledged new data governance controls introduced during 2018. The review concluded, however, that further development was required and that the categorisation of Statistics Under Reservation should remain in place until a stronger data quality management system was in place to support better quality data outputs.

The following important developments in terms of the use of PULSE for statistical purposes have taken place since the publication of the CSO’s third review of PULSE data in 2018:

• Early 2019: Data outputs arising from PULSE 7.3 in respect of crime incident detections were examined by CSO. PULSE 7.3 introduced new data governance controls in respect of recording detections, thus supporting more reliable statistical outputs in respect of detected crime incidents and suspected offenders.
• Nov 2019: A new AGS Chief Information Officer was appointed, assuming overall responsibility for data and ICT in AGS. The appointment of a senior manager with overall responsibility for data quality had been outlined as a key recommendation in the CSO Data Quality Improvement Proposal.
• Dec 2019: AGS complete the final report of the Homicide Internal Review Team (HIRT) following a detailed internal review of the classification and quality of investigation of homicides by AGS. The report outlines a range of data quality issues and recommendations in respect of homicides.
• Aug 2020: Publication of the Guide to How Crime is Recorded and Counted by An Garda Síochána ('PULSE crime recording guide'), a publicly accessible document which explains the current rules and procedures for recording crime data on PULSE. The publication of such a document had been outlined as a key recommendation in the CSO Data Quality Improvement Proposal.
• Sept 2020: AGS demonstrate the operation of the Performance and Accountability Framework (PAF) to CSO. PAF is a system of regular operational performance review by local Garda management, and adds additional review and supervision to crime incident records on PULSE.

In line with Principle 4 Commitment to Quality of the Irish Statistical System Code of Practice (ISSCOP), the CSO has an ongoing role in monitoring the quality of the data source used to compile Recorded Crime statistics and in communicating information about quality to users. Key data quality dimensions include:

• Relevance - “Statistics meet the needs of users”
• Accuracy and Reliability - “Statistics accurately and reliably portray reality.”
• Coherence and Comparability - “Statistics are consistent internally, over time and comparable between regions”
• Accessibility and Clarity - “Statistics are presented in a clear and understandable form, released in a suitable and convenient manner, available and accessible on an impartial basis with supporting metadata and guidance.”

Recorded Crime statistics are based entirely on PULSE crime incident data as there is no alternative source to support or verify them. CSO recognises that the primary purpose of Garda information systems is to support operational policing needs, but that where practicable and resources permit, the information systems will also support statistical needs. Nonetheless, CSO must consider the data quality management framework and quality assurance procedures which underpin data collection on PULSE when evaluating the quality of PULSE data for statistical purposes. This requirement is strengthened by the level of public interest served by the statistics and the nature and scale of data quality concerns which have arisen in the past.

The review is presented in three main parts:

• A summary of user needs for Recorded Crime statistics and the current availability of data from PULSE to address those needs is presented in the chapter on User Needs.
• An assessment of current statistical outputs in terms of the quality of PULSE data (based on data extracted from PULSE for reference year 2020) is presented in the chapter on Data Quality.
• An outline of what is meant by a “quality management system” in order to build further confidence in Recorded Crime statistics is presented in the chapter on Quality Assurance.