As part of its commitment to quality, CSO has an ongoing role in monitoring the quality of the data source used to compile the statistics and in communicating information about quality to users. Since March 2018, the CSO has applied a categorisation of Statistics Under Reservation to Recorded Crime statistics to inform users about its concerns regarding the quality of PULSE as a source for official statistics.
The CSO Review of the Quality of Recorded Crime Statistics publication is intended to provide statistical users with a broad summary of the limitations of using PULSE data as the source for Recorded Crime statistics. However, as a downstream data user, separate from the recording and data quality assurance processes of AGS, the CSO is limited in its ability to accurately measure and report on the quality of microdata at the source. In this regard, the absence of provision of data quality indicators for PULSE by AGS themselves hampers the CSO’s ability to properly inform users about PULSE data quality. The CSO checks are necessarily limited in scope and happen a considerable time after the data has been recorded, and are not an appropriate substitute for regular, systematic quality reporting by AGS themselves.
The last CSO Review of the Quality of Recorded Crime Statistics, published in December 2018, reported both developments and challenges in the quality of PULSE data used for statistical purposes in the period since the previous review. The rectification of errors in homicide records by AGS during the period leading up to that review had resulted in an upward revision of 18% in homicides reported in Recorded Crime statistics for the period 2003 to 2016. The review noted that the data quality issues identified (such as the application of crime counting rules on PULSE) would not apply only to homicide records and were a likely feature of other crime types too, but where they would be more difficult to identify and rectify given the volume of non-homicide criminal incidents.
In 2017 AGS began a detailed internal review of its classification and quality of investigation of homicides. A core aspect of this review was the issue of data quality in respect of homicides on PULSE. The review led to recommendations to rectify errors relating to individual records on PULSE, as well as recommendations to make changes to recording procedures so as to avoid such errors in the future.
The HIRT identified several ‘areas for improvement’ concerning PULSE data quality, including the following which are relevant to the quality of CSO’s Recorded Crime statistics:
In its commentary in response to the HIRT findings, the Policing Authority noted the need to implement in full the report’s 21 recommendations (which relate to both policy and recording procedures) to ensure public confidence.
CSO are, at the time of writing, awaiting explanation from AGS as to how the findings of the HIRT have led to recording process changes which ensure better quality data.
The CSO’s Recorded Crime series, published quarterly, provides statistical information on the number of crime incidents recorded by AGS on PULSE, including analysis by the type of criminal offence, the reference period when the offence was reported (i.e. year, quarter), and the location of the offence by Garda administrative areas (e.g. Garda regions, Garda divisions).
A criminal offence is recorded by recording an appropriate PULSE crime incident, and as such the PULSE crime incident dataset can be considered as a register of the criminal offences known to AGS and is thus amenable to standard quality evaluations made of data registers including register coverage, classification, timeliness, incomplete records, etc.
Data quality issues outlined in previous CSO reviews as well as in the Garda internal HIRT must be considered by users when interpreting Recorded Crime statistics. These issues may impact on the volume of incidents presented in the statistics as well as the comparability of data over time and between regions including the validity of time series, trends and geographic comparisons. The main potential areas of risk to quality are as follows:
The section which follows examines further assessments of quality made by the CSO as part of this Quality Review.
Crimes that are reported but are not recorded on PULSE
In 2014, the Garda Inspectorate’s Crime Investigation report highlighted the issue of crimes that are reported to AGS but are not recorded on PULSE as being of concern. In previous CSO quality reviews, the CSO sampled records from the Computer Aided Dispatch (CAD) and other incident logging registers to provide estimates for the proportion of reports which appeared to be crimes (based on the recorded descriptions) but were not represented on PULSE as crime incidents. Based on these checks, the CSO estimated that the non-recording of reported crimes had fallen from 18% in 2011, to 12% in 2015 and to 7% in 2017.
Since the last CSO review, CAD is now used by AGS for all logging of initial reports of incidents made by members of the public by telephone, including via 999 emergency calls. AGS have also integrated the CAD and PULSE systems so that a PULSE incident is automatically created when a crime-related CAD incident is closed. This system was introduced in part as a quality assurance mechanism to prevent a scenario where a crime reported by a member of the public was not recorded on PULSE.
In June 2021, the Garda Commissioner outlined to the Policing Authority details relating to the inappropriate cancellation of CAD calls, and, at the time of writing, an internal AGS investigation into the issue is ongoing. The premature or improper cancellation of calls on the CAD system may mean that records relating to crimes, which were reported to AGS, were not created on the PULSE system, and are therefore not counted in Recorded Crime statistics. The CSO is awaiting clarification on the full impact of the issue from AGS, including the time periods involved (how far back this issue goes), the crime types impacted, and crucially, the estimated numbers of crimes which were not recorded on PULSE due to inappropriate cancellation of CAD calls, before it can determine the impact on Recorded Crime statistics.
While quality issues can arise for records which are on PULSE (e.g. incorrect classification of the incident type), perhaps the greatest concern regarding crime data quality relates to crimes which are reported to AGS, but which are not recorded on PULSE. At least when a reported incident is recorded on PULSE a record of the incident exists and a victim is identified. It is critical that AGS examine the reasons for the non-recording of PULSE incidents and have appropriate internal controls to mitigate against crime recording procedures not being followed, so as to ensure that crime incident and victims records which should be recorded are done so appropriately.
Classification of crimes in ‘non-crime’ incident categories
As well as recording criminal offences, PULSE is used to record other Garda activity where a criminal offence has not occurred (property reported lost or found, traffic checkpoints, etc.). The inappropriate use of non-crime incident categories when recording criminal offences was highlighted as a data quality issue in the Garda Inspectorate’s Crime Investigation report. Non-crime incident records are not counted in Recorded Crime statistics.
The CSO sampled 200 non-crime incident records of three types reported in 2020: ‘Attention and Complaints’, ‘Property Lost’ and ‘Domestic Dispute – no offence disclosed’, and examined the Narrative and related data fields to assess whether the classification of the incident as a non-crime had been accounted for, either in the incident’s description or by explicit statement that no offence had taken place. Of the 200 non-crime incident records examined, only one record (less than 1%) was considered to have been misclassified as a non-crime based on the Narrative field. As such, more that 99% of the examined records were estimated to have been classified appropriately, compared with figures of 98% in 2017, 97% in 2015 and 92% in 2011. See Figure 4.1.
Classified correctly | Classified incorrectly | |
2020 | 99 | 1 |
2017 | 98 | 2 |
2015 | 97 | 3 |
2011 | 92 | 8 |
Crimes marked ‘invalid’ on PULSE
Crime incidents can be marked invalid on PULSE if it is determined that a crime did not occur, or if the record is a duplicate or created in error. Invalidated PULSE crime incidents are not counted in Recorded Crime statistics. The Garda Inspectorate’s Crime Investigation report had outlined the incorrect invalidation of incidents as a data quality concern. AGS introduced control measures to prevent inappropriate invalidation, such as increased supervision. The PULSE crime recording guide states that ‘in all instances, the reason for the incident being invalidated is recorded’.
CSO sampled 100 invalidated crime incident records (of around 7,000 which would otherwise have been counted) reported in 2020, and examined the text of the Narrative and related data fields to assess whether the reason for invalidation had been recorded. 92% of the sampled invalidated records had the reason for invalidation recorded. This compares with findings of 88% in 2017, 79% in 2015 and 77% in 2011. See Figure 4.2. It should, perhaps, be considered by AGS how to ensure that no crime incident record is invalidated without the reason being fully documented as per procedure.
Reason for invalidation recorded | Reason for invalidation not recorded | |
2020 | 92 | 8 |
2017 | 88 | 12 |
2015 | 79 | 21 |
2011 | 77 | 23 |
Adherence to crime counting rules
The PULSE recording guide outlines the rules around three important counting principles which are to be applied by AGS when recording crime incidents. Incorrect application of these rules can impact on the coverage of the register used to compile Recorded Crime statistics. The counting rules are:
In practice, it is the primary offence flag that is frequently used when recording incidents on PULSE to try to ensure that multiple offences are counted correctly, by marking those which are not intended to be counted as non-primary in any of the above scenarios.
The CSO extracted a random sample of 100 non-primary incidents on PULSE reported in 2020, along with all associated primary or non-primary incidents, and examined each set of incidents together to consider compliance with the counting rules. CSO consider that 25 of the 100 sampled sets of incidents did not properly comply with the counting rules, including the following examples:
In each case above, there may be reasons why the crimes are linked (e.g. same neighbourhood, same institution) but the applied linkage as primary and non-primary incidents does not comply with the counting rules. The findings of this assessment indicate there is some inconsistent application of the counting rules with the likely result of an undercount of incidents in Recorded Crime statistics.
Further, CSO extracted all PULSE records reported in 2020 in the categories of sexual violence and fraud where a single victim was linked to more than one crime incident, and considered the application of the ‘Continuous series of offences involving same victim and same offender’ rule based on available data including the date occurred, date reported, suspected offender ID and the narrative data field. CSO estimate that 45 incidents of sexual violence in 2020 (2% of such incidents) and 108 incidents of fraud involving a recorded victim (2% of such incidents) were incorrectly counted in Recorded Crime statistics due to the incorrect application of this rule on PULSE. Of those within the fraud category, many appeared as a straight duplication of records.
Misclassification of incident type
A crime incident is classified by offence type (e.g. theft, burglary) when it is entered on PULSE. All classifications are subject to review and supervision, and a reclassification can be applied where the original classification is deemed to be incorrect due to error or new evidence.
The CSO selected a random sample of 50 incidents reported in 2020 in each of six selected crime incident types on PULSE (i.e. 300 in total). The CSO considered the information contained in descriptive data fields to assess whether the incident classification was appropriate based on the information reviewed, as in previous CSO reviews. Of the 300 incidents reviewed, 292 were considered to have been classified correctly based on the descriptive data fields examined. The remaining eight incidents (3%) were either considered to have been misclassified or there was insufficient detail recorded in the description to support the classification, further improvement on previous CSO reviews where an estimated 4% of records in 2017, 5% of records in 2015 and 7% in 2011 were assessed as having been classified incorrectly or there was insufficient detail recorded in the description to support the classification.
Satisfactory | Incorrect/ insufficient detail | |
2020 | 97 | 3 |
2017 | 96 | 4 |
2015 | 95 | 5 |
2011 | 93 | 7 |
Timely recording of PULSE crime incident records
When a criminal offence is reported to AGS it should be recorded on PULSE as soon as possible. Minimising the delay when recording offences ensures that critical information is available in as close to real-time as possible and enhances the integrity of the record relative to the report.
CSO analysed the date_reported data field for PULSE crime incidents of selected types[1] for each of the reference years of the CSO quality reviews (2011, 2015, 2017, 2020) relative to the system-generated date_created data field for the record. The proportion of crime incidents which were recorded on PULSE either on the same day they were first reported or the next day increased from 89% of incidents in the selected incident types in 2011, to 91% of incidents in 2015 and 2017, to 93% of incidents in 2020. The proportion of incidents where the gap was two weeks or more halved, from 4% to 2%, over the same period.
[1] Homicides (excl. dangerous driving leading to death), sexual violence, assault, robbery, burglary, theft, drug offences, criminal damage, public order offences
Same day or next day | 2 - 3 days | 4 days - 2 weeks | More than 2 weeks | |
2020 | 93 | 3 | 2 | 2 |
2017 | 91 | 4 | 3 | 2 |
2015 | 91 | 3 | 3 | 3 |
2011 | 89 | 4 | 4 | 4 |
The CSO Recorded Crime Detections series outlines the proportion of incidents reported in the Recorded Crime series which have been marked as detected. As such, quality considerations outlined in respect of the coverage of the Recorded Crime series must also be considered in respect of detections.
PULSE Upgrade 7.3, released on 25 February 2018, introduced new data governance controls in respect of recording detections. Following the PULSE update, the recording of a sanction such as charge, summons or caution against a suspect should automatically update other data fields on PULSE, all of which are used by the CSO for statistical purposes:
CSO examined PULSE data relating to charges and summons issued in respect of incidents reported in Recorded Crime statistics in 2020, and checked the dependent data fields of the crime incident and suspected offender for coherence, with the following findings:
These findings highlight the impact of stricter data governance controls implemented in PULSE 7.3 on data quality, ensuring greater coherence of data in respect of detected incidents and suspected offenders across the PULSE system and better data outputs for statistical purposes.
The CSO do not receive microdata relating to cautions or other sanctions and were unable to conduct the same analysis for these sanction types.
The CSO Recorded Crime Victims series analyses the set of victims linked to incidents reported in the Recorded Crime series. As such, quality considerations outlined in respect of the coverage of the Recorded Crime series must also be considered in respect of victims.
Where a person is the victim of a criminal offence, details of the person are recorded on PULSE and the person is classified as the ‘injured party’. Not all criminal offences have a victim who is a person, but crimes such as homicide, physical and sexual violence, robbery from the person and theft from the person would have an identified victim in almost all cases. Details such as the sex and date of birth of the victim are routinely recorded on PULSE and CSO uses this data to compile annual statistics in respect of victims of serious crimes such as homicide, sexual violence and assaults classified by sex and age group.
The accuracy of the profiles provided in Recorded Crime statistics is dependent on two main factors: the extent to which an ‘injured party’ has been recorded; and the completeness and accuracy of the data fields relating to sex and date of birth. CSO examined the extent to which an ‘injured party’ had been recorded in respect of the set of crime incidents examined in the Recorded Crime Victims statistical tables of 2020, as well as the completeness of the sex and date of birth data field, with the following findings:
The CSO Recorded Crime Suspected Offenders series analyses the set of suspected offenders linked to incidents reported in the Recorded Crime series. The accuracy and reliability of suspected offender data is closely tied to the accuracy of detections data, due to the data governance controls implemented in PULSE 7.3. Please refer to the section relating to the Recorded Crime Detections series above for analysis.
PULSE crime incident data is, first and foremost, collected to support operational policing and its use as an administrative source for statistical purposes is secondary. The purpose of this chapter has been to highlight to statistical users some of the practical considerations and limitations of Recorded Crime statistics compiled from PULSE.
The publication of a publicly accessible document which explains the current rules and procedures for recording crime data on PULSE in 2020 provides for greater transparency and accountability in how crime is recorded and counted. The strong data controls implemented in PULSE 7.3 in respect of recording detections and the classification of suspected offenders supports better quality data outputs. The review and supervision attached to incident classification, reclassification and invalidation on PULSE have been enhanced in recent years, and the accuracy and reliability of incident classification has improved steadily over time. The time taken between a crime being reported and the incident being created on PULSE has fallen. There is still considerable evidence of inconsistent application of crime counting rules, and it is suggested that these rules should be reclarified or reinforced for relevant personnel, or supervision enhanced, to ensure better adherence to the rules.
Data quality issues outlined in this report as well as in the Garda internal HIRT must be considered by users when interpreting Recorded Crime statistics. These issues may impact on the volume of incidents presented in the statistics as well as the comparability of data over time and between regions including the validity of time series, trends and geographic comparisons.
The recently reported issued relating to the inappropriate cancellation of CAD calls may mean that crimes reported to AGS were not recorded on PULSE and therefore not included in the register from which Recorded Crime statistics are compiled. Although the full impact of the issue is still not known, any systematic non-recording of crime incidents on PULSE undermines the confidence users can have in the completeness and reliability of the statistics.
The CSO Review of the Quality of Recorded Crime Statistics publication is intended to provide statistical users with a broad summary of the limitations of using PULSE data as the source for Recorded Crime statistics. However, as a downstream data user, separate from the recording and data quality assurance processes of AGS, the CSO is limited in its ability to accurately measure and report on the quality of microdata at the source. In this regard, the absence of provision of data quality indicators for PULSE by AGS themselves hampers the CSO’s ability to properly inform users about PULSE data quality. The CSO checks are necessarily limited in scope and happen a considerable time after the data has been recorded, and are not an appropriate substitute for regular, systematic quality reporting by AGS themselves.
Go to next chapter: Quality Assurance
Learn about our data and confidentiality safeguards, and the steps we take to produce statistics that can be trusted by all.