Decision-making in disasters and major crises faced by emergency services globally is a difficult combination of science and art to master. To assist decision-makers in these environments, a number of cognitive aids have been developed and subsequently adopted by individuals and organisations alike. However, these aids vary according to their intent and the context in which they are intended to be applied. This review explores the use of cognitive aids in the context of emergency management and explores how existing knowledge regarding the use of cognitive aids from other industries may be translated to emergency management. An iterative literature review of academic and industry material related to cognitive aids during incident and crisis response across a broad range of international emergency service and other industries within the last 20 years was completed. Ultimately, cognitive aids are not a silver bullet when it comes to decision-making in the emergency management context. The correct tool (that is correctly designed) must be correctly applied by trained and competent end users. The Australian emergency management sector may benefit from future research exploring how these existing tools adhere to the good practice principles identified in this study.
Introduction
High-consequence decision-making during highly complex events is a difficult combination of science and art (Penney et al. 2022; Reale et al. 2023; Ingham 2009). Although emergency management practitioners and teams generally have good capability to respond and are adaptive to the demands placed on them, their cognitive resources will at times be stretched by dynamic, uncertain, time pressured and high-stakes events. Fundamental cognitive processes (such as perception, attention, memory, reasoning, judgement and decision-making) can become overloaded leading to task performance that is less fluid, slower and susceptible to errors or omissions. Good systems, training, planning and preparedness help practitioners respond effectively to these incidents. However, they are still very likely to find these events challenging. To assist practitioners in these environments, a number of cognitive aids have evolved and have been adopted by individuals and organisations alike.
The term ‘cognitive aid’ was first used in the 1970s and was initially used to describe decision-support systems (McLaughlin and Byrne 2020). In the 1980s, cognitive aid was used to describe various tools and systems that supported other cognitive processes (Reason 1987). For the purposes of this paper, we use an expanded version of the Marshall (2013) definition of cognitive aid to encompass a broad range of tools used to support the operational performance of individuals and teams working under pressure. This definition goes beyond Marshall’s (2013) task focused aids to include decision models, frameworks and systems; checklists, aide memoires, standard operating procedures and standard operating guidelines.
This paper reviews the literature on the different cognitive aids that are or could potentially be used in emergency management with the aim of providing more clarity about what cognitive aids are and how they can be used to support complex task performance in emergency management.
Various attempts have been made to categorise cognitive aids, either based on the cognitive processes needed to complete the task at hand (McLaughlin and Byrne 2020) or based on their methods and target outcomes (Fletcher and Bedwell 2017; Burian et al. 2018). In this paper, we adopt a modified version of the Burian et al. (2018) taxonomy and classify cognitive aids according to their purpose and intended application (prior to, during or after an event). Figure 1 shows this taxonomy, with the distinction between the extent to which they are:
- primarily cognitive or behavioural in nature (vertical axis)
- primarily intended for individual or team application (horizontal axis)
- primarily intended for use prior to, during, or post an event (colour coding).
We acknowledge that these are artificial distinctions, but they are useful to discuss cognitive aids and can help direct emergency management practitioners to the right type of cognitive aid depending on their needs and circumstances. Each of the tools is discussed in relation to this taxonomy in 5 categories that emerged as the review was completed. The categories are:
- decision process and behavioural tools
- tools to support analysis
- checklists
- operational procedures and guidance
- cues and alarms.
Figure 1: Depiction of cognitive aids based on the extent to which they are primarily cognitive or behavioural in nature (vertical axis), primarily intended for individual or team application (horizontal axis) or primarily intended for use prior to, during, or post an event (colour coding).
Source: Based on Burian et al. (2018)
Method
This study involved an iterative literature review to identify academic and industry material related to cognitive aids during incident and crisis response across international emergency services and other industries over the previous 20 years. The review provided a narrative synthesis of the use of cognitive aids within the context of emergency management as well as explored how existing knowledge regarding the use of cognitive aids from other industries translated to the emergency management context. The review used search terms including and synonymous with decision models, frameworks and systems, checklists, aide memoires, standard operating procedures and standard operating guidelines within emergency management and industry contexts. References of included works were reviewed for additional suitable material. Databases included those available through the research team’s tertiary institutions, Google Scholar and Research Gate and open source material. Industry material was also reviewed from emergency services agencies where available. The search identified more than 6,000 titles published in the last 20 years. The papers were reviewed for relevance with 79 papers found to address the topic. Narrative synthesis of the 79 articles was conducted to provide a comprehensive picture of the subject matter and to guide new findings and conclusions (Fielding and Thomas 2001; McNeill and Chapman 2005).
Results
Decision process and behavioural tools
Cognitive aids are designed to:
- facilitate decision-making through structured and semi-structured non-technical processes such as decision frameworks and models
- facilitate enhanced analysis and sensemaking of multiple and complex criteria
- assist the development of a common operating picture where all personnel share a clear understanding of the situation, environment and actions or
- provide real-time intelligence and options.
They can be used prior to an event for planning or during an event in real time. They are key to the training of new personnel and once learnt, are designed to improve the process of decision-making during an incident. When applied correctly they can assist with a more consistent process of decision-making (Launder and Penney 2023a). However, they do not and cannot necessarily ensure that a correct decision is made in any particular circumstance. They require the user to be familiar, if not competent, with their application prior to use in real world situations.
At a tactical level, emergency management decision-making is typically driven by naturalistic intuitive processes (Klein et al. 1993; Penney et al. 2022; Reale et al. 2023). At the strategic level of emergency management, where there is often time and increased political and community consequences, more involvement of structured analytical processes in decision-making is generally required. Within military and emergency management contexts, established tools that facilitate structured analytical decision-making (e.g. decision ladders, step-based protocols) share common elements including extended situational analysis (e.g. risk identification, assessment and evaluation) and the identification and comparison of multiple options.
An example of a cognitive aid is the Situation Awareness – Context – Decision Strategy – Planning – Action – Review (S(CD)PAR) model developed by Launder and Penney (2023a). The model focuses on an individual decision maker, however, it could also be applied as a meta-level framework to help incident management teams understand the overall decision-making process. S(CD)PAR identifies 6 stages of an ideal decision-making process, which can be separated into pre-decision, decision and post-decision phases. The theoretical S(CD)PAR framework has been translated into a practical operational guide in the SPAR(CD) model (Launder and Penney 2023b) for use across industries and contexts allowing for consistent training, application and post-incident examination of high-risk, time-sensitive decisions and the identification of common decision errors.
Other examples of cognitive aids to assist decision-makers are crew resource management (CRM) and non-technical skills (NTS) frameworks. CRM was developed as a training program to reduce the incidence of human error in the aviation industry (Kanki et al. 2019; Gross 2014) but has also been applied to the field of emergency medicine (Kemper et al. 2017). Rather than decisions being made by an individual (the chief pilot), CRM involves the use of all available resources from information, equipment and especially other people. CRM is a systematic way of assisting a decision maker to make more accurate and robust decisions by using ‘collective cognitive skills to gain and maintain situational awareness and develop our interpersonal and behavioural skills to establish relationships and communicate with everyone involved’ (Mulenburg 2011, p.13). This helps to combat the issues of human error and failures of cognitive and social skills that were found to be the primary cause of accidents in complex socio-technical environments (Flin et al. 2003; Kanki et al. 2019; Gross 2014).
NTS frameworks are related to CRM and can help to improve performance and reduce error. Several NTS behavioural marker systems that have been developed for these are:
- The Incident Command Skills (THINCS), Butler et al. (2020)
- Team Process Checklist (TPC), Bearman et al. (2023)
- Emergency Management Non-Technical Skills (EMNoTs), Hayes et al. (2021).
THINCS is focused on the individual while EMNoTS and TPC are team-oriented. Unlike S(CD)PAR which is primarily cognitive in its purpose, CRM and NTS frameworks are more behavioural. EMNoTS, for example, identify 7 behavioural markers that index NTS performance, such as communication, coordination, cooperation, leadership, situation awareness, decision-making and coping with stress and fatigue (Hayes et al. 2021). These systems help people to understand what good performance looks like, allow better management of NTS performance in real time and provide a basis for continuous improvement programs (Butler et al. 2020; Hayes et al. 2021).
A different approach has been adopted by the Joint Decision Model (Lamb et al. 2021) that is designed to encourage responders to bring together available information and coordinate goals, decisions and actions to provide a common structure or frame to support responders to jointly consider single and interagency goals. As such, it is designed to facilitate team decision-making. The model’s framework comprises 5 linear phases (Waring et al. 2020, p.632):
- Gather information and intelligence to establish situational awareness and a multi-dimensional understanding of events.
- Assess risks and develop a joint working strategy.
- Consider powers, policies and procedures relevant to the situation, and whether these may assist or constrain decisions.
- Identify options and contingencies.
- Take action and review what has happened to feed into situation assessments and amend plans if necessary.
When applied correctly, the model can assist to engage all stakeholders, reduce potential blind spots through shared awareness and encourage buy-in to the decision-making process. However, to do this, all participants must be competent in the use of the framework and share common understandings and systems. This may be difficult to achieve (at least without extensive training) in the current Australian emergency management context where different functional command systems including the Australasian Inter-service Incident Management System™ (AIIMS) and the Incident Command and Control Structure Plus (ICCS Plus) (ANZPAA 2022) are applied and different levels of expertise and experience are present across jurisdictions and organisations (AIDR 2023).
Tools to support analysis
There are a number of cognitive aids that help people to analyse aspects of the situation to support decision-making. These are primarily cognitive in their application.
In industries that require the analysis of multiple opposing quantitative and qualitative criteria (e.g. oil spill response, airlines, airports and air traffic management) the application of multi-criteria decision-making methods have become standard practice (Wu et al. 2017; Dozic 2019; Wang et al. 2022; Li et al. 2022; Yang et al. 2021). Multi-criteria decision-making methods involve advanced algorithms and the use of fuzzy logic systems, which can account for uncertainty of outcomes within criteria (Dozic 2019; Wang et al. 2023). While such systems can facilitate enhanced analysis of complex information against multiple criteria, they have not (so far) been applied operationally in an emergency management context.
Decision-support systems are software or applications designed to assist decision-making through the provision of real-time intelligence, the prediction of potential outcomes or suggested courses of appropriate action. Examples include applications designed to evaluate a mortgage or plan a road trip (Becker et al. 2022). Within the context of emergency and military services, decision-support systems have been integrated into firefighting (e.g. Zarghami and Dumrak 2020; Tian et al. 2023; Nagarajan et al. 2023; Ujjwal et al. 2023; Wheatly et al. 2023; Xu et al. 2023; Kc et al. 2023), police operations (e.g. Theodosiadou et al., 2023; Sandhu and Fussey 2021; Wu 2021), emergency management (e.g. Kaur and Bhatia 2023; Sun et al. 2021; Bernabei et al. 2021) and military operations (e.g. Lee et al. 2023; Hunter and Bowen 2024; Johnson 2023). As decision-support systems become more commonplace and the integration of AI-supported decision-support systems into emergency management contexts occurs, the influence of trust and the relationship between decision-makers and decision-support systems becomes critical. Inappropriate levels of trust, both in terms of too little or too much, can result in the potential misuse or disuse of decision-support systems (Appelganc et al. 2022; Parasuraman and Riley 1997; Rieger et al. 2023).
Other simplistic, yet equally important tools such as SWOT analysis (Strengths, Weaknesses, Opportunities, Threats) are embedded in business strategic decision-making (Koseoglu et al. 2019). Within emergency management, PESTLE (Political, Economic, Social, Technical, Environmental, Legal) is commonly applied (Penney et al. 2022; AFAC 2016; Sarwar et al. 2017) and is an analytical method best suited to complex systems that require extensive analysis (Christodoulou and Cullinane 2019). Tools such as SWOT and PESTLE provide predetermined and structured categories to assist users focus on relevant themes and categorise information, while continuing to allow freedom of analysis and interpretation within those categories.
Checklists and aide memoires
Checklists and aide memoires have likely been used informally for hundreds of years (Chapparo et al. 2019) but their formal use is attributed to the United States Airforce following a fatal crash in 1935 of a test flight in a new aircraft (Higgins and Boorman 2016; Hayes et al. 2020). An important distinction between types of checklists is between prescriptive checklists, which are primarily behavioural and specify tasks that must be completed and intent-based checklists, which are cognitive and guide decision-making. At a detailed level Chaparro et al. (2019) categorised checklists according to whether they are:
- sequential, where steps are expected to be completed in order
- laundry lists, where the order of task completion does not matter
- iterative, where the checklist is cycled through a number of times
- diagnostic, such as those commonly used in medicine or aviation to identify a medical condition or troubleshoot a systems malfunction
- criteria of merit, which assists the user to evaluate the performance of candidates under assessment.
Many of the checklists identified in this study were either sequential or laundry list checklists.
Checklists commonly used in emergency and crisis management are the suite of AIIMS and Emergency Management Professionalisation Scheme1 functional role aid memoires. Other checklists in emergency management identify key tasks for regional and state coordination centres (Hayes et al. 2020) and seek to reduce biases (Brooks et al. 2020). Studies from a range of industries have demonstrated the potential effectiveness of checklists in improving team adherence to critical steps, increasing standardisation of performance, reducing mental workload and assisting fault-finding and trouble-shooting (Koseoglu et al. 2019; Greig et al. 2023; Torre-Concha et al. 2020; Hales and Pronovost 2006; Higgins and Boorman 2016).
Good checklists are simple, applicable to different settings and provide the potential for measurement when reviewing performance (Torre-Concha et al. 2020). They can also be beneficial for training purposes especially when the task is complex or requires extensive or detailed sequences of actions (Marshall 2013). However, checklists also have limitations. Checklists that duplicate other guidance, are too complex or are considered inappropriate for the task at hand may lead to reduced rather than improved performance and may be rejected by the intended users (Chaparro et al. 2011; Torre-Concha et al. 2020; Anthes, 2015; Reijers et al. 2017; Marshall 2013). Checklists that are too prescriptive or too long may likely inhibit operational discretion required for adjustment and decision-making in complex environments. Organisationally, even a good checklist that is poorly implemented, not supported by appropriate training or poorly integrated with existing processes can result in poor outcomes and user rejection (Anthes 2015; Reijers et al. 2017; Guy et al. 2022; Rose and Bearman 2013). Finally, completing checklists can provide a false impression that work is well done and the associated tasks are well understood by people completing them (Reijers et al. 2017).
Operational procedures and guidance
Standard Operational Procedures (SOPs) are documented rules and steps that must be followed when a specific incident is encountered (Butler et al. 2021). By comparison, Standard Operational Guidelines (SOGs) are not prescriptive, provide principle-based guidance with inherently greater flexibility and can be considered a ‘starting point’ for operations (Weinschenk et al. 2008). Both SOPs and SOGs serve a purpose to facilitate effective coordinated response to disasters (Taber et al. 2008) allowing different teams to follow predefined steps through SOPs or working towards a unified intent through SOGs.
Effectively an operational equivalent to checklists, SOPs are prone to being inappropriately abandoned in favour of operational discretion (Butler et al. 2021) where fire services commanders disregard required processes and actions in favour of their own strategies and priorities. By comparison, where SOGs were implemented, it was found that firefighters would comply with the SOGs in 90% of situations (Weinschenk et al. 2008). Across military and emergency service environments SOPs are typically used as training tools and are more likely to be stringently followed by novices, whereas experienced practitioners prefer to use operational guidelines and personal discretion (Penney et al. 2022).
Outside of fire services operations, SOPs are extensively used within controlled medical and laboratory settings where they can bring compliance with best practice, harmonise laboratory practices, reduce user errors and can be used as training tools (Barbé et al. 2016; Guerra-Farfan et al. 2023). SOPs are not without fault. Sasangohar et al. (2018) found that ‘an abundance of outdated procedures and procedures plagued by information overload’ were common in the offshore drilling industry. Within dynamic and complex environments inappropriate protocols restrict reasonable and necessary flexible situational action and can become a hindrance to effective coordinated action (Taber et al. 2008).
Cues and alarms
Cues are signals that prompt personnel to execute a specific action. Within the emergency management context they include establishing operational and reporting timelines and rhythms. By comparison, alarms are audible or visual (or a combination of both) warnings used to alert people to critical changes to their environment. Common fire service examples at a tactical level include the low-pressure warning whistle on a self-contained breathing apparatus set, the motion sensitive personal distress alarm carried by search crews and the atmospheric or chemical alarms of chemical and gas detection equipment. At the broader emergency management level, examples include community warnings involving threat levels and required responses (AIDR 2013, 2021). For emergency managers, alarms can be used for purposes such as warning of impending decision and trigger points, the approach of reporting deadlines as well as upcoming meetings. For maximum effect, Omori et al. (2017) report that alarms and alert signals should involve flashing lights accompanied by clear, consistent, concise and candid warning messages (auditory and visual), although the potential for distraction and sensory overload needs to be carefully considered.
An alternate form of alarm is the use of early and weak signal detection design to identify and alert emergency management practitioners to impending natural disasters before they would typically be identified (Jongman et al. 2015). A potential problem with alarms is that people can become attenuated to or complacent of alarms. For example, the frequent sounding of the motion sensitive personal distress alarms during large fires when firefighters leave breathing apparatus sets unattended often leads to complacency. It is also the case that people don’t immediately comply with alarms, even in situations where speed is of the essence. Instead, people typically consider the false alarm rate and other potential reasons why the alarm may have occurred before responding (Endsley et al. 2003; McLeod et al. 2005). As Bearman (2013) noted, it is important to remember that ‘alarms occur in an ongoing stream of events in the operational environment, where the operator is constantly building an understanding of their current situation and responding to external stimuli’ (p.13).
Discussion and conclusion
This study included a literature review and narrative synthesis of cognitive aids within emergency management and industry contexts. It was useful to draw distinctions between the cognitive aids based on whether they were decision process and behavioural tools, tools to support analysis, checklists, operational procedures and guidance or cues and alarms. Decision processes and behavioural tools help people through the process of decision-making and interactions with others. Checklists, protocols and guidelines assist decision-makers step through tasks that will help them resolve an incident. Cues and alarms prompt attention to aspects of the situation. Each of these cognitive aids can be described in terms of the extent to which they are primarily cognitive vs behavioural, team vs individuals and whether they are used prior to, during or after an emergency.
The implications of this review are fourfold. First, emergency management agencies and practitioners need to identify the outcome they are seeking to achieve and then select the correct cognitive aid that will assist to achieve this outcome. Figure 1 can be used to help think through the different ways that cognitive aids can support decision-making. Second, emergency management practitioners need to acknowledge that poorly designed cognitive aids may cause more harm than good regardless of whether they are applied in the right context. Third, to improve the use of cognitive aids during emergency events, agencies need to ensure practitioners are appropriately trained in the aid’s selection and use. Finally, emergency services agencies need to recognise the different needs of their staff depending on their expertise and cater for this in the tools they provide. Critically, there is a difference between the way tools are applied between novices and experts with novices tending to adhere strictly to defined steps and protocols while experts desire greater discretion to apply principles within the dynamic nature of an individual event.
Ultimately, cognitive aids are not a silver bullet when it comes to decision-making in the emergency management context. The correct tool (that is correctly designed) must be correctly applied by trained and competent users. Cognitive aids that seek to extend and support the cognitive limitations of individuals and teams to facilitate skilled performance in demanding conditions are a critical but often under-used aspect of decision-making.