An assessment of the opportunities to improve strategic decision-making in emergency and disaster management

---

Abstract

---

Article

Background

Decision-making in emergency is challenging and stressful due to the dynamism, complexity, uncertainty and temporality that occurs in this environment (Danielsson & Ohlsson 1999). In this paper, the term emergency management is in the context of the decision-makers selected from Australian and New Zealand emergency services agencies who work in the response phase at a management level. In Australia and New Zealand events are termed ‘level 3’ incidents and invariably involve a multi-agency approach. This is not unique and is comparable to other safety-critical, high-consequence environments such as the military and pre-hospital medicine (Baker, Day & Salas 2006, Wildman et al. 2011)

The Australasian Inter-Service Incident Management System (AIIMS) is a system for incident management used by fire agencies, State Emergency Services (SES) and other organisations. AIIMS is based on three principles:

• Management by objectives – an incident controller works with an Incident Management Team (IMT) to determine the desired outcomes (objectives) to be achieved.
• Functional management – this is based on a structure of delegation with five functional areas of control, planning, public information, operations and logistics.
• Span of control - this relates to the number of groups or individuals who can be successfully supervised by one person. Up to five reporting groups or individuals is considered desirable, allowing the supervisor to monitor those groups and individuals.

A simplified description of decision-making in AIIMS is a cyclical interaction between intelligence, planning, operations officers and units and the incident controller (see Figure 1). This is described below in association with the key management system ‘products’ that inform and guide decision-making (in bold).

Figure 1: A simplified model of the AIIMS decision-cycle.

The Incident Controller sets the incident objectives. The Intelligence Officer confirms the Incident Controller’s incident objectives and key questions for decision-making; collects and analyses information for the IC; processes that information into a suitable form for analysis; and organises, collates, disseminates and displays intelligence in the form of a Common Operating Picture (COP). The Planning Officer obtains the intelligence products to support the development of an Incident Action Plan (IAP). They conduct risk management activities (monitors current and emerging risks); undertakes Options Analysis (OA) involving alternative incident objectives and strategies and collects, collates and stores incident records. The Operations Officer implements the IAP and advises the IC about emerging risks, the current control situation. The Incident Controller approves the IAP including setting priorities for action and monitors IAP over time against objectives. (Derived from interpretation of AFAC 2013).

Decision-makers in these environments use a range of techniques to support decision-making efforts. Mental shortcuts or heuristics aid in the decision-making process (Mishra, Allen & Pearman 2013). Decision-makers rely on their expertise (Erkisson 2006) and often apply intuitive processes (Kowalski-Trakofler, Vaught & Scharf 2003). While human flexibility and adaptability is significant, human cognition is subject to bias, errors and limitations (Kahnemann 2011). These are amplified in complex, uncertain, politicised, and temporally-constrained environments, and this does require balancing from a number of perspectives.

Strategic decision-making is a balancing act

Balancing decision styles

Decision-making comes in different ‘styles’. Flin, O’Conner & Crichton (2008) indicate there are four decision styles: creative, analytical, procedural and intuitive. In practice these decision styles differ in terms of the amount of conscious effort required and the processes applied can also vary. At different phases of an emergency event some or all of these decisions styles may be necessary.

Decision-makers at a high level have considerable relevant experience and may, through pattern-matching, intuitively identify appropriate responses to that pattern ( Kowalski-Trakofler, Vaught & Scharf 2003). Nevertheless, intuition can also be considered a source of bias and potentially lead to mistakes ( Eva et al. 2010). The determination of whether intuitive judgments can be trusted requires an examination of the environment in which the judgment is made and of the opportunities the decision-maker has had to learn the regularities of that environment.

Procedural decision-making involves use of rapid tactics that follow some type of rule. One type of rapid decision-making is fast and frugal heuristics (Gigerenzer 2004). Fast and frugal heuristics are a cognitive heuristic that relies on a few relevant predictors to simplify and speed up the decision-making process (Gigerenzer 2004). Decision-makers follow a series of sequential steps prior to reaching a decision. According to Marewski and Gigerenzer (2012), fast and frugal heuristics are built around three rules:

• search rule – one that specifies in what direction information search extends in the search space
• stopping rule – one that specifies when information search is stopped
• decision rule – one that specifies how the final decision is made.

Analytical decision-making is the closest to the ‘classical’ model of decision-making. In situations where there is time to make a thorough analysis of alternatives, the strengths and weaknesses and to compare the value or utility of the outcome, such an approach is warranted. Typically this approach is necessary when the decision has significant consequences, such as the evacuation of a major population centre.

As situations become more novel or unique decision-makers step into the realm of needing to be creative. Unfortunately, in emergency management, this can occur under the most extreme circumstances when all known tactics have failed and life and property are under immediate threat.

Decision-makers need to match decision styles to the decision context. For example, if the decision-maker is establishing whether to evacuate a town that may flood, then they are likely to be drawing data from flood modelling, local observations that feed into rational approaches to decision-making, while also managing political and community interactions. An Urban Search and Rescue (USAR) commander making a decision where to locate a team base may be making intuitive calculations about the exposure to different risks to identify the lowest risk option. Regardless of the style, the literature on decision-making suggests that they need to produce decision outcomes, and also consider the decision process. They should be ‘thinking about their thinking’. This is one definition for ‘meta-cognition’. A meta-cognitive process includes being aware of the decision style as well as managing for bias and typical errors.

Balancing risks and strategies

Risk assessment is an important part of emergency management. Formally, it is a process that measures probability and severity for individual risks, identifies control options and draws conclusions about the acceptability of the residual risk. If only the task were this simple in practice! The emergency manager is generally managing multiple risks as well as making decisions on which of one or several risks should be accepted relative to another, associated with particular tactics or tasks. The combination of risks and tactics with the different types of pressure that impinge on decision-making (lack of time, uncertainty, finite resources, fatigue), tend to make resultant calculations difficult. As demonstrated in the following vignette of a decision being made by a USAR commander:

During the deployment of the Australian USAR team to Fukushima, the team leader needed to manage four key risks associated with after-shocks, tsunamis, extreme cold and radiation exposure. He needed to manage these risks while making decisions about where to locate the Base of Operations (BOO). At any particular site available for the BOO the residual risk combination is different, and this risk combination also affects the ability to meet the objectives of the deployment. So locating the BOO on high ground, away from buildings on a baseball pitch reduced the risks associated with tsunami and building collapse from after-shocks but increased the risk associated with exposure to extreme cold. Being significant distance from the search and rescue area reduced the available time for operations. (Research project participant).

An analytical approach to this particular decision would require the assessment of the four risks for probability and severity of outcome, the control options available, and the residual risk. It would need to be applied for each possible BOO, and considered relative to the deployment objectives. This might be described as ‘benefits’. The result is some sort of risk-benefit assessment. This decision process is not accommodated in current risk assessment flowcharts, as demonstrated in the Dynamic Risk Assessment flowchart (Figure 2). A standard risk assessment matrix might help, but only with the formulation of the probability by (severity of) outcome calculation when all of these outcomes could lead to multiple fatalities that reduces the variability in the calculation to differences in probability.

Figure 2: Dynamic risk assessment (adapted from the NSW Rural Fire Service Crew Safety and Welfare Manual 2012)

The reality is that this is not the way risk-related decisions are typically made once they reach this level of complexity. Decision-makers will often intuitively approximate this process. To continue the vignette:

Intuitive assessment of risks provides an approximation of the risk level, and management options. Step-wise comparison with deployment objectives can then occur. Exposure to extreme cold can be managed, whereas risks of after-shocks and tsunamis was outside of the team’s control, also with catastrophic consequences. The deployment was also quickly in a recovery mode reducing the time pressure on operations. The risk assessment therefore points to locating the BOO at the baseball field, accepting and managing the risk of exposure to the cold. (Research project participant)

A number of other types of balancing arise from this short vignette of a decision. Due diligence requires organisations to record and provide explanations of decisions. What is the balance of record-keeping and the effective support of decision-makers? While an individual may have the legislative responsibility for making decisions, decisions are typically canvassed from a team. How should the decision-maker balance deciding with collaborating? Emergencies can occur over extended periods of time and change dynamically during the event. There are not always discrete decision-points. How does the decision-maker balance the need to make sense over hours and days while also making decisions throughout?

Balancing record-keeping with effective decision-making

When the outcomes of emergencies include significant loss of property and, in particular, loss of life, commissions of inquiry and royal commissions are often the result. During those investigations decisions are scrutinised, and the scrutiny has a tendency to assume that the analytical style of decision-making was both possible and preferable. These legal environments rely on interrogation of witnesses and witnesses rely on recalling evidence. Subsequently, the recording of decisions becomes particularly important.

While this is necessary in terms of meeting due diligence and record-keeping requirements of any modern organisation, the way decisions are recorded and the type of information recorded can influence the actual decision and subsequent actions. Kahnemann and Tversky’s work on cognitive biases and errors (see Kahnemann 2011) suggests that formalising a decision can ‘anchor’ the decision-maker to that decision and, even in the face of contradictory evidence, they will often hold to the original decision. Allowing the record to reflect information or other triggers that would change the decision is a straightforward way to overcome this problem. The requirement to record a decision may lead some people to act hastily (to support perceptions of control) or delay decisions (to avoid making incorrect decisions). From a decision-science perspective there is a range of things that could be recorded, such as how the decision-maker has managed the possibility of bias or error. There are other competency, cultural and legal challenges to implementing such a change. In order to record issues around decision bias and styles the decision-maker needs to feel confident in meta-cognitive (thinking about thinking) decisions. ‘Small target’ legal approaches suggest that the more information provided in a decision record, the greater the opportunity for that information to be misconstrued during an inquiry. Command-and-control cultures demand certainty of leaders so recording information that might change the decision could be considered a weakness. The balance between recording decisions and making effective decisions continues to be a challenge while these issues remain.

Balancing decision-making and sense-making

Sense-making involves ‘turning circumstances into a situation that is comprehended explicitly in words and that serves as a springboard into action’ (Weick, Sutcliffe & Obstfeld 2005). Sense-making could be viewed as an ‘overlay’ on decision-making where the individual engages in iterative cycles of analysis, action and reflection. Although the concept of sense-making has been around since the 1980s, its qualitative difference to decision-making has grown in relevance and importance to emergency management in the last 10 years. Weick notes that ‘sense-making in crisis conditions is made more difficult because action that is instrumental to understanding the crisis often intensifies the crisis’ (Weick 1988, p. 305).

There is also a delicate trade-off between dangerous action, which produces understanding, and safe inaction, which produces confusion. In this regard, sense-making is intimately linked with action because action directly impacts on ‘the psychology of control, effects of action on stress levels, speed of interactions and ideology’ (Weick 1988, p 315).

Some of the key properties of sense-making in incident response point to the importance of the incident commander. ‘Who is he or she as a person? Over the course of a lifetime, what social constructs have created and shaped him or her? How does he or she communicate with others? What type of retrospective learner is he or she?’ (Renaud 2010, p. 45).

Balancing the imperative for control with the need for psychological safety of teams

A final balancing act relates to the fact that incident management is hierarchical, and incidents must be controlled. At the IMT level this control rests in the hands of an Incident Controller. Above the IMT this position tends to be replicated with a single point of command or control. It is possible that this level of control can conflict with issues of building team environments where people feel free to speak up when they feel decisions are incorrect. Edmondson (1999) refers to this as ‘psychological safety’. Edmondson suggests that in order to build psychological safety leaders (those in ‘control’) need to demonstrate vulnerability, to articulate the unknown and unfolding nature of the ‘problem’ (in this case the incident), to frame it as a learning problem and to make sure that everyone in the team commits 100 per cent to the process. Under these circumstances, team members are more likely to speak up about the evidence that indicates the objectives are not being met, or the problems with the Incident Action Plan.

Evidence from the CRC research project

This research involved 18 agencies in Australia and New Zealand that contributed data and discussed issues around decision-making. Semi-structured interviews were conducted with chief officers, deputy chief officers, principal rural fire officers, senior officers, state coordination personnel, regional coordination personnel, and incident management team personnel. These personnel represented the rural fire authorities, urban fire brigades, rural fire agencies, land management agencies, SES, council officers with responsibility for search and rescue, and the Red Cross. Subsequent to the interviews the team collected documentation and reviewed policies, procedures and training approaches.

Research results are summarised in Table 1. It is important to note that Table 1 only identifies the degree to which the formal incident management system captures particular decision characteristics. Although not reported here, it is acknowledged that individual decision-makers identified tools, checklists, tips, heuristics (rules of thumb) and decision models that they had collected as part of their experience, training and professional development to support decision-making.

Table 1: Evidence if implementation of decision concepts in the incident management system from a sample of organisations.

Decision concept	Where recognised in incident management system	Coverage in surveyed Australia/New Zealand organisations
Decision styles: awareness of and an ability to work across the spectrum from intuitive to classically rational decision approaches as the context requires.	Commander’s intent, some organisational decision models.	Less than 15%
Monitoring themselves and their teams for evidence of bias or decision errors (linked with decision styles).	Meta-cognitive strategies.	Less than 15%
Sense-making: recognition of the dynamic nature of the process, and the need to not just decide, but to make sense.	Commander’s intent, some decision models, individual heuristics, some IAP structures.	50%
Record-keeping: balancing the need to record decisions for future reference with the effect recording has in creating bias in decision-making.	Decision logs.	Less than 15%
Creating psychologically safe decision environments that build and maintain trust in teams.	Embedded in broader organisational value statements.	50%

Discussion

This paper identified several decision concepts that have emerged in the literature and identified opportunities to improve the standard of decision-making. Analysis has assessed participating end-users in this CRC project to determine the degree to which they had embedded these concepts in operational environments. Organisations build psychologically safe environments where team members can speak up, and where decision-makers are engaging in ‘sense-making’. There is less effort being put into recognising when shifts in decision styles occur, in monitoring for bias and errors through meta-cognitive processes, or in managing the effect that recording has on ‘anchoring’ or fixing a decision-maker to a particular course of action.

Way forward

The focus of this CRC project is to develop and test cognitive tools. These tools help to integrate knowledge about human performance (related to issues such as information processing and recording) and the management within an organisational culture that fosters a supportive environment for the decision-maker. The aim is to discover whether the decision-maker can be effectively supported. These tools will be demonstrated during training courses and tested during exercises and emergency events where possible over 2016 and 2017. Taking into account the results, Table 2 describes the tools to be tested in the coming 12 months.

Table 2: Concepts and associated tools being tested in the CRC project.

Decision concept	Tool being tested or approach taken
Awareness of and an ability to work across the spectrum from intuitive to classically rational decision approaches as the context requires them to.	Training course to understand decision styles linked with several meta-cognitive tools.
Balancing the need to record decisions for future reference with the effect recording has in creating bias in decision-making.	Modified decision logs to record decisions that map uncertainties and trigger change decisions.
Monitoring themselves and their teams for evidence of bias or decision errors.	Checklist for biases and decision errors.
Creating psychologically safe decision environments that build and maintain trust between teams.	Training course and a simple tool to apply the steps of psychological safety identified by Edmondson (1999).

As the results of this testing emerge the tools that are considered usable by the participants, and why, will be identified. Usability in this regard relates to three concepts of efficiency, effectiveness and user satisfaction (ISO 1998). If the evidence indicates that the tools are usable then end-users can be confident there is value in integrating the tools within the formal incident management system.

Acknowledgments This research is supported by funding from the Bushfire and Natural Hazards CRC.

 

References

Australasian Fire and Emergency Services Council (AFAC) 2013, The Australian Inter-service Incident Management System: Teamwork in Emergency Management (4th Ed). AFAC, Melbourne, Australia.

Baker DP, Day R & Salas E 2006, Teamwork as an essential component of high-reliability organizations. Health Services Research, 41 (Sachs 2005), pp. 1576-1598.

Danielsson M & Ohlsson K 1999, Decision Making in Emergency Management: A Survey Study. International Journal of Cognitive Ergonomics, vol. 3, no. 2, pp. 91–99.

Edmondson A 1999, Psychological safety and learning behavior in work teams. Administrative Science Quarterly, vol. 44, no. 2, pp. 350-383.

Ericsson KA 2006, Cambridge Handbook of Expertise and Expert Performance. Cambridge Universal Press, Cambridge UK.

Eva K, Link C, Lutfey K & McKinlay J 2010, Swapping horses midstream: factors related to physicians’ changing their minds about a diagnosis. Academic Medicine, vol. 85, no. 7, pp. 1112-1117.

Flin RH, O’Connor P & Crichton M 2008, Safety at the sharp end: a guide to non-technical skills. Ashgate Publishing, Ltd., Aldershot, UK.

Gigerenzer G 2004, Fast and Frugal Heuristics: The Tools of Bounded Rationality. In D. Koehler & N. Harvey (Eds.), Blackwell Handbook of Judgement and Decision Making (pp. 62-88). Oxford, UK: Blackwell Publishing Ltd.

ISO 1998, International Standardisation Organization, Ergonomic requirements for office work with visual display terminals (VDTs), ISO 9241 Part 11: Guidance on usability.

Kahneman D 2011, Thinking, Fast and Slow. London: Penguin Books.

Kowalski-Trakofler KM, Vaught C & Scharf T 2003, Judgment and decision making under stress: an overview for emergency managers. International Journal of Emergency Management, vol. l, no. 3, pp. 278-289.

Marewski J & Gigerenzer G 2012, Heuristic decision making in medicine. Dialogues Clinical Neuroscience, vol. 14, no. 1, pp. 77-89.

Mishra J, Allen DK & Pearman AD 2013, Information use, support and decision making in complex, uncertain environments. Proceedings of the American Society for Information Science and Technology, vol. 50, no. 1, pp. 1-10.

Renaud C 2010, Making Sense in the edge of Chaos: A Framework for Effective Initial Response Efforts to Large-Scale Incidents. Masters Thesis. Naval Postgraduate School, Monteray, California.

Weick K 1988, Enacted Sensemaking in Crisis Situations. Journal of Management Studies, vol. 25, no. 4, pp. 305-317.

Weick KE, Sutcliffe KM & Obstfeld D 2005, Organizing and the process of sensemaking. Organization science, vol. 16, no. 4, pp. 409-421.

Wildman J, Fiore S, Burke C & Salas E 2011, Trust in Swift Starting Action Teams: Critical Considerations. In N. Stanton (Ed.), Trust in Military Teams (pp. 71–88). Farnham, Surrey: Ashgate Publishing Ltd.

About the authors

Dr Benjamin Brooks is a human factors researcher and Senior Research Fellow in the Australian Maritime College at the University of Tasmania.

Steve Curnin is a research fellow at the Tasmanian Institute for Law Enforcement Studies at the University of Tasmania.

Chris Bearman, Central Queensland University, is a researcher and project leader for the Bushfire and Natural Hazards CRC decision-making, team monitoring and organisational learning project.

Dr Christine Owen is an organisational behaviour and learning researcher at the University of Tasmania.

Sophia Rainbird is a post-doctoral researcher and anthropologist specialising in movement, risk and resilience at the Central Queensland University.