Designing a scenario-based curriculum using cognitive development principles: insights from the Northern Territory Fire and Rescue Service

In emergency management contexts, realism in training is necessary to prepare personnel to effectively and safely undertake their roles. However, scenario-based training, if not implemented effectively, can be costly, resource-intensive and may not accurately reflect on-the-job requirements. This report offers guidance for emergency services organisations in the development and application of scenario-based training. Using examples from the Recruit Firefighter Program delivered by the Northern Territory Fire and Rescue Service, this paper exemplifies how scenario-based training methodologies, underpinned by best practice adult learning and cognitive development theories, have been used to enhance individual learning and agency training outcomes.

Introduction

In 2021, the Northern Territory Fire and Rescue Service (NTFRS) reviewed its Recruit Firefighter Program, seeking feedback from employees at all levels. In response to recommendations, the NTFRS shifted its training philosophy from a didactic, compliance-driven structure to a ‘learning through doing’ approach. The re-designed curriculum uses experiential, scenario-based pedagogies to create a cohesive sequence of activities to achieve industry-specific learning goals and meet national training requirements. The curriculum was also restructured to align with best-practise adult education and cognitive development principles. This ensured that new content and information was sequenced to provide strong underpinning knowledge of particular topics before expanding and linking that knowledge to new topics.

The NTFRS Recruit Firefighter Curriculum

Feedback received from personnel during the curriculum review process highlighted the need to move towards more experiential learning models. Past participants and trainers delivering the Recruit Firefighter Program indicated that learning needed to be more ‘hands-on’ with learners able to ‘discover’ and ‘internalise’ knowledge rather than simply being asked to ‘memorise’ information and ‘mimic’ actions. Similarly, feedback from operational crews suggested that participants needed to understand not only how to undertake specific tasks, but also needed the knowledge to know when and why to do each task. The NTFRS uses scenario-based training to provide participants on the Recruit Firefighter Program the opportunity to experiment with and apply their learning through the reinforcement of strategies, techniques and behaviours required for operational response. Scenario-based training and assessment can simulate high‐pressure response situations and test technical and behavioural skills in a safe and supportive environment (Hjalmarsson 2011; Prasolova-Førland et al. 2017; Sinclair et al. 2012), making it ideal for use in the emergency management sector. The use of scenario-based methodologies provides simultaneous development of collaboration, teamwork, critical thinking, and problem-solving skills (Hjalmarsson 2011; Rantatalo et al. 2019).

The Recruit Firefighter Program sequences scenarios on a continuum, from concrete to abstract, following the hierarchy of Bloom’s Taxonomy (Table 1). While Bloom’s Taxonomy is widely used for primary and secondary (K-12) education, the underpinning education theory has a high degree of relevance and practical implementation for use within adult teaching and learning contexts (Anderson and Krathwohl 2001). Bloom’s Taxonomy has been successfully used in the emergency management sector (van Haperen 2001) and is considered a highly effective approach for the sequencing of training activities in high-risk environments due to its focus on critical reflection and learner autonomy.

The Recruit Firefighter Program learning content is structured along a continuum, moving from concrete and simple to complex and abstract. The early stages of the course focus on developing understanding and application of skills and behaviours through repetition of simple drills. These drills become integrated, with time and practice, to form complex scenarios. As participants progress through the course, the complexity of tasks and realism of scenarios increases to provide ongoing challenges and development of integrated competencies, requiring decision-making, applying rationale and logic and evaluating their own judgements. The course culminates in a 14-hour simulated night shift where recruits are split into operational crews and dispatched to a series of ‘call-outs’.

Table 1: Cognition levels defined by Bloom's Taxonomy. 

Table 2 illustrates how the NTFRS has structured the Recruit Firefighter Program to provide scenarios that sequence knowledge and skill development, incrementally increasing cognitive complexity until participants are competently responding to simulated real-life incidents. The Recruit Firefighter Program uses its structured sequence of drills and scenarios to get participants to trial, observe and evaluate knowledge and skills as they transition through the training program. By incorporating behavioural modelling, repeat practice and a self-reflective dialogue, participants develop their own mental models and integrate learning in ways that are meaningful to them (Ricci and Bravo 2022; Van Hasselt et al. 2008).

NTFRS recruits initially undertake short drills with a specific and singular focus, for example, donning and doffing of personal protective equipment, demonstrating different knots, erecting ladders or using stretchers to transport casualties. During repetitive drill practice, participants can try different methods, cement knowledge or adapt their performance through immediate and specific feedback. Performance during drills is benchmarked to performance criteria with knowledge also being tested through verbal questioning. Outcomes are recorded on individual drill sheets and the evidence captured provides formative assessments over time. Due to the short duration and singular focus, drills can be easily reset and rerun multiple times, which provides cost, time and reassessment efficiencies (Australian Institute for Disaster Resilience 2023).

Table 2: The Recruit Firefighter Program scenario sequence. 

Once participants have demonstrated competency in static drills, complexity is increased by merging short drills together. The deliberate sequencing and repetition of information reaffirms underpinning knowledge that is expanded to gradually form more complex behaviours. Simple scenarios are a useful way to mimic reality and are highly effective to provide targeted practice of specific skills and competencies in discrete settings.

A simple scenario that focuses on the demonstration of specific competencies, such as the example in Figure 1, incorporates the reinforcement of other underpinning and adjacent skills. While the primary focus in this scenario is on the administering of first aid and the demonstration of casualty management techniques, additional competencies may be practised or assessed. These might include the correct use of personal protective equipment, following protocols in using communications equipment or the ability for the members to work and communicate effectively in a team.

Figure 1: Simple scenario. 

As participants progress through the course, complexity is increased until participants are simulating authentic response jobs. Typically, these involve between 3-5 participants and one qualified operational member who plays the role of crew leader. As the example in Figure 2 highlights, in complex scenarios a full operational response is required.

Figure 2: Complex scenario. 

Complex scenarios integrate skills and knowledge from a range of units of competency and test the participant’s ability to analyse the simulated incident through synthesis and evaluation of their prior knowledge to determine the most appropriate response options. This scenario encompasses actions from the time the call out is received until the operation is concluded. Duties include all aspects of a functional response including arrival on scene, briefings, securing the scene, identification and assessment of the source and extent of the spill, implementation of appropriate containment protocols, mitigation of environmental effects and evacuation of live role-play participants. Activities such as decontamination, debriefing and equipment maintenance are also included as part of standard procedures.

In complex scenarios, the incidents and information are structured in a way that allows participants to perform as they would operationally. Participants only complete tasks and functions as dictated by their ‘role’ in the crew. Validity and reliability in the assessment process is improved as learners are only assessed on tasks and competencies they individually performed. Additionally, roles within the response team can be targeted to a learner’s strengths or weaknesses and can be used for reassessment purposes if competencies have not been successfully demonstrated in prior drills and simple scenarios.

Ways to enhance scenario use in high-risk training environments

Successful implementation of scenarios relies on an appropriate and realistic narrative engine (context) and a chronological sequence (timeline) to detail how the scenario will unfold, including the purpose of specific roles or trigger points that will be used to control the flow of events (Australian Institute for Disaster Resilience 2023). Instructions and briefings for scenario management should be documented and include defined aims and objectives, organisational competencies to be achieved, safety considerations and stakeholder roles and responsibilities.

A significant learning for the NTFRS was the distribution of roles and responsibilities for scenario management to ensure that participants had a ‘singular’ focus. Assigned roles may include safety officer, assessors and role-players. During complex scenarios, assessors are essentially ‘invisible’ observers with a focus on capturing and recording evidence for assessment decisions and do not intervene unless there is a safety breach or wellbeing concern. The NTFRS involves a multi-professional team of first responders, including medics, police, emergency services personnel or operational fire crews to simulate authentic interactions during scenarios. This adds additional layers of complexity to the scenarios and positions the assessors as observers to the training rather than as role-play participants.

The NTFRS brings realism to training where possible with the inclusion of special effects (smoke, fire, explosions), simulated medical injuries, use of public housing locations and incorporating interagency role-players. The effective use of role-players to simulate affected individuals or other relevant roles (e.g. crew leader, ambulance officer) adds tension and complexity and can be used to progressively develop the scenarios by posing problems, restricting options or forcing actions of participants (Australian Institute for Disaster Resilience 2023). Exposing participants to stress and decision-making in controlled and measured ways increases their tolerance for and ability to make informed and critical decisions under pressure (Hjalmarsson 2011; Rantatalo et al. 2019). Realistic scenarios also stimulate the emotions and behaviours that occur in real-life emergencies (van Haperen 2001) and promote connections between the subject matter and a participant’s emotional memory, further reinforcing their learning.

Scenarios and scripts need to be developed purposefully (Rantatalo et al. 2019) to ensure scenarios play out to meet the defined aims and objectives but also so that role-players do not inadvertently influence the scenario and cause learner failure. Within the NTFRS, role-players are briefed with sufficient instruction and guidance to perform specific objectives (e.g. causing a change in incident conditions). In situations where live role-players are not possible, the same integrity can be applied by making up ‘identification scripts’ for mannequins (e.g. ‘57-year-old male, unconscious, not breathing’). The participants are able to read the identification script and understand the purpose and then act accordingly. This maintains scenario flow and focus without the need for assessors to provide cues to direct the scenario or influence participant actions.

The NTFRS employs multiple assessors to observe and record learner performance during scenarios. In some circumstances, assessors may be used to capture evidence in different locations (e.g. assessor inside a building) or divided up to assess different groups of individuals. The NTFRS uses an Assessor Observation Record to document performance during drills and scenarios. The template provides prompts to help assessors capture sufficient summative evidence of competencies. With multiple assessors used, all results are compared, discussed and aggregated to determine the assessment outcome. Assessment decisions are strengthened by assessor note-taking that captures specific details of tasks performed. Detailed notes provide evidence of assessment decisions. Using note-taking to support evidence gathering during practical and observational activities enables the outcomes to be consistently interpreted and validated as the information provided documents what was done, and also how it was done.

Learning outcomes from scenario training may not always be clear for participants and, therefore, effective debriefing is required (Rantatalo et al. 2019; Ricci and Bravo 2022). The provision of timely and targeted feedback allows assessment to be reinforced as a learning opportunity. Higher-order cognitive processing skills are developed through critical reflection and debrief (Australian Institute for Disaster Resilience 2023). Debriefs are conducted at the conclusion of all training scenarios and occur in several stages. A hot debrief is conducted by the crew leader (role-player) on conclusion of the incident following a SMEACS format (a format used to guide briefings and de-briefings) to measure the performance of the team in providing an effective operational response. The hot debrief is a key part of the scenario and, as such, assessors observe and take notes that contribute to assessment evidence and outcomes. Debriefing is a critical part of the process to stimulate learning and reflection and is essential to validate discrete components of performance. Discussion at the conclusion of scenarios is used to test foundational understanding and to determine how this knowledge contributed to learner decision-making. Notes taken during debriefs can be used to demonstrate knowledge evidence within units of competency. Participants will also receive a cold debrief from the assessors with a specific focus on individual competency demonstration. Feedback provided on scenario performance includes the combined assessor feedback and commentary and can also include reflections on participant performance from the role-players. Additionally, cold debriefs with role-players helps the NTFRS to validate the assessment process and means training and assessment activities are reflective of organisational practices and that lessons learnt can be incorporated to continually improve agency training programs and practices. This ‘full-circle’ feedback process makes scenario-based learning an effective tool for both individual and organisational learning (Borodzicz and van Haperen 2002).

Conclusion

Considerations of cost, risk, flexibility, fidelity and replicability are often reported as barriers to the use of complex or live role-play scenarios for emergency management training. The NTFRS found that the pedagogical change of approach in the design and delivery of the Recruit Firefighter Program has reduced costs, created greater cohesion in the training syllabus and promoted consistent outcomes for participants. The key insights that NTFRS gained through the curriculum redevelopment process are:

• increasing scenario use and slowly developing complexity has improved recruit performance
• enhancing realism and scenario scope to more accurately reflect real operations has improved ‘job readiness’
• agency staff of all levels are being developed through participation in the Recruit Firefighter Program
• training is continuously being improved because there is a direct feedback loop linking operations and training teams
• decreasing reassessment has reduced training costs, and improved participant wellbeing
• note-taking as assessment evidence is more robust and outcomes are clearly contestable.

The sequenced and progressional nature of the NTFRS’s scenario-based curriculum and the opportunity for learner experimentation and self-reflection responds to the needs of participants through the reinforcement of cognitive development and adult education principles. Aligning scenarios with stages of cognitive development created a training program where learning activities support individual autonomy, promote teamwork, collaboration and critical self-reflection, all of which are necessary skills to provide effective operational responses in high-pressure emergency environments. The fostering of cognitive and behavioural skills alongside technical skills enhances learning outcomes for personnel while strengthening the agency's operational response capacity. The changes to training and assessment products made as a part of the curriculum review have simplified the administrative and compliance processes of documenting training activities and outcomes. By using a scenario-based training methodology and creating sequential learning progression where participants are encouraged to explore and engage with content meaningfully, the NTFRS has created a more realistic training experience that is developing capacity at all levels.