Community preparedness programs make an important contribution to disaster resilience, but their outcomes can be difficult to measure. Improved preparedness requires behaviour change, which can take a long time and multiple interventions.
Individuals and communities have different needs and levels of readiness to engage in preparedness. Therefore a one-size-fits-all approach does not work. Challenges such as timeframes for change, local differences and lack of data make it difficult to evaluate the achievements of community programs. However, overcoming these challenges is essential to demonstrate results and to build knowledge of how best to design and implement programs that make a difference into the future. This is an overview of how the Department of Fire and Emergency Services (DFES) is responding to this challenge, drawing on examples from our Bushfire Ready program.
Evaluating preparedness programs
The DFES preparedness programs aim to improve disaster resilience in Western Australia in line with the strategic vision of: ‘Resilient Western Australian communities that work together to build capability and capacity to prevent, prepare for, respond to and recover from disasters’1 and the National Strategy for Disaster Resilience.2
DFES has developed a monitoring and evaluation framework to support evaluation of its community preparedness programs. The framework is aligned to monitoring and evaluation practice and program theory.3 It provides a structure for evaluating preparedness programs against the following objectives:
- individuals and householders have an increased understanding of risk and undertake effective actions to prevent, prepare for, respond to and recover from disaster
- community leaders, networks and organisations have an increased understanding of risk, increased capacity to work with their communities towards disaster resilience and improved self-reliance
- organisations involved in emergency management show increased collaboration and coordination in enhancing community preparedness and disaster resilience.
Contribution of preparedness programs to disaster resilience
Disaster resilience is a broad concept with multiple influences. Parsons and colleagues (2016)4 identify two resilience capacities and eight resilience themes as a framework for the Australian National Disaster Resilience Index:
- Coping Capacity: social capital, economic capital, planning and the built environment, emergency services, community capital, information access.
- Adaptive Capacity: governance and leadership, social and community engagement.
The index describes disaster resilience from a top-down perspective using national data sets. These capacities and themes also provide a useful conceptual framework to clarify how programs contribute to disaster resilience from the bottom up. DFES community preparedness programs contribute improvements in the areas of social capital, information access and social and community engagement. However, there are multiple other influences on these areas. Other resilience themes, such as economic capital, are unlikely to be affected by preparedness programs but could affect a community’s capacity to prepare.
DFES provides preparedness information to residents as part of the Bushfire Ready program. Image: Department of Fire and Emergency Services, Western Australia
Bushfire Ready program and disaster resilience
Bushfire Ready is a local community action program supported by emergency services volunteers that encourages residents to work together to prepare for bushfire. Local networks that can support the community in the response and recovery phases of an incident are also developed. The program contributes to the coping capacity of the community by increasing the availability of localised bushfire prevention and preparedness information and by encouraging householders to take preparedness actions that reduce the risk posed to property by bushfire. Social and community capital are built through activities that bring the community together around bushfire prevention and preparedness. Adaptive capacity is improved by increasing the level of social and community engagement. In this way, the program improves community resilience to natural disasters.
Bushfire Ready program results
Research undertaken in May 2017, following a ‘near miss’ bushfire in Argyle-Irishtown in the Lower southwest region of Western Australia, demonstrates positive program results. The Bushfire Ready program was introduced to the area two years prior to this incident, and grew rapidly. The post- incident survey showed a high level of engagement with Bushfire Ready (66 per cent of respondents) and that most who had participated in Bushfire Ready activities (79 per cent) had undertaken actions to plan or prepare for bushfire. Those who had participated in Bushfire Ready were more likely to have a bushfire plan. On the day of the incident, they were more likely to have communicated with others in the area about the fire.5
The Bushfire Ready program includes practical activities householders can do to safeguard their property from fire. Image: Department of Fire and Emergency Services, Western Australia
Challenges to measuring program outcomes
The DFES approach to engagement is partnership-based, building on existing local strengths, skills and networks.6 Program implementation is flexible and localised and a significant amount of the program implementation work is undertaken by volunteers or other partners. This approach is well grounded in community development practice but also creates challenges for evaluation. Flexible implementation means that gathering consistent data is difficult. In addition, there are multiple factors that affect a community’s resilience and level of engagement with preparedness activities. Evaluation results will always be affected by context and only partially be attributable to the program itself. Unqualified quantitative results or comparisons between areas are not possible.
Where to next
Being specific about the mechanism by which a program affects disaster resilience is an essential basis for evaluating the contribution of preparedness programs to disaster resilience. The themes developed for the Australian National Disaster Resilience Index provide a useful framework for conceptualising how preparedness programs contribute to disaster resilience. DFES is now clarifying how each of its programs supports specific outcomes related to disaster resilience. DFES is also trialling a database for recording and managing stakeholder information. The database contains information about program activities that can be used to monitor and evaluate programs. DFES has also developed a longitudinal survey to assess the contribution of community engagement in targeted high-risk communities to increased disaster resilience.
Footnotes
- DFES 2016, Department of Fire and Emergency Services 2016-2028 Strategic Plan, Department of Fire and Emergency Services, Perth.
- Council of Australian Governments 2011, National Strategy for Disaster Resilience, Commonwealth of Australia, Canberra.
- Markiewicz A & Patrick I 2016, Developing Monitoring and Evaluation Frameworks, Sage, Thousand Oaks.
- Parsons M et al. 2016, The Australian National Disaster Resilience Index Conceptual Framework and Indicator Approach, East Melbourne, Bushfire and Natural Hazards Cooperative Research Centre. At: www.bnhcrc.com.au/research/hazard-resilience/251.
- DFES 2017, Argyle-Irishtown Community Bushfire Survey - Report on Results.
- DFES 2018, Community Engagement Framework, Department of Fire and Emergency Services, Perth.