All Hands on Deck … for Disaster Recovery!

What fully engages all the participants in a conference for IT administrators? Why, a disaster recovery simulation exercise, of course!

During April, the ICCM (International Conference on Computing and Mission) and ITCC (IT Connect Conference) met for a joint conference of IT Administrators serving evangelical missions throughout Anglophone Africa. Training modules were presented on a wide range of topics including low power servers, network troubleshooting, Wasta Linux, Google File Stream, etc.

One module that fully engaged all 42 participants was a disaster recovery simulation exercise. After being split into table-groups of 5-6 participants each, the conference was presented with a disaster scenario followed by two additional ‘injects’ that fed additional complexity and stress to the scenario.

During the exercise, the table-groups were given 15 minutes to respond to the initial scenario and to the two subsequent injects. Three roles were assigned at each table:

  • Recovery Operations Leader – Leads their table-group to consensus on their responses.
  • Recorder – Records the team’s responses to the Scenario and Injects.
  • Scribe – Makes note of observed problems or shortcomings during the exercise, particularly in the Disaster Recovery Plan supplied as part of this exercise.

Following the initial scenario and injects, the table-groups were given 15 minutes to develop recommended solutions to problems identified during the exercise. For example, for a scenario of a fire destroying the server room and local backups, a recommendation might include off-site/cloud backups.

Finally, an additional 15 minutes were reserved for a large group debrief. This gave the table-groups opportunity to share their feelings, observations, and recommended solutions.

The objectives of these exercises include at least the following:

  • An increased awareness of the need to develop effective disaster recovery plans.
  • A better understanding that unmitigated risk can result in severe impact threatening not only their IT operations but the mission it supports.
  • Self-discovery, i.e., the participant transfers the lessons presented to their own context.

Lessons Learned during such exercises help to build a stronger disaster recovery capability.