Use a Scorecard to Scale Your Resilience Program

Can you really say your program and your site are resilient if you only have a yearly drill to measure against? And if you are in charge of a whole system of sites, how can you make sure every site is resilient to the same standard?

Let’s say you’re the emergency management professional in charge of a business-critical site – or even a mixed-facility system with multiple sites and site types. You know you have to maintain a level of resiliency, but your only actual measure is how often you conduct evacuation drills. 

So, let’s say you do conduct your evacuation drills once per year. But no one can find their vest. Or you do the drill, but you don’t activate your response team or get your site leadership together and practice how they would activate their plan for what happens if this is not a drill and something is on fire and you are not going to reoccupy your site. And maybe your critical equipment got cannibalized the last time there was an issue in the gen yard and now no one knows where the flashlights have gone.

What if you identified your critical applications five years ago and haven’t updated them since? And when did you last check your leak detection system and your eye wash stations? It takes work to keep up with everything.  

In a mission-critical or mixed-facility system common standards matter since any facility that experiences an incident can open a link in the chain and bring down the whole company.

The answer is data, and that means a scorecard

In my work building resiliency programs for large systems I leverage something very common in the franchise industry – and that’s a scorecard. Franchisors will often develop scorecards to ensure franchise units are maintaining their standards to an agreed-upon level. This is one of the ways systems make sure the chicken sandwich and smiling service you get at one unit is exactly the same as what you get in a unit two states away.

If you really want your resiliency program to be world-class and to know that every site in your system is maintaining a well-equipped, rehearsed, and ready program, this is how you do it.

Why a scorecard?

A key function of a scorecard is – wait for it – to turn resiliency into a numeric factor. Kind of like getting grades on tests in school, which most of us hated except perhaps those of you who never failed geometry (not naming names here).

For a scorecard to work best, you need to give a final tally of points that will yield a single point of data that gives you a simple metric by which to measure your program. The data behind the actual score will be complex, but the idea of attaching numeric significance to aspects of your program in a single score is something you can use to prioritize your efforts.

If you set a benchmark score, then you know that any facility that falls below your benchmark needs a deeper dive into the data to know where to apply additional effort. And for sites that consistently maintain a high score, you know where to go to find innovation and implementation ideas.

The difference between a scorecard and an audit

It’s a pretty basic, but important, distinction. An audit is typically conducted by an impartial third party against an industry set of standards to prove compliance for regulatory purposes.

A scorecard is maintained and overseen by the team at the facility under your direction and is meant to foster world-class practices across multiple departments and disciplines. The audit is industry-specific. The scorecard is company/type of facility-specific.

If your facility needs to be ISO-certified, it’s easy to build a scorecard that ties into an ISO audit so you’re looking at resiliency from a lens of both security and response.

How to build a Resiliency Scorecard

When we build a Resiliency Scorecard for our clients, we focus on items that are specific to the company, the facility, and the operations. We might build a set of common categories with variations based on whether or not the system has buying offices, stores, distribution centers, manufacturing, data centers, etc. Some companies will have all of those, some will just have offices, some just want to focus on one segment of their business, etc.

We’re not setting a low bar here – we’re aiming for world class. That means creating a scorecard that represents the very highest standards, but that is also tailored to the actual operations of the system.

1. Determine how data will be collected and represented

• You can use online tools like Smartsheet or Tableau. Either of these will give you a good dashboard. 
• If you don’t have the budget for extravagant data collection, I’ve successfully used an Excel spreadsheet designed to automatically tabulate scores.
• Set the reporting period where data is due and then set up an intake to collect the completion of the items for that period.
• If you’re using a tool like Tableau, you’ll be able to create a dashboard with overall scores that will have drill-down capability to see the data that makes up each score.

2. Identify your facility/system groupings

• Because what we’re using is data, it’s also highly useful to build your scores into a dashboard so you (and your executive team) can see the composite scores across the system.

Some types of groupings:

• Type of facility
• Business line
• Region
• Reporting line (if you group your facilities under specific positions)
• Maturation level (newly implemented, mature, aging)

3. Identify your categories

• Your categories should be broad enough to cover all of your types of facilities.
• Name your categories in a way that’s Evergreen (terms won’t change over time).
• Consider that each category will need to have a composite score.
• Identify which categories will be scored in which time frame (monthly, quarterly, semi-annually, annually).

The categories we most commonly use:

• Equipment
• Drills and System Checks
• Plan and Recovery Validation
• Exercises, Training, and Onboarding
• Activations and After Actions

4. Create specific and actionable items in each category

• Think about the things that matter to your overall program vs what matters to the specific type of facility.
• Match your items to your actual corporate structure and tolerance for risk and desire to remediate.
• Any item on the scorecard should be there with the full support of your organization.
• Create a guide that clearly details the expectation for each item. Be very specific.

5. Weight the items and determine their cadence

• Because you’ll be using a numeric system, assign weights to each category and then to each item in a category.
• This sounds simple on the surface, but it can be a challenge to decide what weight to give specific items. Is a notification system test more or less weight than an equipment check? There’s not a simple answer.
• For a manufacturing plant, data center, or distribution center (i.e., any large complex mechanical facility), it may be more critical to have functional and tested equipment.
• For a 46-story office building in the heart of Manhattan, it may be more important to conduct evacuation drills.

My philosophy with weighting is two-fold:

1. Review past incidents and after actions. Did root cause analysis reveal commonalities for the failure? Weight those items higher.
2. Give higher weight to any item that requires teamwork and that cannot simply be checked off, such as tabletop exercises, plan validation, and RTO confirmations.

6. Assign responsible parties

• This is super critical. If you can put a specific role/function next to each item, then you are giving that person responsibility for seeing that the item is completed as specified.
• Accountability should be determined based on the most logical person to oversee each item. In other words, the item should sit in its functional wheelhouse. EHS should verify systems that keep people safe. Security should verify alarms and system alerts. Facilities should verify equipment. The leadership of the site should verify the comprehensive plan and tabletop exercises. HR should verify notification and activation and employee communications. And so on.
• Someone – typically your trained and named Incident Commander for the site – should have OVERALL accountability.
• One suggestion that has worked very well for my clients is to put the scorecard on the docket of the safety committee for the site. Then, make the Incident Commander responsible for putting it on the agenda and making sure the whole committee is working together and that each accountable person is supported.
• Caution: This isn’t always easy.

7. Set a minimum score

• You want to set a single numeric score that will be used for an overall representation of resiliency.
• In the first years of completing a scorecard you may want to set your score at a point at which you believe the majority of your facilities can meet. You can increase the score over time as all facilities increase their resiliency.
• I usually recommend a minimum score of 85%. This means the facility meets more than three-fourths of the requirements. It’s a good metric. A site that gets an 85% is basically a solid “B” if you’re looking at this like a teacher would.
• Anything beneath your minimum score means greater effort and greater scrutiny.

8. Decide who will receive notice of scores

• If you have some type of quarterly reporting on your program, a composite of scores using basic charting available in Excel or PowerPoint can make for a powerful reporting mechanism.
• You may want to provide scores by your groupings determined in Step 3 and then include executive stakeholders related to each grouping.
• The team at each facility should always receive their scores before they are seen by anyone in their reporting chain. The idea is to partner with them to increase resiliency, not shoot them in the foot.

9. Determine low score remediation and high score rewards

• A little competition is not a bad thing. There’s a reason many manufacturing facilities have large boards that list the number of days since their last incident.
• For sites that fall below the minimum standard you set in Step 7, you may want to establish either a formal or informal triage process that prioritizes the order in which you address their score. For example, a distribution center that scored low because they just went through a major retrofit and have a new site manager may not need the same intensive redress effort as a data center that had an arc flash incident that escalated to a crisis when it wasn’t handled correctly.
• For sites with high scores, your action might be to have them write a case study about the things they are doing to achieve their score. They may be innovating or doing an especially good job at building their incident team. Those best practices can be bundled into a newsletter or presentation that might increase the resiliency across your entire system. At the very least, highlighting the work of high-score sites reinforces the importance of resiliency.

10. Set up a yearly refresh

• Because you’ll be scoring in set time periods (monthly, quarterly, etc.), you’ll want to make sure you have a period each year where you evaluate the scorecard itself and tweak it for new/different needs.
• You’ll need to begin this process in the third quarter each year, so you can release any updates and updated expectations before the new year rolls around.
• This is also the time to refresh any training or presentations and reach out to each site to address questions.
• Caution: Tweak carefully. If you change too much year over year you won’t be able to build a good data set over time. Try to keep the categories consistent, even if you need to move around some of the items in each category. This is where using the Evergreen principle (Insert link to MM article) is important. The more commonly you name your categories the less likely you’ll need to change them.

The danger

Of course, there’s a downside to building and using a Resiliency Scorecard. Much like a franchise system, you have to be willing to take corrective action if a site does not make your passing score. Difficult consequences can result from not acting on a known gap in systems, equipment, or training if something does go wrong at that facility – you can face legal liability, regulatory action, and your insurance carrier won’t be pleased.

The upside

Embracing a Resiliency Scorecard as a tool for excellence gives your sites a better chance of achieving and maintaining a world-class level of resiliency. And world-class means an unbroken chain of strong sites that can withstand an incident and resume operations, protect life safety, and protect the reputation and viability of your brand.

Michelle Newcome

Founder

Michelle Newcome is the founder of Risk Resiliency. She excels in designing highly effective and comprehensive resiliency programs for complex facilities.