How to Retire a Weak Safety Metric in 21 Days

F2 | Safety Indicators and Metrics

A metric can stay on an executive dashboard long after it stops improving a safety decision. This 21-day review gives EHS leaders a practical way to test whether a measure still reveals control quality, assign an owner, and retire it when it only creates reporting activity.

Andreza Araujo's work on safety culture starts from a demanding premise. A number deserves attention only when it changes what leaders verify, fund, or stop. The goal is not a larger dashboard. The goal is a smaller set of measures that makes weak control visible before harm occurs.

What you need before starting

Before reviewing a metric, collect its definition, data owner, source system, reporting frequency, current target, last 3 reporting periods, and the decision it is supposed to support. Do not begin by debating whether the number is good or bad, because a measure can be accurate and still be irrelevant to the decision in front of the team.

Set aside 2 review meetings, one with the people who produce the number and one with the leaders who act on it. A safety measure should not survive because it is familiar. It should survive because someone can explain what changes when the result moves.

Step 1: Name the decision the metric should change

Write one decision in plain language before examining the data. Examples include increasing verification of a critical control, changing maintenance priority, pausing a task, or funding a design improvement.

If the team cannot name a decision, classify the measure as an observation rather than a decision metric. That distinction matters because a count can describe activity without showing whether exposure is falling. Andreza Araujo makes this separation central to the argument in A Ilusão da Conformidade, where visible compliance is not treated as proof that the underlying practice is reliable.

Record the decision, the decision owner, and the maximum acceptable delay between the signal and the response. A 30-day review cycle is too slow for a measure intended to detect a deteriorating control during a shutdown.

Step 2: Describe the risk pathway behind the number

Connect the metric to a specific risk pathway that has a beginning, a control, and a possible consequence. For example, a verification measure might track whether isolation was checked before maintenance, rather than counting how many checklists were uploaded.

Use 3 questions. What condition creates exposure? Which control is meant to interrupt it? What evidence shows that the control worked under the actual work conditions? If the measure cannot answer the third question, it may be tracking paperwork instead of protection.

This step keeps the review close to the work. It also gives the team a defensible reason to replace a familiar indicator whose connection to serious risk has weakened.

Step 3: Test whether the definition survives ordinary work

A metric is usable only when two reasonable people classify the same event in the same way. Read the definition against 5 recent examples, including one normal shift, one abnormal task, one missed control, one late record, and one case where the work changed after the original plan.

Look for undefined words such as complete, safe, adequate, timely, or compliant. Those terms sound precise until a supervisor must decide whether a control passed at 6:00 a.m. after a night-shift handover and a last-minute equipment change.

If the classification changes from person to person, redesign the definition before changing the target. A data dictionary can help, and the Headline Podcast guide to safety metric data dictionaries gives a related structure for that work.

Step 4: Check the delay between signal and action

Measure the time between the event, the record, the review, and the decision. A useful indicator can still fail when it arrives after the work has moved on.

Map the 4 timestamps in a simple table. Include when the condition existed, when someone recorded it, when the responsible leader saw it, and when the operation changed. If the last two timestamps are separated by 14 days, the measure may be suitable for trend review but not for frontline control.

The point is not to make every measure real-time. The point is to stop pretending that a delayed administrative count can guide a decision that must happen before exposure.

Step 5: Look for incentives that can distort the result

Ask what people gain by improving the number and what they risk by reporting the truth. A target tied to recognition, budget, or performance review can make the metric look cleaner while reducing the quality of the information behind it.

Review 3 patterns. Does the denominator change when workload rises? Can a record be closed without proving control effectiveness? Does the measure reward more observations even when the observations are shallow? These are not accusations against the people entering data. They are tests of the measurement design.

Andreza Araujo's book Muito Além do Zero is relevant here because a zero result can represent strong control, low exposure, or a reporting system that has learned to protect the number. The review should distinguish those possibilities instead of celebrating the result automatically.

Step 6: Compare the measure with field evidence

Pair the metric with at least 2 independent sources from the work, such as maintenance history, control verification, worker concerns, permit quality, or repeat exposure. A measure that says green while field evidence says unstable needs investigation before it receives more authority.

Choose one recent high-risk task and compare the dashboard result with what the supervisor, control owner, and worker actually saw. The control-effectiveness tests published by Headline Podcast offer a practical comparison point for this check.

Cam Stevens has made a related point on the Headline Podcast. A safety metric earns trust when it is tied to a real problem and a usable decision, not when its collection process is technically impressive.

Step 7: Decide whether to keep, redesign, or retire

Use the evidence from the first 6 steps to make one of 3 decisions. Keep the metric when its definition is stable, its risk pathway is clear, and its result changes an accountable decision. Redesign it when the signal matters but its timing, denominator, or evidence rule is weak. Retire it when no owner can explain what action the result should trigger.

Write the decision in a one-page record that includes the old measure, the reason for the decision, the replacement measure if one exists, the owner, and the effective date. The record prevents a retired metric from returning through habit during the next quarterly review.

Do not replace every retired measure immediately. Removing one weak metric can create room for a better field verification routine, a maintenance review, or a direct conversation about an unresolved control.

Step 8: Set a 21-day proof period

A redesigned measure needs a short proof period before it becomes part of executive reporting. For 21 days, review whether people can collect it consistently, whether the result reaches the right owner, and whether the resulting decision is visible in the work.

Use 4 proof questions. Was the event classified consistently? Did the signal arrive before the decision window closed? Did the owner act or explain why no action was needed? Can a worker or supervisor point to the change that followed?

At the end of the period, keep the measure only if the answers are clear. This is a better standard than asking whether the dashboard looks complete.

How to run the next executive review

Bring fewer measures and more decision context to the next meeting. For each number, show the risk pathway, the current control condition, the owner, the response deadline, and the evidence that supports the reading.

• Remove measures with no named decision owner.
• Separate activity counts from control-effectiveness evidence.
• Show the latest field example behind a material change.
• Record the action, owner, and due date before closing the review.

The Headline Podcast decision-rights framework can help leaders clarify who interprets a metric, who acts on it, and who can stop work when the signal indicates unacceptable exposure.

A weak metric can remain harmless only while nobody uses it to authorize risky work. Once leaders treat it as proof of control, the cost of keeping it becomes operational.

Andreza Araujo's experience across more than 250 cultural transformation projects supports a practical conclusion. Safety measurement improves when leaders are willing to retire a familiar number and replace it with evidence that changes the work.

The best dashboard is not the one with the most indicators. It is the one that makes the next safe decision easier to see, assign, and verify.

Read more practical safety leadership and risk analysis on Headline Podcast.