Eliminating Confusion within your SMS – SPIs, SPTs vs KPIs

For those operators within IBAC’s IS-BAO and IS-BAH programs who are voluntarily developing and implementing a Safety Management System (SMS) to move above mere regulatory requirements, IBAC standard enhancements over the past year or so have directed more attention to the “data-driven” portion of SMS performance. Confusion, however, exists when operators try to apply commercial measures rather than “safety” measures – those that have a direct and indirect impact on your operation’s safety performance.

We are focusing on safety performance, and many people confuse safety performance with quality performance and Key Performance Indicators (KPIs). In fact, there is a difference, and it is a big one. We are not focusing on typical KPIs such as customer service standards, financial measures, on-time performance, time to greet a customer, phone calls answered before the third ring, customer comment card submissions, and so forth. Some KPIs can flow over to Safety Performance Indicators (SPIs), like in the example of maintaining proper temperature of the catering holding refrigerators or the cleanliness of bathrooms and floors. A negative performance here can result in illness and injuries.

Our experience has shown that most organizations have had a fair number of the twelve¹ core elements of the SMS in place, in some form, but maybe lacking structure and consistent education and communication to and amongst all employees and management staff. For the purposes of this article, we are focusing on SPIs, Safety Performance Targets (SPTs), and the monitoring of these metrics.

Without the use of performance data, we can only get a subjective sense of how well our safety program is performing. We can look out and “see” certain elements of our safety program functions such as procedural compliance, hazard identification, training and education, and safety communication, but how can we determine the overall functionality of our safety performance?

Management benefits from timely and factual data provided in a clear, concise, and easily understood format. Executives need to see the benefit of their efforts and often use a set of data points to judge the direction of their efforts. Likewise, for executives looking to understand the impact of their efforts on assuring safe processes, environment, and workplace, relying on a signed safety policy alone is pure folly. Equally, safety managers need to present safety performance data to validate their efforts or the need for resources from management. All of this can be easily presented in a dashboard view of SPIs and SPTs.

The solution is data collection, analysis, and reporting! This means facilitating the act of data collection from several of the core elements or ”spokes” that, with analysis and reporting, acts as the outer rim of the wheel and consequently enables the overall wheel to perform and move forward. Without effective data collection, analysis, and reporting, you have a program whose performance is unpredictable.

What we have learned from experience, is that sometimes history repeats itself. Past performance may then be an indicator of future performance if conditions and controls remain unchecked. By looking at past performance, you can visualize trends and better predict your organization’s future performance.

Let’s understand some common terms:

• Safety Performance is expressed by SPIs and their corresponding alert and target values.
• Target and Alert settings should take into consideration recent historical performance for a given indicator.
• Lagging Indicators are metrics that measure safety events that have already occurred, such as ramp accidents, hangar rash events, loss incidents, and OSHA type statistics like injuries, lost time injuries, first aid, and so forth.
• Leading Indicators are metrics that provide information on the current state of affairs, which may impact future performance and can be either negative or positive indicators.
  – Positive Indicators can be the percentage of Near Miss Reports
  – Negative Indicators can be SOP non-conformance, practical drift observations, Procedural Intentional Non-Conformance (PINC), etc.
  – High Consequence: accidents, injuries, bent metal
  – Low Consequence: delays, potential safety implications

Ideally, the traditional FBO would use a mix of both lagging and leading indicators with a combination of both high and low consequence. The indicators would be based on the risk profile of the operation and overall enterprise, providing a mix of corporate-level high-profile events such as accidents and workers’ compensation lost time injuries, and local operation-level events such as SOP conformance, safety committee participation, GSE pre-operations check performance, and so on.

Indicators and target performance would be reviewed by the safety manager and safety action group or review board on a routine basis and shared with senior leadership on an equally frequent basis either by direct participation or scheduled briefing to the executive on the operation’s safety performance.

For the SMS to properly perform, senior leadership (the accountable executive) and the organization’s team would be directly engaged in all aspects of the SMS performance and management review. Determining where additional resources, budget, staffing and so on, need to be applied are key components to managing performance in the right direction.

At the end of the program year, the SMS’ performance would be reviewed and analyzed for overall effectiveness. New indicators and targets may be identified and set at the direction of the accountable executive, enabling planning (management and budgetary) to re-align objectives and goals for the new year.

The bottom line is SPIs and SPTs are management’s best tools for determining the overall safety health of the organization, while providing an effective way to predict future performance. Without them, you’re just hoping for the best. Or, as famously said, “Do you feel lucky? Well, do ya?”

References

¹ ICAO SMM Document 9859: Management Commitment & Responsibility, Safety Accountabilities, Appoint of Key Safety Personnel. Coordination of ERP, SMS documentation, Hazard Identification, Safety Risk Assessment & Mitigation, Safety Performance Monitoring, Management of Change, Continuous Improvement, Training & Education and Safety Communication.