Measuring Change (and what Frank Zappa would say about it)

Walt Rostykus

Early in my scuba diving days I learned a critical lesson about measuring progress. While diving in the dark, cold, and fast moving waters of the Pacific Northwest, I learned not to trust just what I saw, but also what was measured.

Watching the gauge for depth and compass for direction was essential for ensuring we were moving in the right direction and at the correct pace and depth to reach our objective. This practice of checking gauges (measures) and comparing the two points of (1) where you were and (2) where you are now, to verify progress and direction, has served me well on road trips, while hiking in the back country and in managing  environment, health and safety (EHS) processes.  Comparing metrics to verify improvement is a critical element of any environment or safety management system. But I am still amazed at how many organizations do not apply this practice to individual workplace improvements as part of their safety and ergonomics improvement process.

Milton Friedman, American Economist and Nobel Prize Winner, stated it clearly: “The only relevant test of the validity of a hypothesis is comparison of prediction with experience.” This simple act of comparing two points of data to validate change is a core element of EHS management system and continuous improvement process.

During Humantech’s recent benchmarking study of ergonomics program/process management, we explored if and how organizations verify the effectiveness of workplace changes and improvements. We found that:

• Only 59 percent conducted formal follow-up assessments (reassessments) using the same risk assessment tool used in the initial “ergonomic” assessment. Of these,
  • 80 percent used quantitative assessment methods, allowing them to compare “before” and “after” scores to verify that the improvement reduced the level of risk. These were predominately programs in the ‘proactive’ and ‘advanced’ levels of maturity.
  • 41 percent interview or survey employees as their only or a supplementary method. These were predominately programs in the ‘reactive’ level of maturity.
• 23 percent use a lagging or activity-based method.  These included tracking reduction of injury and reviewing project improvement records.
• 18 percent of participants do not conduct follow-up assessments.

The benchmarking study looked at the level of maturity and effectiveness of each ergonomics process and found that all of the highest performers included comparison of before and after metrics at both the program (strategic) level, and at the tactical (workplace improvement) level.

Frank Zappa summarized it well when he said, “Without deviation, progress is not possible.”

I’d appreciate hearing from you about your experience and methods for validating improvement of EHS and/or ergonomics programs and conditions.

• Do you compare before and after, or trust that change happens?
• Are you confident of the direction and magnitude of change?
• How do you measure change? By activity or results? With lagging or leading indicators?
• What challenges have you encountered?
• What best practices have you learned?

Walt Rostykus is a vice president and consultant with Humantech Inc., a consulting firm that combines the science of ergonomics with their unique 30-Inch View^® – where people, work, and environment intersect–to deliver practical solutions that impact safety, quality, and productivity.  When he is not travelling for work, Walt resides in New Mexico enjoying the great outdoors.