In any group of Mobile Elevating Work Platform (MEWP) operators, there will be highly proficient operators as well as those who are less proficient – potentially even posing a safety risk on the jobsite. Understanding who these operators are is critical to maximizing safety and efficiency, yet most site managers are left using subjective observation and anecdotes to determine their workforce’s MEWP proficiency. This becomes even more challenging with large crews of dozens or hundreds of people working at height.
Understanding proficiency through data
The Serious Labs MEWP VR Simulator was developed as a tool to gather objective data on operators’ MEWP proficiency levels. This data enables never-before-seen insights into which operators are highly proficient and which operators’ low proficiency may pose a safety risk. Even more importantly, the simulator is designed to zero in on areas of low proficiency (e.g. safety checks or control selection and function) and improve them through targeted training.
Low-proficiency operators: What’s the risk?
The vast majority of MEWP accidents occur when low-proficiency operators carry out work at height and cause incidents as a result. Their proficiency might be low because they’re inexperienced, lack equipment familiarity, they’re poorly trained, or because it’s been a while since they used a MEWP. The average MEWP incident costs CA$40,000, and there are thousands of incidents each year. In the airline sector, damage to aircraft means the average cost of an accident is much higher. In addition, each year there are around 70 to 80 fatalities involving MEWPs.
In 2020, Serious Labs partnered with the Alberta Construction Training Institute (ACTI) and Building Trades of Alberta (BTA), to assess 133 operators drawn from a wide cross-section of trades using the MEWP VR Simulator.
These operators all participated in MEWP 3a/3b (i.e. scissor and boom lift) practical evaluations on the simulator. All operators were overseen by evaluation on a screen from outside the simulator. Each session took about 30 minutes.
Based on their MEWP performance in safety (e.g. safety checks and hazard avoidance) placement, and control use), the operators of 100.
This is Serious Labs’ ESP Scoring™ system, designed to roll up dozens of data points into one easy-to-interpret number.
ESP Scoring™ categorizes operator’s proficiency based on their score: Good (80 and above), Average (between 50 and 79), or Poor (below 50).
Results:
The study found that:
- 49.6% (66) of the operators had Good proficiency (ESP of 80 or more)
- 43.6% (58) of the operators had Average proficiency (ESP between 50 and 79)
- 6.8% (9) of the operators had Poor proficiency (ESP of less than 50)
The study also tracked how many operators had a potential incident or near miss (defined as driving or elevating without looking in that direction for 5 seconds) or a collision with an object or hazard in the simulation, as follows:
- 6% of the Good operators had a potential incident or a collision
- 67% of the Average operators had a potential incident or a collision
- 88% of the Poor operators had a potential incident or a collision
Findings and Interpretations
- The MEWP VR Simulator was able to assess operators’ proficiency far more accurately than traditional assessment approaches (i.e. observation and anecdote).
- As expected, more proficient operators were much less likely to experience a potential incident or a collision.
- The vast majority of safety incidents resulted from not looking in the direction of travel before driving, elevating, or lowering the MEWP.
- Even Average operators had a high likelihood (67%) of having a potential incident or a collision in a given half-hour session.
- It was possible to assess 133 operators in a relatively short time – approximately 80 hours, with each assessment requiring only about 35 minutes in total.
Applying these findings
Having access to a detailed breakdown of a team’s MEWP proficiency gives site managers several ways to reduce incidents and save costs. Using the MEWP VR Simulator, several industry-leading companies and organizations are already applying these steps:
Assign work tasks based on proficiency: Assign challenging or complex tasks to the most proficient operators, reducing the likelihood of an incident. Operators with lower proficiency can concentrate on ground-based work until they have a chance to upskill.
Some of our customers use a sticker system based on proficiency to help make these work assignments easier on the jobsite.
Provide targeted upskilling for low-proficiency operators: Each MEWP VR Simulator session produces a detailed report including areas to focus on. The simulator also includes practice modules to help upskill in these areas if necessary. Training and safety managers practice modules to help upskill these areas if necessary. Training and safety managers can use this targeted training to efficiently address deficiencies and make operators safer.
Practice for greater efficiency: If operators are consistently taking a long time to complete tasks in the MEWP VR Simulator, this can reflect a lack of confidence or best-practice knowledge. After identifying which operators are taking too long, the simulator can help them work more quickly while maintaining a high standard of safety.
Identify team-wide issues: If entire crews are experiencing challenges in similar areas, this can help identify gaps in previous team training and allow site-wide safety refreshers on these specific topics.
How realistic is the simulator?
Data that is generated in simulation is valid as long as the simulation is an accurate reflection of what it’s meant to simulate. To ensure that outcomes from the MEWP VR Simulator are reflective of actual operator skills, it has been rigorously tested with hundreds of individuals.
A 2021 report from the International Powered Access Federation (IPAF) compared over 100 operators in simulation alongside traditional evaluation on a real MEWP. The report found that the simulator was a 97% match for scores from a skilled, experienced human evaluator, and a strong predictor for behaviours and proficiency levels with a real MEWP. The simulator has also been audited and approved to deliver a range of operator credentials in simulation.
To learn more about the MEWP VR Simulator and how data-driven decision-making can support safety and efficiency for your organization, contact info@seriouslabs.com
