The Next Step in Smart Plant: Excavators that Record and Analyse Near Misses

Key Takeaways

• AI extracts near‑miss video clips from excavator footage
• NEC’s generative AI creates detailed incident reports automatically
• System slated for market launch in FY 2027/28
• Future versions will detect hidden site hazards beyond operator view
• Collaboration leverages SHI’s equipment data and NEC’s vision tech

Pulse Analysis

The construction sector has long grappled with high injury rates, and near‑miss incidents are a goldmine of untapped safety data. By automatically pulling risk scenes from hydraulic excavator cameras, Sumitomo Heavy Industries and NEC turn routine footage into actionable intelligence. This approach mirrors trends in autonomous vehicle telemetry, where continuous video streams are mined for anomalies. Instead of waiting for an accident to trigger an investigation, the system surfaces hazardous moments in real time, giving managers a proactive tool to tighten safety protocols before a loss occurs.

The backbone of the solution is a two‑stage AI pipeline. First, an extraction model trained on SHI’s extensive excavator datasets identifies frames that contain potential safety breaches, leveraging both visual cues and hydraulic sensor signatures. Those clips are then fed into NEC’s proprietary multimodal engine, which fuses video, temporal, and spatial data to generate a structured narrative via generative AI. The resulting report includes a concise description, root‑cause hypotheses, and recommended corrective actions, all stored in a searchable database that can be cross‑referenced with site‑specific safety rules.

Commercial rollout is slated for fiscal 2027/28, positioning the partnership at the forefront of what analysts call ‘smart plant’ ecosystems. Early adopters stand to lower insurance premiums and reduce downtime by catching unsafe patterns before they evolve into costly incidents. Beyond operator‑plant interactions, the roadmap envisions monitoring environmental factors such as ground instability or unauthorized equipment placement, expanding the safety net across entire sites. As construction firms increasingly digitize workflows, the SHI‑NEC platform could become a benchmark for integrating AI‑driven risk management into legacy heavy‑machinery fleets.