A claim such as “98% accuracy” does not help you decide unless you know what was measured, on which data and under which conditions. To know whether a system works on your site, measure two kinds of error separately.
Two kinds of error
| Term | Meaning | Effect on site |
|---|---|---|
| False positive | Alert without a real event | Alarm fatigue; system gets ignored |
| False negative | Real event without an alert | Risk goes unnoticed |
| True positive | Event correctly detected | The actual value of the system |
Precision and recall
- Precision: the share of alerts that are real. Low precision floods the team with false alarms.
- Recall: the share of real events that are caught. Low recall means risk is missed.
There is a trade-off: lowering the threshold catches more events but raises false alarms. The right balance depends on the use case — missing a fire is worse than missing a single PPE violation.
Validating on your own footage
- Select footage from the pilot cameras for a defined period.
- Have people mark the real events.
- Compare system events against those marks.
- Report true detections, false alarms and misses per camera.
- Adjust rules and thresholds, then measure again.
When you see an accuracy figure in a proposal, ask: on what data, under which camera conditions, and how were false alarms and misses measured separately?
Not sure what is feasible on your site? Send a few camera frames and your use case.
Evaluate your camerasFrequently asked questions
Does accuracy change over time?
It can, if camera positions, lighting or processes change. Periodic checks and recalibration are recommended.