See Smarter: How sensor fusion is powering the next-gen object recognition
See Smarter: How sensor fusion is powering the next-gen object recognition
Business Overview
A leading mobility solutions provider wanted to build a robust object localization and recognition system by combining the data from camera and radar, leveraging the strengths of both sensors for higher accuracy in dynamic environments. Their goal was to enhance its situational awareness in their ADAS system applications.
The whole project involved full-stack development from hardware design to perception-level intelligence, enabling real-time object classification with key attributes like position, velocity, orientation, and vehicle type.
Challenge
While radar sensors provided reliable distance and velocity information, they lacked classification detail. Cameras, on the other hand, offered rich visual context but struggled in low-visibility conditions during fog, rain, as well as night.
Key pain points included:
- Inconsistent object detection accuracy across varying environments
- High false alerts were seen in the standalone camera & radar solutions
- Need for low-latency data processing on edge devices
- Integrating diverse sensor data into a single, coherent perception model
Solution
NeST Digital engineered a Sensor Fusion system combining the radar and camera data streams, built on a high-performance edge platform.
Our team along with the SMEs built a smart and reliable system by blending the strengths of both camera and radar technologies. Cameras provided visual details, helping the system recognize whether an object was a car, pedestrian, or something else, while RADAR gave precise information on how far the object was and how fast it was moving.
The “2 senses” were brought together through a technique called sensor fusion, enabling the system to understand not just what the object is, but also where it is, how fast it’s going, and which direction it was heading to.
The team using ROS on NVDIA platform implemented this solution and updated the firmware of the radar sensor to improve its responsiveness and accuracy.
On the software side, smart algorithms were programmed, that could learn from real-world scenarios, recognize patterns, and make fast decisions—instantly and reliably. All of this was designed to run smoothly on a high-performance computing platform at the edge, ensuring real-time operation without delay. Information regarding the vehicle being driven such as, speed was used to analyze chances of collision from other vehicles using sensor fusion.
Key Features
- Multi-object tracking with real-time localization
- Velocity estimation from RADAR, visual classification from camera
- Orientation prediction and object-type tagging
- Fusion approach with weighted confidence boosting hit rate
Impact
- The sensor fusion approach led to a 14% improvement in object detection accuracy, even in challenging environmental conditions.
- False alert notifications were reduced by over 60%, significantly boosting system reliability and trust in real-time applications.
- Object classification for identifying vehicle types, rose from 65% to 88%, enabling more accurate decision-making.
- The system also demonstrated consistent performance in low-visibility scenarios like fog and night time, validating the robustness of the fusion model.
SHARE
