New robotic vision can see ‘invisible’ objects

A group of South Korean researchers is pioneering a new technology that could push the envelope for visual imaging via robotic systems. This means scrap could be more easily and accurately identified in the future.

When artificial intelligence systems encounters a section of material on the belt where objects are not fully visible, it has to make estimations based only on the visible parts of the objects. This partial information leads to detection errors, and large training data is required to correctly recognize such scenes.

The Gwangju Institute of Science and Technology believes it has found the answer to this: a framework that allows robot vision to detect overlapping or totally ‘hidden’ objects in the same way they are perceived by human eyes.

Gwangju’s Associate Professor Kyoobin Lee and PhD student Seunghyeok Back developed a model called “unseen object amodal instance segmentation” (UOAIS) for detecting any material in such cluttered scenes. To train the model in identifying object geometry, they built a database containing 45 000 photorealistic synthetic images containing depth information.

The system was able to positively ID a variety of previously missed objects  by segmenting a variety of objects into a ‘visible mask’ and an’“amodal mask’. The innovative approach saw Lee and Back introduce a hierarchical occlusion modeling (HOM) scheme, which assigned a hierarchy to the combination of multiple extracted features and their prediction order.

By testing their model against three benchmarks, they validated the effectiveness of the HOM scheme, which achieved ‘state-of-the-art performance,’ the researchers claim. They will present their findings at the upcoming IEEE International Conference on Robotics and Automation in Philadelphia.

Previous methods are limited to either detecting only specific types of objects or detecting only the visible regions without explicitly reasoning over occluded areas. By contrast, our method can infer the hidden regions of occluded objects like a human vision system. This enables a reduction in data collection efforts while improving performance in a complex environment,’ Back notes.