The ISE department congratulates Ashis Banerjee on winning a highly competitive 2019 Amazon Research Award. The Amazon Research Awards help fund outstanding, innovative research proposals across machine learning, robotics, operations research, and other areas. Ashis’s project is entitled, “Sparse, Deep and Persistent Visual Features Based 3D Object Detection and 6D Pose Estimation in Indoor Environments.”

Says Ashis, “Autonomous mobile robots are being increasingly used in smart warehouses to enhance efficiency, minimize waste, and ensure safe operations. The robots are invariably capable of certain real-time functionalities such as multi-modal perception, simultaneous localization and mapping, collision-free navigation, and precise manipulation. In the recent past, machine learning has played a central role in enabling all these functionalities. However, the generalizability of the learning models remains a key challenge, where the models have to be mostly re-trained for any changes in the environment, task list, or robot hardware.

Schematic illustration of generalizable visual perception framework in indoor environments

“In this project, we are working toward addressing this challenge by identifying a sparse set of generalizable features that can be used by multiple learning models across different problem scenarios. Specifically, we are combining deep learning with sparse sampling and topological persistence to compute features from multi-channel RGB images. These images enable accurate detection and pose estimation of indoor objects in a manner that is robust to scene appearance variations. We are also evaluating the usefulness of the developed methods on ground robots equipped with stereo vision and powerful GPU-based on-board processors.”

The award “boosts my research program,” Ashis says, “by providing no strings-attached funding to support novel work on generalizable robot perception. I am particularly excited at the prospect that this project will provide a foundation for realizing long-term robot autonomy in all kinds of indoor environments, from warehouse centers and manufacturing plants to assisted living homes.”