Multimodal Sensing In The Autonomous Store

Customers are required to wait in line for checking out in the traditional retail store, especially during the rush hour. The autonomous store can automatically identify what item has been taken by whom through large deployment of camera sensing systems.

However, camera just does not work when it cannot see, which is very likely because of the shelf organization (e.g. bottom shelves) or the occlusion from human bodies. Therefore, I am trying to incorporate multiple sensing modalities like RFID, vibration, sound, weight and vision together to achieve a robust and highly autonomous self-checkout system. Please look forward to my recent work. (Hope to submit to conference next February!)

11-09-2024