摘要详情

ID / 提交时间

133 / 2023-09-20 20:54:34

标题

Gesture recognition based on the fusion of millimeter-wave radar and camera

关键字

millimeter-wave radar,camera,deep learning,gesture recognition,sensor fusion technology

主题及专题

Special Sessions > SS11 - Human Activity and Scene Understanding

状态

终稿

作者

Chuang Ren / XiDian university

Liang Zhang / Xidian University

摘要

To overcome the limitations encountered in traditional gesture recognition methods, such as sensitivity to illumination and background interference, this study proposes an integrated approach that combines millimeter wave radar and camera information. By leveraging millimeter wave radar technology, valuable motion information including distance, speed, motion reversal, and hand movement angles can be captured. Concurrently, camera images are employed to extract vision features of gestures. The fusion of these two modalities enables accurate capturing of spatial position, shape, and motion information during the recognition process. A series of experiments was conducted to assess the performance of the proposed method, comparing it with conventional camera-based gesture recognition methods. The experimental results substantiate the significant improvement in recognition accuracy and robustness achieved by the gesture recognition method that integrates millimeter wave radar and camera information.