报告摘要

With the development of smart wearable cameras such as Microsoft HoloLens and Google Glass, the egocentric vision and its applications have drawn lots of attention of people from various fields. We are particularly interested in its vision-based interaction with human. The hand and fingertip-based gesture interaction could be the most appropriate approach because it acts like a virtual mouse to which people get used.

The hand and fingertip tracking is the crucial part in the egocentric vision interaction, and it remains a challenging problem due to various factors like dynamic environment and hand deformation. We propose a convolutional neural network (CNN) based method for the real-time and accurate hand tracking and fingertip detection in RGB sequences captured by an egocentric mobile camera. Firstly, we build a large scale dataset, Ego-Finger, containing plenty of scenarios and human labeled ground truth. Secondly, we propose a two stage CNN pipeline, i.e., the human vision inspired Attention-based Hand Tracker (AHT) and the hand physical constrained Multi-Points Fingertip Detector (MFD). Comparing with state-of-the-art methods, the proposed method achieves very promising results in the real-time fashion.

报告人简介简介

张鑫，女，博士，副教授。2003年本科毕业于西北工业大学自动化学院，之后分别于2005年和2011年于美国俄克拉荷马州立大学电子与计算机学院获得硕士和博士学位。2011年9月至今在华南理工大学电子与信息学院任职。主要的研究方向包括人体运动分析，手部运动跟踪，手势识别以及智能人机交互等，目前已发表相关论文数十篇，其中SCI检索近十篇，主持科研项目七项，包括国家自然基金青年基金，教育博士点新教师基金、广东省重点实验室基金，广东省科技计划项目及中央高校业务费等；同时，作为联合主持人负责Microsoft微软亚洲研究院高校关系研究项目中Kinect主题项目和第一视角主体项目。主要学术兼职包括：IEEE会员；IEEE Signal Processing 广州分会秘书，相关主流国际期刊和国际会议的审稿人。