Object handover is a common task arising frequently in many cooperative scenarios. Therefore, it is crucial that robots perform handovers well when working with people. However, determining the proper handover method for an object is a difficult problem since it varies depending on each object’s affordances. Towards enabling effective human-robot cooperation, this thesis contributes a framework that enables robots to automatically determine handover methods for various objects by observing human handovers and object usages.
This thesis first documents a user study conducted to characterize and compare the handover orientations used by humans in different conditions. It puts forth the novel idea of object affordance axes for identifying patterns in handover orientations, and a distance minimizing method for computing mean handover orientation from a set of observations.
Next, this thesis presents an object grouping and classification method based on observed object usage for generalizing learned handover methods to new objects. Until now, a demonstrated method for generalizing handover methods to new object has been lacking. The presented method focuses on a set of action features extracted from the movement patterns and inter-object interactions observed during usage. An experiment demonstrates the effectiveness of the method on grouping objects and then classifying new objects and computing proper handover methods for them.
The described framework for learning and generalizing handover methods is implemented onto a Kawada Industries HRP2V robot, and this thesis also documents the verification experiments. The implementation in this thesis overcomes the robot perception challenge of identifying a held object’s pose at handover by detecting the object at the pre-occluded state and tracking its pose using a sequential Monte Carlo method. Results show that the framework allows robots to learn handover methods from demonstrations and compute proper handover methods for new objects. This is the first demonstrated system capable of automatically learning and generalizing handover methods from observations. Finally, integration into a household service robot application shows how this work this can enhance the capabilities of robots working in the real world by enabling them to work effectively with humans.
Through enabling better human-robot object handovers, this thesis contributes towards improving the interaction between humans and robots, thus, allowing safer, more natural, and more efficient human-robot cooperation.
Handing over objects is a common basic task that arises between people in many cooperative scenarios. On a daily basis, we effortlessly and successfully perform countless unscripted handovers without any explicit communication. However, handing over an object to a person is a challenging task for robotic “hands”, and the resulting interaction is often unnatural. To improve human-robot cooperation, the work described in this thesis has led to the design of a human-inspired handover controller based on analysis and characterization of the haptic interaction during human-to-human object handover.
The first experiment in this thesis documents novel experimental work done to measure the dynamic interaction in human-human handovers. The grip forces and load forces experienced by the giver and the receiver during a handover are examined, and the key features are identified. Based on these experimental results, guidelines for designing human- robot handovers are proposed. Next, this thesis describes a handover controller model that enables robots to hand over objects to people in a safe, efficient, and intuitive manner, and an implementation of the handover controller on a Willow Garage PR2 robot is documented. Finally, a second experiment is presented, which compares various tunings of the novel controller in a user study. Results show that the novel controller yields more efficient and more intuitive robot-to-human handovers when compared to existing handover controllers.
Krishneel C. Chaudhary, Xiangyu Chen, Wesley P. Chan, Kei Okada, and Masayuki Inaba. STAIR3D: Simultaneous Tracking and Incremental Registration for Modeling 3D Handheld Objects. In Proceedings of the 2017 IEEE International Conference on Advanced Intelligent Mechatronics (AIM 2017). pp. 185-192. 2017. (Best Student Paper Award)
Matthew K.X.J. Pan, Vidar Skjervøy, Wesley P. Chan, Masayuki Inaba, and Elizabeth A. Croft. Automated detection of handovers using kinematic features. International Journal of Robotics Research. Volume 36. Issue 5-7. pp. 721-738. 2016.
Krishneel C. Chaudhary, Chi Wun Au, Wesley P. Chan, Kotaro Nagahama, Hiroaki Yaguchi, Kei Okada, and Masayuki Inaba. Retrieving Unknown Objects Using Robot In-The-Loop Based Interactive Segmentation. In Proceedings of the 2016 IEEE/SICE International Symposium on System Integration (SII 2016). 2016.
Xiangyu Chen, Kohei Kimura, Hiroto Mizohana, Moju Zhao, Krishneel C. Chaudhary, Wesley P. Chan, Shunichi Nozawa, Yohei Kakiuchi, Kei Okada, and Masayuki Inaba. Development of Task-oriented High Power Field Robot Platform with Humanoid Upper Body and Mobile Wheeled Base. In Proceedings of the 2016 IEEE/SICE International Symposium on System Integration (SII 2016). 2016.
Hiroto Mizohana, Wesley P. Chan, Kohei Kimura, Xiangyu Chen, Kei Okada, and Masayuki Inaba. Object Recognition Considering Shadow from Environment and Self Body for Outdoor Tool picking and Manipulation Task by Humanoid with Movable Vehicle. In Proceedings of the 17th SICE System Integration Division Annual Conference (SI 2016). 2016.
Wesley P. Chan, Masayuki Inaba. Status-Based Behavior Selection for Creating Culturally Aware Robots. In Proceedings of the 3rd International Workshop on Culture Aware Robotics (CARs 2015). 2015.
Wesley P. Chan, Kotaro Nagahama, Hiroaki Yaguchi, Yohei Kakiuchi, Kei Okada, Masayuki Inaba. Implementation of a Framework for Learning Handover Grasp Configurations through Observation during Human-Robot Object Handovers. In Proceedings of the 15th IEEE RAS International Conference on Humanoid Robots (Humanoids 2015). 2015.
Hiroaki Yaguchi, Kazuhiro Sasabuchi, Wesley P. Chan, Kotaro Nagahama, Takuya Saiki, Yasuto Shiigi, Masayuki Inaba. A Design of 4-Legged Semi Humanoid Robot Aero for Disaster Response Task. In Proceedings of the 15th IEEE RAS International Conference on Humanoid Robots (Humanoids 2015). 2015.
Wesley P. Chan, Matthew K.X.J. Pan, Elizabeth A. Croft, and Masayuki Inaba. Characterization of Handover Orientations used by Humans for Efficient Robot to Human Handovers. In Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2015). pp. 1-6. 2015.
Justin W. Hart, Sara Sheikholeslami, Matthew K.X.J. Pan, Wesley P. Chan, and Elizabeth A. Croft. Predictions of Human Task Performance and Handover Trajectories for Human-Robot Interaction. In Proceedings of the 2015 Human-Robot Interaction Workshop on Human-Robot Teaming. 2015.
Wesley P. Chan, Yohei Kakiuchi, Kei Okada, Masayuki Inaba. Determining Proper Grasp Configurations for Handovers through Observation of Object Movement Patterns and Inter-object Interactions during Usage. In Proceedings of the 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014). pp. 1355 – 1360. 2014.
Wesley P. Chan, Iori Kumagai, Shunichi Nozawa, Yohei Kakiuchi, Kei Okada, Masayuki Inaba. Implementation of a Robot-Human Object Handover Controller on a Compliant Underactuated Hand Using Joint Position Error Measurements for Grip Force and Load Force Estimation. In Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA 2014). pp. 1190-1195. 2014.
Wesley P. Chan, Iori Kumagai, Shunichi Nozawa, Yohei Kakiuchi, Kei Okada, Masayuki Inaba. Creating Socially Acceptable Robots: Learning Grasp Configurations for Object Handovers from Demonstrations. In Proceedings of the Workshop on Advanced Robotics and its Social Impacts (ARSO 2013). pp. 94-99. 2013.
Wesley P. Chan, Iori Kumagai, Shunichi Nozawa, Yohei Kakiuchi, Kei Okada, Masayuki Inaba. Enabling Effective Object Handovers between Humans and Robots Through the Use of Shared Autonomy for Physical Human-Robot Interaction. In Proceedings of the 31st Annual Conference on Robotics Society of Japan. 2S1-05. 2013.
Wesley P. Chan, Chris A. C. Parker, H. F. Machiel Van der Loos, Elizabeth A. Croft. A Human-Inspired Object Handover Controller. International Journal of Robotics Research. Volume 32. Issue 8. pp. 971-983. 2013.
Daniel M. Troniak, Junaed Sattar, Wesley P. Chan, Ergun Calisgan, Ankur Gupta, James J. Little, Elizabeth Croft, Machiel Van der Loos. Charlie Rides the Elevator–Integrating Vision, Navigation and Manipulation Towards Multi-Floor Robot Locomotion. In Proceedings of the 10th Conference on Computer and Robot Vision. 2013.
Wesley P. Chan, Chris A. C. Parker, H. F. Machiel Van der Loos, Elizabeth A. Croft. Grip Forces and Load Forces in Handovers: Implications for Designing Human-Robot Handover Controllers. In Proceedings of the 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI 2012). pp. 9-16. 2012.
Wesley P. Chan, Chris A. C. Parker, H. F. Machiel Van der Loos, Elizabeth A. Croft. Teaching Robots How to Share: Grip Forces and Load Forces in Handovers. Abstract in Proceedings of the 2012 Human-Robot Interaction Pioneers Workshop (HRI Pioneers 2012). pp. 42-43. 2012. (Poster presentation).
Ankur Gupta, Wesley P. Chan, Daniel Troniak, Ergun Calisgan, Parnian Alimi, Amir Haddadi, Koosha Khalvati, Sina Radmard, Nathan Tomer. PR2 Rides the Elevator. AAAI’12 AI and Robotics Multimedia Fair. 2012. (Video and poster presentation).