Rapid haptic perception using force and thermal sensing

Abstract

Tactile sensing can enable a robot to infer properties of its surroundings. Recent research has focused on robots that haptically perceive the world through exploratory behaviors that occur over tens of seconds. During manipulation, many opportunities arise for robots to gather information about the environment from brief (<= 2 seconds) contact due to simple motions (e.g., linear). The goal of our work was to enable robots to infer haptic properties under these conditions using force and thermal sensing. We used a data-driven approach with various machine learning methods. Key challenges were obtaining adequate haptic data for training and developing methods that performed well on haptic data that differed from the training data due to common real-world phenomena. For example, haptic sensory signals vary significantly due to the robot, including its velocity, stiffness, and sensor temperature. To collect suitable data, we used a variety of platforms, including simplified robots, handheld human-operated devices, and a mobile robot. We also generated synthetic data with physics-based models. Through careful empirical evaluation, we identified machine learning methods that better handled common signal variations. We also used physics-based models to characterize common perceptual ambiguities and predict the performance of data-driven methods. Overall, our research demonstrates the feasibility of robots inferring haptic properties from brief contact with objects in human environments. By using force and thermal sensing, our methods rapidly recognized materials, detected when objects moved, detected contact with people, and inferred other properties of the robot’s surroundings.