Vision-based Contingency Detection

Abstract

We present a novel method for the visual detection of a contingent response by a human to the stimulus of a robot action. Contingency is de ned as a change in an agent's be- havior within a speci c time window in direct response to a signal from another agent; detection of such responses is essential to assess the willingness and interest of a human in interacting with the robot. Using motion-based features to describe the possible contingent action, our approach as- sesses the visual self-similarity of video subsequences cap- tured before the robot exhibits its signaling behavior and statistically models the typical graph-partitioning cost of separating an arbitrary subsequence of frames from the oth- ers. After the behavioral signal, the video is similarly ana- lyzed and the cost of separating the after-signal frames from the before-signal sequences is computed; a lower than typ- ical cost indicates likely contingent reaction. We present a preliminary study in which data were captured and analyzed for algorithmic performance.