Show simple item record

dc.contributor.authorWang, Pingen_US
dc.date.accessioned2010-09-15T18:28:24Z
dc.date.available2010-09-15T18:28:24Z
dc.date.issued2010-06-29en_US
dc.identifier.urihttp://hdl.handle.net/1853/34673
dc.description.abstractParent-child social games, such as peek-a-boo and patty-cake, are a key element of an infant's earliest social interactions. The analysis of children's behaviors in social games based on video recordings provides a means for psychologists to study their social and cognitive development. However, the current practice in the use of video for behavioral research is extremely labor-intensive, involving many hours spent extracting and coding relevant video clips from a large corpus. From the standpoint of computer vision, such real-world video collections pose significant challenges in the automatic analysis of behavior, such as cluttered backgrounds, the effect of varying camera angles, clothing, subject appearance and lighting. These observations motivate my thesis work - automatic retrieval of social games from unstructured videos. The goal of this work is both to help accelerate the research progress in behavioral science and to take the initial steps towards the analysis of natural human interactions in natural settings. Social games are characterized by repetitions of turn-taking interactions between the parent and the child, with variations that are recognizable by both of them. I developed a computational model for social games that exploits the temporal structure over a long time-scale window as quasi-periodic patterns in a time series. I presented an unsupervised algorithm that mines the quasi-periodic patterns from videos. The algorithm consists of two functional modules: converting image sequences into discrete symbolic sequences and mining quasi-periodic patterns from the symbolic sequences. When this technique is applied to video of social games, the extracted quasi-periodic patterns often correspond to meaningful stages of the games. The retrieval performance on unstructured, lab-recorded videos and real-world family movies is promising. Building on this work, I developed a new feature extraction algorithm for social game categorization. Given a quasi-periodic pattern representation, my method automatically selects the most relevant space-time interest points to construct the feature representation. Our experiments demonstrate very promising classification performance on social games collected from YouTube. In addition, the method can also be used to categorize TV videos of sports rallies, demonstrating the generality of this approach. In order to support and encourage more research on human behavior analysis in realistic contexts, a video database of realistic child play in natural settings has been collected and is published on our project website (http://www.cc.gatech.edu/cpl/projects/socialgames), along with annotations. The unsupervised quasi-periodic pattern mining method represents a substantial generalization of conventional periodic motion analysis. Its generality is evaluated by retrieving motions of a range of quasi-periodicity from unstructured videos. The performance was compared with that of a periodic motion detection method based on motion self-similarity. Our method demonstrates superior retrieval performance with a 100% precision when the recall is up to 92.04%, with much fewer parameters than that of the other method.en_US
dc.publisherGeorgia Institute of Technologyen_US
dc.subjectActivity retrievalen_US
dc.subjectPattern miningen_US
dc.subjectQuasi-periodic motion analysisen_US
dc.subjectSocial interaction retrievalen_US
dc.subject.lcshBehavioral assessment
dc.subject.lcshAutism
dc.subject.lcshAutism Diagnosis
dc.subject.lcshDigital media
dc.subject.lcshInformation science
dc.titleSocial game retrieval from unstructured videosen_US
dc.typeDissertationen_US
dc.description.degreePh.D.en_US
dc.contributor.departmentComputingen_US
dc.description.advisorCommittee Chair: Rehg, James; Committee Co-Chair: Abowd, Gregory; Committee Member: Bobick, Aaron; Committee Member: Starner, Thad; Committee Member: Sukthankar, Rahulen_US


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record