Assessing Program Visualization Systems as Instructional Aids

Abstract

Recently, program visualization systems have received much attention as learning tools and as software understanding aids. How to evaluate these systems, however, is an open and unexplored area. In order to determine what factors may be important, we conducted an exploratory study using XTango, an algorithm animation system. First, we asked professors to complete surveys intended to solicit information regarding current practices in the teaching of algorithms. Next, we observed two groups of students: one group received a handout and viewed an animation of the Shellsort algorithm, the other received the same handout and listened to a lecture featuring drawings on the blackboard. The students were queried on their understanding of the sort and their impressions of the animation system. Comments indicated a high perceived value for the system, with most students favoring its use as a teaching tool. It was clear from students' responses that an algorithm animation system can be used more effectively as a supplement in the classroom environment than as a substitute for the teacher. The results of this study identified changes to the animation system that will help integrate it into the classroom environment, and provided several important factors to consider in future empirical studies.