Failure-Driven Learning as Model-Based Self-Redesign
Learning is a competence fundamental to intelligence. Intelligent agents who solve problems in a realistic environment need to learn in order to improve their performance in terms of the quality of the solutions they produce, the efficiency of their problem-solving process, and the class of problems they can solve. Failures in problem solving signify the need and the opportunity to learn. One way in which an agent may effectively use its failed problem-solving experiences to learn, is by reflection upon its own problem-solving process. To that end, the agent needs an explicit model of its own problem-solving behavior. This work adopts a design stance towards reflective, failure-driven, learning. This stance gives rise to a specific computational model which is based on three key ideas: (i) agents can be viewed as abstract devices; (ii) their problem solving can be understood in terms of structure-behavior-function (SBF) models; finally, (iii) failure-driven learning can be viewed as a model-based redesign process, in which the agent uses its comprehension of its own problem solving to repair itself. When the agent fails, it uses feedback from the world, and the trace of the failed process, to search through this model and identify the cause(s) of its failure. Then, it proceeds to repair its problem solving, in order not to fail again for the same reason.This theory of reflective learning has been implemented in a fully operational system, AUTOGNOSTIC. AUTOGNOSTIC is like a "shell" in that it provides the SBF language for specifying a problem solver, and the inference mechanism for monitoring this problem-solver's reasoning, assigning blame when it fails, and repairing it appropriately. Three different systems have been modeled in AUTOGNOSTIC's SBF language: ROUTER, a path planning system, KRITIK2, a design system, and an autonomous, reactive agent implemented in the AuRA architecture. Extensive experiments conducted with AUTOGNOSTIC demonstrate the generality and the effectiveness of its learning process.