Clearwater: An Extensible, Pliable, and Customizable Approach to Code Generation

Abstract

Since the advent of RPC Stub Generator, software tools that translate a high level specification into executable programs have been instrumental in facilitating the development of distributed software systems. Developers write programs at a high level abstraction with high readability and reduced initial development cost. However, existing approaches to building code generation tools for such systems have difficulties evolving these tools to meet challenges of new standards, new platforms and languages, or changing product scopes, resulting in generator tools with limited lifespan. The difficulties in evolving generator tools can be characterized as a combination of three challenges that appear inherently difficult to solve simultaneously: the abstraction mapping challenge translating a high-level abstraction into a low-level implementation), the interoperable heterogeneity challenge stemming from multiple input and output formats, and the flexible customization challenge to extend base functionality for evolution or new applications. The Clearwater approach to code generation uses XML-based technologies and software tools to resolve these three challenges with three important code generation features: specification extensibility, whereby an existing specification format can accommodate extensions or variations at low cost; generator pliability, which allows the generator to operator on an extensible specification and/or support multiple and new platforms; and flexible customization, which allows an application developer to make controlled changes to the output of a code generator to support application-specific goals. The presentation will outline the Clearwater approach and apply it to meet the above three challenges in two domain areas. The first area is information flow applications (e.g., multimedia streaming and event processing), a horizontal domain in which the ISG code generator creates QoS-customized communication code using the Infopipe abstraction and specification language. The second area is enterprise application staging (e.g., complex N-tier distributed applications), a vertical domain in which the Mulini code generator creates multiple types of source code supporting automatic staging of distributed heterogeneous applications in a data center environment. The success of applying Clearwater to these domains shows the effectiveness of our approach.