ACDS: Adapting Computational Data Streams for High Performance
MetadataShow full item record
Data-intensive, interactive applications are an important class of metacomputing (Grid) applications. They are characterized by large data flows between data providers and consumers, like scientific simulations and remote visualization clients of simulation output. Such data flows vary at runtime, due to changes in consumers' data needs, changes in the nature of the data being transmitted, or changes in the availability of computing resources used by flows. The topic of this paper is the runtime adaptation of data streams, in response to changes in resource availability and/or in end user requirements, with the goal of continually providing to consumers data at the levels of quality they require. Our approach is one that associates computational objects with data streams. These objects offer services like data filtering and transformation. Runtime adaptation is achieved by adjusting objects' actions on streams, by splitting and merging objects, and by migrating them (and the streams on which they operate) across machines and network links. The resulting adaptive computational data streams maintain high performance by responding to changes in the needs of data consumers, as exemplified by variations in the resolution or rate at which they desire to receive data. Adaptive streams also react to changes in resource availability detected by online monitoring. The experimental demonstrations presented in this paper utilize computational data streams emanating from a global atmospheric simulation model and/or from stored model outputs, consumed by visualization clients that display this data. Experiments are performed on heterogeneous cluster machines and visualization clients connected by LAN or WAN networks.