Network Data Streaming: Algorithms for Network Measurement and Monitoring
MetadataShow full item record
With the emergence of computer networks as one of the primary modes of communication, and with their adoption for an increasingly wide range of applications, there is a growing need to understand and characterize the traffic they carry. The rise of large scale network attacks adds urgency to this need. However, the large size, high speed and increasing complexity of these networks imply that tracking and characterizing the traffic they carry is an increasingly difficult problem. Dealing with higher level aggregates, such as flows instead of packets, does not solve the problem because these aggregates tend to be quite numerous and exhibit dynamics of their own. In this thesis, we investigate a novel approach to deal with the immense amounts of data associated with problems in network measurement and monitoring. Building upon the paradigm of Data Streaming, which processes a large stream of data using a small working memory to answer a class of queries, we develop an architecture for Network Data Streaming that can accommodate additional constraints imposed in the context of network monitoring. Using this architecture, we design algorithms for monitoring properties of network traffic that have traditionally been considered too difficult to monitor at high speed network links and routers. Our first algorithm provides the ability to accurately estimate the size of individual flows. A second algorithm to estimate the distribution of flow sizes enables network operators to monitor anomalies in the traffic. Incorporating the use of packet sampling, we can extend the latter algorithm to estimate the flow size distribution of arbitrary subpopulations. Finally, we apply the tools of Network Data Streaming to the operation of packet sampling itself. Using the ability to efficiently estimate flow-statistics such as approximate per-flow size, we design a family of mechanisms where the sampling decision is guided by this knowledge. The individual solutions developed in this thesis share a common architectural theme, supporting the monitoring of highly dynamic populations. Integrating this with the traditional sampling based framework for network monitoring will enable a broad range of applications for accurate and comprehensive monitoring of network traffic.
Showing items related by title, author, creator and subject.
Enhancing capabilities of the network data plane using network virtualization and software defined networking Anwer, Muhammad Bilal (Georgia Institute of Technology, 2015-11-13)Enhancement of network data-plane functionality is an open problem that has recently gained momentum. Addition and programmability of new functions inside the network data-plane to enable high speed, complex network ...
Experiences Applying Parallel and Interoperable Network Simulation Techniques in On-line Simulations of Military Networks Perumalla, Kalyan S.; Fujimoto, Richard M.; McLean, Thom; Riley, George F. (Georgia Institute of TechnologyInstitute of Electrical and Electronics Engineers, Inc., New York, 2002)We present a case study in which we apply parallel simulation methods and interoperability techniques to network simulations for simulation-based on-line control of military communication networks. The on-line simulations ...
Yang, Weilai (Georgia Institute of Technology, 2004-04-12)We developed a price-based resource allocation scheme for Differentiated Service (DiffServ) data networks. The DiffServ framework was proposed to provide multiple QoS classes over IP networks. Since the provider supports ...