Project

Back to overview

Decentralized and scalable analysis of high-speed IP traffic (DaSAHIT)

English title Decentralized and scalable analysis of high-speed IP traffic (DaSAHIT)
Applicant Stiller Burkhard
Number 118128
Funding scheme Project funding (Div. I-III)
Research institution Institut für Informatik Universität Zürich
Institution of higher education University of Zurich - ZH
Main discipline Information Technology
Start/End 01.01.2008 - 31.12.2009
Approved amount 86'573.00
Show all

Keywords (10)

high-speed accounting of IP traffic; scalability; decentralization; Real-time Traffic Analysis; High-speed Links; Internet Protocol (IP); Scalable Analysis; Robustness; Packet Sampling; Accounting

Lay Summary (English)

Lead
Lay summary
High-speed network links become a challenge to traditional centralized IP traffic analysis due to the high demand of hardware resources which are limited and costly. Thus, the DaSAHIT project (Decentralized and Scalable Analysis of High-speed IP Traffic) will develop a scalable and distributed architecture for collecting, analyzing, and storing IP traffic data with the highest necessary level of accuracy in real-time. The key idea is to utilize resources of a large number of nodes, which collaboratively process and store traffic data in a highly decentralized, flexible, and robust manner. Mechanisms to be developed will provide a high level of self-configuration such that new nodes may be added or removed easily to or from the analysis network. This approach will leverage the resources of cheap, unreliable, and otherwise unused nodes and can reduce the high cost of high-speed traffic analysis.
Although existing work alleviates some of these high-speed packet processing problems related to the high demand of hardware resources, sampling mechanisms are not very effective or accurate in scenarios, where complete information is required or no compromise can be made on the accuracy, such as intrusion detection or usage-based charging. Dedicated network monitoring tasks proposed, like flow processing or detection of flow paths within networks, show the major drawback of a lack of flexibility to adapt to processing workloads due to changes in the current load of the network link to be analyzed and the current processing capacity of the analysis network. They also lack scalability with respect to higher link speeds, mainly due to a limited degree of work distribution.
Therefore, DaSAHIT will develop appropriate self-configuration mechanisms in order to automate the joining and leaving of nodes to or from the analysis network. The resulting monitoring and analysis platform will form the basis for real-time traffic analysis scenarios such as flow accounting, flow path monitoring, or intrusion detection. To show the applicability of the proposed approach, the developed mechanisms will be demonstrated based on the IP flow accounting scenario.
This work is highly relevant to future accounting systems, since it develops alternative mechanisms to packet analysis for high-speed network links. The approach will lead to a better efficiency of core network processes such as routing and switching by removing the burden of packet inspection. At the same time the resulting platform will lead to better analysis results by leveraging processing capabilities of multiple, otherwise unused nodes in order to minimize the sampling rate.
Direct link to Lay Summary Last update: 21.02.2013

Responsible applicant and co-applicants

Employees

Name Institute

-