Characterizing Network Infrastructure Using the Domain Name System

Abstract

From the early 90’s until the recent years we have seen a significant amount of protocols and applications being built on top of the Internet Protocol (IP). The ever growing use of off-the-shelf solutions and vertically integrated software is quickly transforming the Inter- net to an end-to-end encrypted network. This creates a great burden on security applications and the security industry as a whole, which rely on techniques like Deep Packet Inspection (DPI) to secure networks. However, the Domain Name System (DNS), the Internet’s phone book, is still available to the security community for both research and applied security. At the same time, DNS monitoring is less invasive, since it is separate from applications using it, preserving the privacy level encryption attempts to set. Hence, DNS is expected to be available to security applications for the foreseeable future and can still be used to reason about the IP even though encryption may make the underlying data unavailable to network security solutions.

This thesis shows how to actively query domain names in order to assist in detecting security threats and provide context around Internet Protocol addresses. Specifically, it introduces the Active DNS data, a public dataset that maps almost 70% of the registered domain names to IP addresses from 75% of the Top Level Domains (TLDs) in an active and scalable fashion, as an alternative to extensively used passive DNS datasets. Moreover, this thesis, describes problems faced after operating the Active DNS data generation system for almost five years and how architectural changes improved system availability, reliability, and scalability. Finally, it demonstrates the value in the Active DNS data by performing the first large scale study of Combosquatting, an attack technique that utilizes over 2.1M domain names, resolved more than 10B times per day, and attempts to hide malicious activity in at least seven different types of online abuse.