Benchmarks
 for
DDoS Defense Evaluation


A University of Delaware Subcontract to UCLA





DARPA Homeland Security Grant:  FA8750-05-2-0197 (June 05 to May 07)
Univ. Delaware PI:  Jelena Mirkovic, Computer & Info. Sciences Dept.
UCLA PI:  Peter Reiher , Laboratory for Advanced Systems Research (LASR)


This two-year research effort is headed up by the University of Delaware with subcontracts to UCLA,  Purdue University, and McAfee Research.  The focus of this work is to develop a common evaluation methodology for DDoS defense systems to enable independent evaluations and comparisons.  This methodology will consist of:
  • A benchmark suite that will define all the necessary elements needed to recreate typical DDoS attack scenarios in a testbed setting.
  • A set of performance metrics that expresses a defense system's effectiveness, cost and security.
  • Specification of a testing methodology that provides guidelines on using benchmarks and summarizing and interpreting performance measures.
This basic evaluation methodology will provide means of assessing the ability of a given DDoS defense to defend against today's threats, and the potential damage to a given target network (with or without defense) from these threats.  The benchmark suite will be further enriched with two additional benchmark categories:  future scenarios which will contain sophisticated attack scenarios. and stress-test scenarios which will contain attacks targeting specific critical network resources that are being developed in a related effort.   Further, we will provide tools to update benchmarks as attacks and network-use patterns evolve in the real Internet.

The specific tasks earmarked for UCLA are:
  • Examine internals of numerous networks to generate realistic topology specifications.  We will accomplish this using available tools such as SMW developed at the University of Washington; Internet maps and routing respositories such as Oregon RouteViews, RIPE, CAIDA Skitter; and tools from the PREDICT project.  We will develop a NetProf tool that engages topology-mapping and network-mapping software and summarizes results from the software in a manner that does not  divulge the identiy and detailed internal organization of the mapped network.   We will work with universities, companies and ISPs to apply NetProf on their networks.
  • Develop measures to be used in combination to evaluate DDoS defense systems.  The selection of the measures that best characterize DDoS impact on a network and DDoS defense effectiveness will be our focus.  These can include:  legitimate traffic service level, attack detection, attack response, false positives, deployment cost, operation cost, overall effect and security.