No More Snake Oil –

Objective Evaluation Environment

for Security Technologies



A project by NUS School of Computing and Insider Security

In collaboration with National Cybersecurity R&D Lab (NCL)

Seed Grant, NCL and Insider Security


Motivation

Cybersecurity awareness has been arising over the years. The emerging market attracts consumers as well as providers. For both parties, evaluation for cybersecurity product become critical to make decision among overwhelmed choices or to showcase the product capability.

The project developed a proof-of-concept testing environment for security technologies.

Achievements

The project adopted the attack-oriented approach, which differs from other approaches in that:
• It focused on product performance by simulating various network-based attack scenarios and techniques, as opposed to Common Criteria compliance-oriented approaches with focus on product development cycle.
• It tested how the product would perform under certain system environment and attacking scenarios, as opposed to product-level evaluation that targets the standalone product.

Users have the flexibility to define the system environment and evaluation criteria. The PoC testing platform provides a virtual enterprise network environment which includes:
• Necessary system and networking elements, e.g., IDS/firewall, servers for web/email/DNS, endpoints with different OSes, etc.
• Tools to generate benign and attack traffic (probe, R2L, U2R, and DoS).

Evaluation report is presented to the user along with score and execution log of each evaluation item, to provide more insights in how the security product would behave under certain settings.

The PoC evaluation platform will be made available to the local cybersecurity community. In the long run, when a number of solutions are showcased in this platform, the collective strength will help to raise awareness of, and build confidence on, the local security companies/researchers and assist them in engaging potential customers, including those from overseas.