Attivo Networks & MITRE® Engenuity ATT&CK® Evaluations Trials
Attivo Networks Blogs

Attivo Networks Participation in MITRE® Engenuity ATT&CK® Evaluations Trials

MITRE® Engenuity ATT&CK® Evaluations Trials

Authored by: Carolyn Crandall, Chief Security Advocate – MITRE® Engenuity ATT&CK Evaluation Trials for deception are about to begin, and Attivo Networks is excited to announce that we will participate in the research project.

These evaluation trials test best-of-breed security solutions and deliver clarity around their performance and coverage aligned to ATT&CK.

How MITRE® ATT&CK Evaluations Work

ATT&CK evaluations are a collaborative research program where MITRE conducts tailored and focused evaluations for different types of technologies.  MITRE® Engenuity ATT&CK Evaluations launched in 2018 with a focus on the endpoint protection and detection markets. During the evaluations, it became clear that other security solutions delivered value but did not match the criteria for this project. For this reason, MITRE created a new program called ATT&CK Evaluations Trials. The Trials project is different in that they will work with vendors to showcase new methodologies that better capture vendor propositions honestly and transparently. Because of this, each trial will have different objectives, designs, and outputs that will illustrate the benefits of each technology.

MITRE’s team will exercise common ATT&CK techniques to objectively emulate known adversary behavior against vendor capabilities during each trial. In the exercises, they will capture how each vendor uniquely approaches threat detection. They will then publish the results to objectively inform the public on the performance of the tested products.

Why Cyber Deception Evaluation Trials

Cyber deception celebrated the 30th anniversary of honeypots this year and has made leaps and bounds in technology enhancements during the last decade. These enhancements include automated machine-learning deployment, scalability, environments covered, ease of operations, and more. Most noteworthy have been the addition of endpoint deceptions, cloud environment coverage, and data concealment and misdirection technologies that go beyond interweaving decoy objects amongst real ones. This “data cloaks” approach hides and denies access to real assets, including credentials, Active Directory objects, and the files that attackers seek to steal or encrypt. Deception has been on a global boom over the last five years, quietly finding its place amongst large and small customers. Most organizations remain tight-lipped about its usage because it is used extensively for insider threat programs and in the interest of not letting their adversaries know.

In my last six years of working with deception technology, it is safe to say that this is one of the most misunderstood technologies.  It often bears a stigma that it is only for the “rich and famous” and something that a mature security operations team should only do last in their defense-in-depth strategies. I am extremely optimistic about these trials. They will shed massive light on the technology’s value and illustrate use cases that have driven our company’s growth and tremendous customer loyalty. This value can range from easy and high-fidelity detection (common mid-market use case) to in-depth defense and depth strategies applied to combat the most sophisticated nation-state attackers.

How the Evaluation Trials will work

MITRE is constructing a deception methodology that will provide meaningful results to end-users, articulate key differences in vendor product strategies, and do so fairly and openly.

It will seek to answer two main questions:

  • Did the adversary encounter the deception (i.e., could the deception capability affect the adversary)?
  • Did the adversary engage the deception (i.e., did the deception capability affect the adversary)?

Determining whether the adversary encountered deception is pretty straightforward. The evaluation can determine it by running the adversary technique and recording whether it sees something different from a scenario that did not deploy deception.

On the other hand, engagement can be harder to measure because you need to consider the human element. Some of the things that MITRE is factoring in related to this are:

  • Did they engage it out of happenstance, or did they make the conscious decision to pick it because it seemed the better target?
  • Would they have engaged the deception again if they were presented with the same choice again?
  • Would a different tester make the same choice?
  • Would that choice change if they were aware, or not, that there was deception technology in the environment?
  • Was the effect a short-term inconvenience, or did it affect their long-term mission?

Given the diversity of the results, MITRE will need to identify common measures that will allow them to talk about products in a similar language while still understanding each vendor’s unique capabilities and use cases. There are various value propositions within deception, and so for this reason, MITRE is considering this a research project. Some of the areas they will evaluate include:

  • Detection through high confidence tripwires
  • Interaction that keeps the adversary engaged and causes attackers to waste time/resources  

Attivo Networks is proud to have confirmed its participation in this research program. MITRE welcomes other deception vendors to join in and help advance the adoption and understanding of deception capabilities. Interested parties should register by the end of November to accommodate early 2022 execution. Should you wish to participate in the ATT&CK Evaluations Trial program, please contact the MITRE ATT&CK team.

More information on the deception ATT&CK Trials is available at MITRE’s post on the topic.

Gartner, Inc. has listed deception as a priority technology for the last few years, and IDG found it the most researched technology in 2020. Gartner’s last coverage of the technology can be found here, in its 6 Solution Vendor Comparison. We are thrilled MITRE will now independently evaluate deception solutions. We are excited to be a part of this trial and look forward to sharing more as they progress.

Go here for more information on how Attivo maps to MITRE ATT&CK.

About MITRE® Engenuity

MITRE Engenuity is the tech foundation for public good and forms part of the MITRE community. This group of innovators and leaders harnesses MITRE’s 60+ years of R&D in the cybersecurity sector and utilizes the lessons and knowledge learned to solve problems for a safer world by accelerating innovation with industry in the public interest.

Driven by research and technology, MITRE Engenuity’s goal is to stabilize the industries that make up critical infrastructure, like finance, telecommunications, and healthcare. The solutions that Engenuity drives forward are available for all businesses and important for innovation that changes industries and becomes a global standard. Engenuity works on the basis that these global challenges require partnership.

Share on:

Free Active Directory Assessment

Get Visibility Into Privilege And Service Account Exposure

For a limited time, Attivo Networks is providing free Active Directory Security Assessments to demonstrate how ADAssessor provides unprecedented and continuous visibility to AD vulnerabilities.

Try Our Endpoint Detection Net (EDN) for Free


Free use offer of our Award-winning security solution to prevent attackers from lateral movement, credential theft, and privilege escalation, fast and easy.

ADSecure 90-Day Free Trial


  • Hide and deny access to AD objects
  • Get alerted on unauthorized queries
  • Attack details easily viewable in dashboard
  • Your data remains on-premise


Ready to find out what’s lurking in your network?

Scroll to Top