All Eyes on PCAP: The Gold Standard of Traffic Analysis
PCAP, or full packet data capture for analysis, does what it says – it captures the entirety of every packet that comprises the network traffic (both metadata and content). If something happens on the network, PCAP knows about it. Whether it is malware moving data around, or staff arranging a private party, it can be captured and then analyzed.
PCAP provides what CISOs seek but rarely achieve – total visibility into the network.
The security potential for this type of traffic monitoring is clear, and probably explains the motivation for a number of U.S. federal agencies investigating their options. Toward the end of 2020, in the first flush of the SolarWinds debacle, the DHS, the Department of State, Aberdeen Proving Grounds, the U.S. Marine Corps (USMC), and the Missile Defense Agency (MDA) all issued requests for proposals (RFPs) and requests for information (RFIs) for PCAP solutions.
The Homeland Security Department’s Enterprise Security Operations Center stated that it considered “Full Packet Capture a cornerstone of the cyber security visibility stack enabling analysts to perform investigation analysis while also satisfying DHS security requirements.”
This sudden rush to PCAP poses a couple of obvious questions. If PCAP is such a powerful security tool, why hasn’t it already been widely adopted among the agencies? And is this movement within the federal agencies likely to migrate to the general business sector?
“The packets never lie,” says Vectra’s EMEA director, Matt Walmsley. “Packet capture has long been the gold standard of primary evidence sources for network security forensics. It’s a bit-for-bit direct copy of the exact traffic that was transmitted across the monitored network. It’s not an interpretation, it’s not a summary description – it’s the raw truth.”
PCAP collects everything. It is not designed to provide real time – or any – analysis. Analysis is left to add-ons or other security tools. The value of PCAP is the ability to see and capture in detail exactly what has happened.
For this reason, many analysts believe that PCAP is best suited to (recent) historical analysis. “Full packet capture is purely for historical analysis,” says Oliver Tavakoli, CTO at Vectra. “It usually depends on some other detection capability to point the finger at packets of interest.”
Not everyone entirely agrees with this. “It is absolutely not only for historical analysis, “Joseph Carson, chief security scientist and advisory CISO at Thycotic, told SecurityWeek. “Full PCAP can also be used in real-time; however, this requires very well-crafted algorithms to help SOC Analysts determine which packets should be investigated and what can be fully automated.”
The addition of artificial intelligence to PCAP could well change the use and value of PCAP in future years.
Carson summarizes the value of PCAP. “Recently, I analyzed a severe ransomware incident. With the log data remaining it was only possible to get a partial view on how the attackers worked – but if I had full PCAP data then it would be possible to create a much more detailed attack path.”
Axellio SVP Stefan Pracht explains further: “Only packets,” he said, “can offer the insight into the timing and sequence of events, where the attack came from and which enterprise resources were involved with the malicious activity, what data was accessed in the attack or even exfiltrated and how the attack spread laterally through the network. Being able to play out the traffic of the actual attack also provides important insight into whether this happened before but went undetected and to determine whether any implemented mitigation is actually working.”
SolarWinds is a case in point. “We’d really like to know how whoever got onto the networks actually did what they did – and I mean exactly what they did,” says Sammy Migues, principal scientist at Synopsys. “Normal logs will capture that HostA talked to HostB and things like that; but what did they say? Enquiring minds want to know! What exactly did they change? How? How did they remain undetected? So many questions that might be answered with full packet captures.”
Brandon Pearce, AVP of federal and intelligence products at AT&T Cybersecurity (and former CISO at the National Geospatial-Intelligence Agency), goes into more detail. “Because PCAP allows direct examination at the packet level, government agencies can review it on their network forensically to find any anomalous behavior at this level. Compared to logging data, there is more of a chance to find previously unknown behavior like covert communication/exfiltration channels, command and control signals embedded in otherwise expected traffic, and so on.”
Finding previously unknown activity at the packet level can reveal not only that something has happened, but how it happened. This gives agencies the data to form an actionable plan to counter an unwelcome presence in their network, and the hard evidence to show what happened. Logging at higher levels of the OSI model doesn’t give that same forensic level of information.
The cost problem
Apart from being the gold standard for network forensics, Walmsley adds, “It’s also highly voluminous, expensive to store, and extremely arduous to search and analyze manually at any meaningful scope.”
Cost is a problem. Capturing everything that crosses the entirety of a network requires a huge amount of storage. While storage costs are coming down, network traffic is going up, and it remains inhibitively expensive to store more than a few days of PCAP data. “Performing full packet capture on a one gigabit per second link (note that fast links are now running at 100 gigabits per second or more),” explains Tavakoli, “can require upward of 10 terabytes a day in storage. This practically means that it becomes a race against time: if an organization can afford to store 10 days of full packet capture, it is effectively betting that it will find an attack within 10 days of its initiation.”
If a company does not have the resources to effectively analyze that amount of data quickly, it is hard to justify the expense of PCAP. “PCAP has a place,” says Richard Bejlitch, principal security strategist at Corelight, “but one must balance trade-offs of storage, ability to query, and other factors. I would encourage agencies looking into upgrading their network security monitoring infrastructure to first see if transaction logs could solve their problems, with targeted or ‘smart’ PCAP for edge cases and additional inquiry. A ‘full PCAP first’ approach can be costly and slow compared to the alternatives.”
Apart from the choice between full or just partial PCAP collection, there is a potential problem in staffing. “Storage limitations may necessitate choosing from which network segments to capture traffic,” says Joseph Salazar, technical deception engineer/ technical marketing engineer at Attivo. “Much like storage constraints result in lost evidence, lack of coverage leads to missing PCAPs and blind spots during analysis. Finally, analyzing PCAPs requires experience and training to extract the relevant data needed for an investigation, whether it is a binary payload or an exfiltrated data file. With the training and personnel challenges facing the cybersecurity industry, this is not a need that organizations can quickly fill.”
Read the full article by Kevin Townsend on Security Week Network.
Free Active Directory Assessment
Get Visibility Into Privilege And Service Account Exposure
For a limited time, Attivo Networks is providing free Active Directory Security Assessments to demonstrate how ADAssessor provides unprecedented and continuous visibility to AD vulnerabilities.
Try Our Endpoint Detection Net (EDN) for Free
FAST AND EASY
Free use offer of our Award-winning security solution to prevent attackers from lateral movement, credential theft, and privilege escalation, fast and easy.
ADSecure 90-Day Free Trial
GET PROTECTION AGAINST UNAUTHORIZED ACCESS TO ACTIVE DIRECTORY
- Hide and deny access to AD objects
- Get alerted on unauthorized queries
- Attack details easily viewable in dashboard
- Your data remains on-premise