Wed, Jan 31, 2018

PCI Data Security Ordeals: Inadvertent Storage Attacks

Inadvertent Storage Part Deux

Several months ago I published an article about Inadvertent PCI Data Storage. That article served as a succinct summary of the problems around Inadvertent Storage and its relevance to PCI Forensics Investigator (PFI) reporting. It detailed that most instances of Inadvertent Storage were the result of antiquated business process or poor data hygiene – sometimes both. I mentioned that unknown and antiquated cardholder data is rarely the target of attackers. In some ways that article downplayed the potential dangers around Inadvertent Storage – I even said at one point in that article, “In most cases this data isn’t even subject to compromise. The bad guys don’t even know it’s there.”

Well, if the last four months have shown me anything, it’s that attacker methods have changed.

Do you know what your inadvertent storage is doing right now?

In the time since that article was posted, I’ve worked at least half a dozen PCI related cases where Inadvertent Storage was an issue in each. And I don’t just mean that during the course of an investigation we discovered that the victim entity was retaining - unbeknownst to them - old cardholder data. I mean that in every instance attackers were purposefully identifying, collecting, and exfiltrating inadvertently-stored cardholder data. Even worse – in most cases, the victim didn’t even know they were storing it. How can you protect data you don’t even know you have?

Let’s take it back to 2005, when Point-of-Sale (POS) solutions frequently stored cardholder data in plaintext post transaction. This data was typically stored in transaction log, or “T-log”, files that were quickly identified and exfiltrated by attackers. As POS solutions stopped storing cardholder data, attackers began to use other techniques, such as capturing data off of the wire. When vendors started to encrypt transactions on the wire, they moved to memory scrapers. Today, in the era of EMV and P2PE POS solutions, we are finding that many vendors and/or organizations are reverting back to the original 2005 problem. Data is being retained, at rest, in plain text, post-transaction. PCI data may also be inadvertently stored by organizations enabling “debug mode” during testing, and forgetting to disable during deployment.

Let’s examine two cases where attackers went after inadvertent storage:

Case Study 1 – No Code Needed

This recent case involved cardholder data stored in XML transaction log files that were identified and exfiltrated by the attackers. Figure 1 provides a sample of these logs – and we (and thus, the attackers) found thousands of entries like this.


PCI Data Security Ordeals: Inadvertent Storage Attacks

Figure 1 - Sample XML-formatted transaction log. Obfuscated to protect the innocent. 

The files were stored on an e-commerce web server, in a directory called “transaction history.” No kidding.

The rest of the case was simple. The attackers accessed the victim shopping cart application with default vendor credentials, installed a quiet webshell in the /images/ directory, and then used that channel to access, compress, and exfiltrate log data. 

That was it - no complex malware necessary. No altering the Javascript of checkout pages to intercept transaction data on-the-fly. Just identification and exfiltration of at-rest cardholder data. It felt like 2005 all over again.

Case Study 2 – Peeking Back in Time

This second case introduced something I’d never seen before. We quickly identified RAM scraping malware (referred to as PinkKite by Kroll Cyber Security) that affected the POS environment at several hundred restaurant locations nationally. The malware was installed on September 9, 2017 and the first evidence at all of unauthorized attacker activity in the environment was a week earlier on September 2, 2017. The attackers used a software deployment system within the corporate environment as a pivot point to access restaurant locations, and subsequently uploaded RAM scraping malware to capture magnetic stripe cardholder information during the transaction process.

However, something strange happened as we began to articulate and communicate case findings to our client, the acquiring bank, and the affected credit card brands. Each of the affected credit card brands advised that they were seeing fraud patterns against cards transaction prior to September 9, 2017 – in some cases, years prior. There were cases of legitimate transactions taking place as far back as 2015 that were only reflecting fraudulent transactions after the September intrusion. To be sure, we had no evidence at all that there was unauthorized or malicious activity taking place as far back as 2015.

Somehow, this malware was able to peek back in time. It was not only capturing current transactions in system memory – it was somehow capturing old - sometimes years old - transactions. Here is a sample of what one of the RAM Scraper output files looked like:

PCI Data Security Ordeals: Inadvertent Storage Attacks 

Figure 2 - Sample of data capture from RAM-scraping malware.

The malware output file contained two key pieces of information for each compromised record. Not only did it capture Track II magnetic stripe data (which can be used to create fake cards), but it also captured the specific process in memory from which the Track data was written. In this case there were two system processes interacting with Track II data in memory – “POSrun.exe” and “SQLpossrv.exe.”

“POSrun.exe” we knew about. That was the process that was responsible to taking the card data on-swipe and forwarding on to the merchant acquirer for authorization. “SQLpossrv.exe” though was less easy to explain. What was obvious though, was that a majority of the accounts being pulled from the “SQLpossrv.exe” process looked older than those coming from “POSrun.exe.” Simply, their expiration dates (the 4 digits to the right of the equals sign separator) made them look like older cards. They were expiring sooner. We began to guess that perhaps those accounts reflected earlier transactions. The puzzle pieces were starting to come together.

What we learned was that each POS server actually ran an SQL database that maintained two separate tables containing historical track data. Whenever the system would boot, SQL would load both tables into system memory – hence, “SQLpossrv.exe.” And because the RAM scraper was pulling all Track data that it found in memory, it had a nice historical view going back several years. Cards that were transacted in 2015 weren’t actually stolen until 2017.

Real consequences of inadvertent PCI data storage

The inherent problem with Inadvertent Storage is that organizations presume it may not be something to worry about – some may even chalk it up as the cost of using vendor ABC or product XYZ. Unfortunately, attackers are aware of this, and may be looking for ways to take advantage of this data. Depending on how your POS architecture interacts with your inadvertent storage, you may be serving this data up to attackers on a silver platter.

Cyber Risk

Incident response, digital forensics, breach notification, managed detection services, penetration testing, cyber assessments and advisory.

Forensic Investigations and Intelligence

The Kroll Investigations, Diligence and Compliance teams consists of experts in forensic investigations and intelligence, delivering actionable data and insights that help clients worldwide make critical decisions and mitigate risk.

24x7 Incident Response

Kroll is the largest global IR provider with experienced responders who can handle the entire security incident lifecycle.

Malware and Advanced Persistent Threat Detection

Our expertise allows us to identify and analyze the scope and intent of advanced persistent threats to launch a targeted and effective response.


Proactively monitor, detect and respond to threats virtually anywhere – on endpoints and throughout the surface, deep and dark web.