I read an interesting analysis (during my morning RSS Grind) on Branden Williams blog – titled "PCI Compliant Companies Don’t Suffer Breaches"
There is a big misnomer out there that needs to be cleared up. […] In our investigations of PCI related breaches, we have NEVER concluded that an affected company was compliant at the time of a breach. PCI Assessments are point-in-time and many companies struggle with keeping it going every day.
These leads us to the nuance between PCI compliance and PCI Validation – Mike Dahn does a great job here at this post:
Compliance vs Validation
There is a difference between ‘compliance’ and ‘validation’. Compliance is a state of being, one that must be maintained at all times. Validation is a point-in-time check on that state of compliance. The example I give is auto insurance. In order to comply with state laws I must maintain auto insurance at all times. When I go to register my car I have to show proof of insurance. I am validating my compliance with the law. What if I decide to cancel my insurance because it costs too much? Am I still compliant? No. Now, I still validated, but remember validation is a point-in-time while compliance is measured day by day.
One of the first things that people asked when they heard about the latest cardholder breach, it was "weren’t they PCI complaint" ? If you look on the Visa List – you will note the validation date and QSA that performed the review. It is important the understand the Validation Date – that was the last date of the review – that is a point in time where a QSA considered the organization it reviewed "compliant".
So how is this possible ?, Well an example that I can think of is brushing your teeth really well before going to the dentist, or how many companies fear auditors and get really busy before and audit because of the things they need to do because the auditors are coming ? Do people continue to brush their teeth with this vigor after the dentist visit ? perhaps they floss for a week or so, but behavior is hard to change and people go back to their normal routine. The focus of many organizations is on passing the audit.
I briefly touched on this point when I blogged my thoughts on the Heartland Breach.
If you ever read through some of the actual audit procedures of PCI – notice what the auditors actually test: some tests are just based on Inquiry or documentation alone, furthermore some tests that do not test the operating effectiveness of some of these controls to a period of time.
The current focus is that on existence of a control and not its effectiveness.
Operating Effectiveness – that is defined as: "How an internal control was applied, the consistency with which it was applied, and by whom."
Consistency – Should the PCI council consider reviewing the audit procedures and testing procedures to test for consistent controls ? Should PCI audits of large service providers or merchants require multiple visits by the QSA to test for this consistency ? Would it even help ?
Another last thought: Do QSA’s and auditors cheat on 2nd and subsequent year follow-up reviews ? Well, they passed it last year… – does the Report on compliance (ROC) look similar to that of previous years ? Was the control still tested like it was in the first year ? I guess some of this can be addressed (hopefully) in the PCI QSA QA program.
I agree with Branden that if one ever finds the real exploit that occurred at Heartland, it probably can be mapped ( even liberally squeezed ) to a current PCI control and is more then likely the result of a control breakdown and the controls poor operating effectiveness.