Control Effectiveness is measured by looking at the maturity of the process.  Most people agree that mature processes are documented.  Why?

Transferring knowledge from the human brain requires conversion from tacit knowledge to explicit knowledge so that it can be shared, reviewed, updated and tested for.  Think about it?  If we relied on tacit knowledge all the time there is a good chance that the outcomes would be different every time the process was executed unless they had a plan to follow which is where explicit knowledge comes into play.  Quality Management requires that we integrate feedback loops to push a process even higher in maturity.  Continuously monitoring and making adjustments to perfect the process can only be achieved with explicit knowledge.

Building the perfect control to mitigate risk is one thing but making sure that it gets implemented, monitored and maintained adequately so that its functioning 100% is yet another.  This requires the assessment of competence for those employees or contractors that have been assigned the responsibility to get her done!!

I like to leverage my knowledge as a teacher using Blooms Taxonomy I create at least 6 basic questions to determine how much the employee knows.  I recently created a one page assessment for CyberSecurity Leader that makes a good example. You can find that one at this link; http://tinyurl.com/qbrz5ha

We have now evaluated the control for maturity, assessed competence of the administrator and now we need to verify and validate that the control is functioning as planned. There are similar approaches that work from using Quality Assurance techniques to using Penetration Testing. This part really should be looked at ever time changes occur that touch the control, in any way. We need a solid baseline for assessment control effectiveness and to accomplish that I like to integrate the use of DQ, IQ, OQ and PQ.

Based on my experience the most security systems are those that establish and maintain absolute control over the environment.  I often joke with senior management about my number one rule, “No Surprises!”  Quality Management is deep in knowledge about establishing control and assessing process so it’s only logical that we would assimilate this knowledge into information security.

DESIGN QUALIFICATIONS
DQ represents Design Qualifications.  This is the architecture or specifications used to build a service or product.  Any changes must be strictly controlled, so while DQ sets out the Design – Installation Qualifications, which defines the specifications or standard operating procedures for installing a new piece of software or hardware.  Control Design is a related topic that would allow you to map where this control applies within the Risk Universe as it mitigate risk to a specific asset that is used to deliver a service or product. Examples of Control Design http://tinyurl.com/pl746mb and Risk Universe http://tinyurl.com/loqw56e .

Operational Qualifications document is the configuration specifications, which could be recorded in the configuration management database used by ITIL also ISO 20000.  Once everything has been documented and procedures have been followed the Performance Qualifications are reviewed.   What was the expected response times? How can we optimize them to meet customer expectations?

Who gets the job to review control effectiveness should be looking at these three key elements, maturity, competence and testing to verify and validate that what we said we would do we have actually achieved.  The importance of assessing control effectiveness during regular audits is obvious.

The assessment of control effectiveness during risk assessment as part of the risk management and governance process is absolutely crucial to provide all the facts to management quantified in a meaningful way.  I have seen plenty of external audits that have gone in a direction where hundreds of thousands of dollars and sometimes millions are spent on new controls that may not have been necessary if the current investment in security was not better quantified and managed.

This is an evidence approach that can easily be shared and reviewed and scrutinized.  Too much control could negatively impact the business model, organizational culture, agility or time to market and resilience by created more complexity that is expensive to maintain and difficult to replicate under emergence situations.

Bio:

Mark E.S. Bernard, CRISC, CGEIT, CISA, CISM, CISSP, PM, ISO 27001, SABSA-F2

Information Security, Privacy, Governance ,Risk Management, Consultant.  Mark has 24 years of proven experience within the domain of Information Security, Risk, Governance and Compliance.  Mark has led teams of 30 or more as a Director and Project Manager and managed budgets of $5 Million +.  Mark has also provided over sight as a senior manager during government outsourcing contract valued at $300 million and smaller contracts for specialized services for ERP systems and security testing.  Mark has led his work-stream during RFP process, negotiations, on-boarding, contract renegotiation and as Service Manager.  Mark has architected information security and privacy programs based on ISO 27001 and reengineered IT processes based on Service Manager ITIL/ISO 20000 building in Quality Management ISO 9001.   He can be reached at: mesbernard@gmail.com