ISO 19011 sets the standard for auditing of management systems. When conducting an audit, ISO 19011 sets out 6 principles for an effective and reliable audit with sound outcomes:
- Integrity: the requirement to act in a professional manner
- Fair presentation: the obligation that an auditor has to report truthfully and accurately (including as stated in ASAE 3150 to maintain professional scepticism).
Due professional care: the application of diligence and judgement when undertaking audits.
- Confidentiality: ensuring that evidence remains secure.
- Independence: ensuring that the auditor remains impartial and objective.
- Evidence-based approach: the rational method for reaching reliable and reproducible audit conclusions in a systematic audit process.
- Risk-based approach: an audit approach that considers risks and opportunities.
Further, ISO 19011 is clear about the definitions of audit (Table 1). So, given that it is not feasible to audit 100% of an auditee’s systems and processes, how do you ensure that your process for obtaining evidence on their drinking water management system is systematic, independent, sound and provides you with confidence in the audit findings and outcomes? Here we take a look at some of the things we find useful in helping to frame and conduct our audits.
Table 1. Audit definitions within ISO 19011.
| Term | Definition |
| Audit | Systematic, independent and documented process for obtaining audit evidence and evaluating it objectively to determine the extent to which the audit criteria are fulfilled (can be first party (internal), second party (e.g. supplier audits) or third party (e.g. statutory audits by external auditors). |
| Audit criteria | Set of policies, procedures or requirements used as a reference against which audit evidence is compared. |
| Audit evidence | Records, statements of fact or other information which are relevant to the audit criteria and verifiable. |
| Audit findings | Results of the evaluation of the collected audit evidence against audit criteria (can be internal findings to help identify non-conformity and identify areas of improvement, or if a legal requirement, can be a non-compliance finding). |
| Audit conclusion | Outcome of an audit, after consideration of the audit objectives and all audit findings. |
Sampling Risk
There are various areas of risk associated with auditing but this article is primarily concerned with sampling risk. As noted by ASA 530, sampling risk is defined as the risk that the auditor’s conclusion based on a sample may be different from the conclusion if the entire population were subjected to the same audit procedure. Sampling risk leads to erroneous outcomes and inefficient/ineffective outcomes as shown in Table 2. So, an important question to ask yourself is:
“How do I ensure that I have a reasonable basis from which to draw conclusions about the population from which the sample is selected?”
Table 2. Affects of sampling risk on audit efficiency and effectiveness (adapted from ASA 530).
| Type of Test | Sampling Risk Outcomes | Audit Affect |
| Controls | Controls are found to be more effective than they actually are. | Conclusion is erroneous.
Sampling risk affects audit effectiveness, and is more likely to lead to an inappropriate audit opinion. |
| Details | A material misstatement is found not to exist, when in fact it does. | |
| Controls | Controls are found to be less effective than they actually are. | Conclusion is erroneous.
Sampling risk affects audit efficiency as additional work is likely to establish that initial conclusions were incorrect. |
| Details | A material misstatement is found to exist, when in fact it does not. |
Understand the Context
We have found that getting the best value for the auditee in terms of review outcomes and for regulators in terms of scrutiny and compliance findings, is by a thorough review of the auditee’s context prior to going on site. So, what do we look for and how do we determine our sampling method based on this information? Here are some of our key tips:
- When determining the sampling scope, consider how long the current process has been in place – any findings should help to improve the current process, not pick holes in a superseded process.
- When determining the sample size, consider the body of data available and how many sites or categories it spans. For a representative view across the organization, it may be necessary to spread the samples across multiple sites but for a comprehensive assessment of a specific process, time may be better spent thoroughly investigating a smaller number of sites.
- Mathematical rules may be applied to sample size, for example, the square root of the total number of samples within the audit period or alternatively a fixed percentage. The sample size and test criteria should ultimately focus the auditor’s on-site time on achieving best value for the stakeholders.
- Sometimes it may be appropriate to choose a random sample and at other times use auditor’s judgement, or a mixture of both. Data analytics can be helpful in highlighting areas for further investigation.
- Always keep the auditee’s key organizational objectives and the regulator’s priorities front of mind – make sure that the sample reflects what is really important.
- When determining the sampling and testing methodology, keep asking “so what?” to make sure any findings will be relevant.
Are you due for an audit or simply would like a gap analysis of your water management system performance? Annette Davison and Sarah Loder are experienced auditors (water quality and business processes). We can assist you with a review of your drinking water management system, identify opportunities for improvement, identify potential compliance gaps, and help you get ‘audit-ready’. Our aim is to provide exceptional outcomes for our customers. Learn more here.
BIOS:
Annette Davison, Director and Principal, Risk Edge® Pty Ltd; Managing Director and Chief Risk and Product Officer, D2K Information Pty Ltd
Sarah Loder, Senior Risk Analyst, Risk Edge® Pty Ltd and Director and Principal, Praktik