ISO 19011:2011 tells us that auditors are supposed to operate with “Integrity: the foundation of professionalism” (Principles of Auditing, 4 a). Another principle of auditing, per ISO 19011 is “due professional care.” Later on, in section 6, ISO 19011 also tells us that management system documentation is supposed to be reviewed during preparation for an audit.
If all of the above were effectively implemented, it seems certifying body (CB) auditors would never arrive at an audit using a standard as the audit criteria. If they have no choice because the audit client provided inadequate documentation for review, THAT should be a document review finding that prevents the auditors from proceeding with a certification audit.
WHAT IS MANAGEMENT SYSTEM DOCUMENTATION
In the case of ISO 9001:2008, the expected documentation includes a quality manual (level one) and operating procedures (level two), a.k.a. (quality) management system procedures or standard operating procedures. Together, these documents define the system (level one) and describe how it’s implemented (level two). At level one, the system is defined as a system of processes affecting quality; a level two procedure dedicated to each process identified at level one describes how that process is performed.
Does QMS documentation have to be in three levels? No. It’s a typical (and sensible) structure for management system documentation. Policies direct at level one, procedures implement at level two, and whatever third level documentation is needed controls the details of getting it done. Regardless of structure, however, system documentation needs to adequately indicate conformity to applicable ISO 9001 requirements.
A procedure describes the sequence and interaction of processing activities, just as a manual describes the sequence and interaction of a system’s processes. A procedure describes process planning, while a manual describes system planning. This planning corresponds to the “plan” phase of plan-do-check-act (PDCA). The plan is the important thing, that’s what is supposed to be improved by application of PDCA.
In keeping with the three-level structure, while level one references level twos, level twos reference level threes. Level three documents describe processing at the level where the rubber meets the road.
For example, about production, a manual might say, “We make good product by controlling production. See the Production procedure.” The Production procedure says, “We make good product by transforming inputs into outputs via activities that are sequenced according to a Router; blueprints convey product characteristics and contain acceptance criteria.” A work instruction might tell an operator how to perform a specific task: “Insert nut A onto Bolt B and torque to 15 foot pounds.” Blueprints, routers, work instructions, any forms used to control processing are all level three documentation.
So, the documentation expected to prepare for a management system audit would include levels one and two—documents defining the system. Level three documentation could be far too voluminous for review during stage 1, and the objective of stage 1 isn’t to verify conformity of all documents needed for the entire system, anyway. Third-level documentation is examined during stage 2 auditing.
When management system documentation is raised in response to ISO 9001 requirements (as is common), system documentation fails to document the actual system in operation. The actual system in operation outputs quality product every day. Using a process approach, management system documentation is written to manage this system of processes using PDCA, starting with definition of the plan.
On the other hand, standard-based documentation (e.g., based on ISO 9001) is written to direct auditors toward conclusions of conformity, by clearly stating how operations comply with each individual requirement of the standard. Management system documentation is not structured to describe actual operations, as its structure was instead determined by the clauses and sub-clauses of ISO 9001. So standard-based documentation does not effectively document any management system.
MANAGEMENT SYSTEM DOCUMENT REVIEW
At 6.3.1, “Performing document review in preparation for the audit,” [italics mine] ISO 19011 tells auditors to review documents help prepare audit activities and working documents, and to identify any system gaps. A certification audit involving ISO 9001 is a management system audit. The objective of the audit is to assess the system for conformity to ISO 9001 (as opposed to an internal audit of, say, a particular process). In the case of a management system audit, management system documentation defining the system is the relevant documentation.
[For further reading, please see: “A System for Satisfying Customers,” Quality Digest, June 2014. Here’s a link:
https://www.qualitydigest.com/inside/quality-insider-column/system-satisfying-customers.html ]
During document review of a management system audit, the manual and procedures are reviewed to ensure they collectively define the system and processes adequately to determine their internal consistency and conformity to ISO 9001 requirements. Problems identified here should result in findings. Trouble starts when standard-based systems are viewed to meet the above criteria. Standard-based documentation does not begin to describe the actual management system or its processes, but instead describes perfectly how the system meets ISO 9001 by pandering to the very requirements against which the system will be assessed.
So, is document review confined to stage 1? No. See 6.4.3 of ISO 19011, “Performing document review while conducting the audit.” Bear in mind what documentation is being reviewed here. This is not the place to be reviewing system documentation for completeness and conformity to the standard. Instead, third-level documentation is being reviewed for consistency with the system documentation describing its use, to ensure it is being used as dictated by management, as well as to the requirements of ISO 9001 (e.g., that it is approved and controlled).
AUDIT CONDUCT (ONSITE)
When auditor arrives to conduct audit activities, management is signed up only to prove that it operates as it has defined its operations in the manual and operating procedures. Audit preparation should have verified that the planned arrangements (or, internal processing requirements) contained in the system documentation were acceptable in meeting ISO 9001 requirements. Rather than being focused on conformity to ISO 9001 requirements, auditors are interested in assessing conformity of working practice to planned arrangements defined in system documentation.
Audit guidance for the aerospace and defense industries was recently released in the form of AS9101E. AS9101E speaks more clearly about a two-stage approach to auditing, actually using the terms “stage one” and “stage two.” Stage one corresponds with “preparing audit activities” of ISO 19011, 6.3, while stage two corresponds to “conducting the audit activities” (ISO 19011, 6.4).
At 4.3.2.1 b of AS9101E, it tells us that stage 1 auditing should include an on-site visit. The purpose of this visit is to allow the CB to get its arms around the system, actually looking at what is being done and how it is being done. Does that mean that stage 1 and stage 2 can be combined—an auditor can arrive on-site to perform stage 1 auditing activities (i.e., management system document review) in combination with stage 2 auditing activities?
No. AS9101E, at 4.3.3, “Stage 2 Audit,” the standard clearly states that stage 1 and stage 2 auditing must not be performed on the same day, or even on consecutive (back-to-back) days. They are separate auditing activities. To schedule stage 2 the day after stage 1 would seem to suggest that any problems discovered during stage 1 could be resolved appropriately overnight.
For more information about AS9101E, please see: “AS9101E—Best Yet”– https://insights.cermacademy.com/2014/05/47-9101-e-best-yet-t-dan-nelso/
UNDOCUMENTED OR STANDARD BASED MANAGEMENT SYSTEMS DON’T CUT IT
If a CB auditor arrives to conduct audit activities using ISO 9001 requirements as the audit criteria, rather than using the organization’s own (verified compliant) system documentation as the audit criteria, a process approach is not being applied to auditing. Instead, absent adequate (process-based) documentation to determine conformity to ISO 9001 requirements during stage 1, the audit plan is essentially to use stage 1 audit criteria to conduct stage 2 audit activities. For more about applying a process approach to auditing (and how it differs from a standard-based approach), please see this article, “Audit My Process Please”:
https://insights.cermacademy.com/2013/10/27-audit-my-process-please-t-dan-nelson/
Proceeding to stage 2 without adequate documentation to pass stage 1, an auditor arrives to effectively verify conformity to individual requirements of ISO 9001, since management’s undefined planned arrangements are not objectively known like they would be if they were adequately documented. Again, it was these very planned arrangements that should have been the subject of the stage 1 audit.
AS9101E requires the auditor to find evidence that the requirements of AS are addressed by documented QMS procedures at stage 1 (4.3.2.2 d). So a 6-procedure-only system wouldn’t demonstrate conformity, as the six don’t exhaustively address applicable requirements, particularly the product realization requirements.
If effectively applied, requirements for QMS documentation describing the processes and their sequence and interaction (4.3.2.2 b) should eliminate clause-by-clause procedures, which don’t describe QMS processes nor their sequence and interaction. At best, they represent QMS requirements that haven’t been integrated into real business processes.
Regardless, neither the 6-procedure-only or the clause-by-clause standard-based approach effectively documents a QMS adequately for AS9100 or ISO 9001 certification.
If on-site auditing reveals that the planned arrangements are in fact unsatisfactory, auditors discover this only after arriving on-site. The company is paying for the auditor’s airfare, hotel, meals, local transportation, etc. I f the auditor arrives to find a major nonconformity, preventing registration and potentially requiring another visit, it’s because the stage 1 auditor didn’t do his job. So the CB gets double compensated for its incompetence.
CBs could make lots of money this way, but isn’t that unethical? Stage 2 auditors should not arrive until planned arrangements have been verified to be adequately documented to demonstrate conformity to ISO 9001 requirements. If the plans are not adequately documented, it seems unethical to proceed to stage 2.
When auditors DO proceed to stage 2 without confirmed adequate QMS documentation (which seems very common), findings the auditor raises are written against ISO 9001—as there is no procedural provision upon which to base a finding. (Or, findings are raised against standard-based procedures, which are often basically a regurgitation of the requirements, so the finding may not point to the problematic planned arrangement for processing.) There is no defined plan to identify as being inadequate.
These are stage 1 findings being improperly raised during stage 2 auditing—due to poor stage 1 auditing. Stage 2 auditors walking around with ISO 9001 checklists as their audit criteria seem to be acting without integrity and due professional care, unaware of the ethical problem built into their audit plans.
Bio:
T. D. (“Dan”) Nelson has been closely involved with ISO 9000 since 1994 as a technical writer, quality manager, management representative, consultant, author, and CB auditor. Holding an MA in Business Administration from the University of Iowa, Dan also has 12 years of experience as an IRCA-certified QMS Lead or Principal Auditor, conducting registration audits and surveillance audits, and training Lead Auditor candidates in accredited courses. Using a process approach, Dan has taken several scores of clients of various shapes and sizes through registration to ISO 9001:1994/2000/2008 and related sector schemes (e.g. QS 9000, AS9100, ISO 13485, and ISO 17025). Dan’s numerous articles about the process approach have also been published by Quality Digest, Inside Quality, ASQ’s Quality Management Division, the Society for Manufacturing Engineers (SME), and the South African Quality Institute (SAQI); Dan has been featured as a guest blogger by RABQSA, and has been featured on Quality Digest Live. Dan is available for management consulting, training, and coaching, as well as auditor training and coaching. Contact: dan@tdnelson.com 319.210.2642