Sometime after the release of ISO 9001:2000, a shortcoming of ISO 9001 auditing became clear: certifying body (CB) auditors were often using a standard-based approach to auditing. Auditors would arrive at an audit client’s worksite armed with a copy of ISO 9001 as a stage 2 audit checklist.  This appeared to allow inconsistent application of the requirements.

Since the 2000 standard only reputedly required six procedures, stage 1 document review consisted of reviewing an organization’s 6 procedures against requirements contained in the 6 ISO 9001 clauses calling for documented procedures. This same mind-set is a carry-over from the 1987/1994 idea of auditing.  Then, document review consisted of reviewing an organization’s 20 procedures against the 20 elements calling for documented procedures; stage 2 auditing consisted of assessing working practice against procedures responding to ISO 9001 requirements.

Since 2000, however, no procedural description of realization processes appeared to be required. Documentation describing realization processes is thus commonly not available for review during stage 1 auditing.  So, it is viewed as being the stage 2 auditor’s job to assess conformity of working practice directly against the requirements of ISO 9001. Armed with a copy of ISO 9001 as audit criteria, a stage 2 auditor can effectively do little more than that.

STANDARD BASED MIND SET
In the midst of this prevailing mind-set, the ostensible shortcoming was this: using a single copy of the requirements as audit criteria, parts of a quality management system (QMS) are not assessed.  Once conformity to a requirement is found, other areas where the requirement applies are not assessed for conformity to that requirement.

For example, if an auditor arrives at a manufacturing company armed with a checklist consisting of the standard’s requirements, the trouble starts when assessing realization processes.  Let’s say the next blank check-boxes appearing on a standard-based checklist contain the product identification requirements.  The auditor’s current mission is to verify conformity to those requirements.  So, the auditor goes into, say, the receiving area.  There, the auditor finds conformity to the product identification requirements.  The auditor duly ticks the check-box.  Conformity verified.  Good job.  Next check-box.

But when the auditor proceeds to manufacturing processes, say, an assembly process, the product identification requirements are no longer at issue.  Why?  Because those check-boxes have already been ticked.  The system has already demonstrated conformity to this requirement during this audit.  So processes to which requirements pertain were not consistently being assessed to those requirements.

THE SOLUTION OF A STANDARD-BASED MIND SET
The answer to this problem, reasons a standard-based mind-set, is to determine which ISO 9001 requirements pertain to which QMS processes.  Then, be sure to have enough copies of the relevant requirements to apply them to each process to which they pertain during stage 2.   So, if it was determined that, say, four processes encountered product in need of identification, then stage 1 audit preparation involved developing standard-based checklists for each of those four processes.

Each checklist would include a copy of the product identification requirements (among others). That way, the product identification requirements would be addressed in an audit of each of those four processes. This was regarded as a process audit, thought to represent the “process approach to auditing.”

STILL NOT A PROCESS APPROACH
No, THAT does not represent the process approach to auditing.  This solution simply represents a more thorough standard-based approach to auditing.  The stage 2 auditor under this arrangement is still left assessing conformity directly to ISO 9001 requirements at stage 2.  The auditor arrives asking “How do you  . . . [conform to this ISO 9001 requirement]?” Not knowing, the auditor must ask.  Then, once an answer has been tendered, the auditor checks to see if it’s consistently done that way on that day. Just because processing is compliant and consistent with criteria discovered that day, it does not follow that processing complies with management’s planned arrangements for processing—the focus of a stage 2 audit.

If the idea of assessing conformity to management’s defined planned arrangements (or, to the organization’s internal processing requirements) is at all recognized as being desirable by auditors applying a standard-based approach to auditing, stage 2 auditors surmise these arrangements by interviewing personnel on-site, observing activities, and examining records.  They rely upon information gathered during stage 2 to develop audit criteria that should have been clear during stage 1.  How clear? From a processing perspective, clear enough to assure proper processing; from an auditing perspective, clear enough to allow effective verification.

When management’s planned arrangements are not somehow documented and known to a stage 1 auditor, as is commonly accepted, it is the auditor’s job at stage 2 to determine what those arrangements are.  Right there on the spot, “good” standard-based stage 2 auditing requires auditors to determine what planned arrangements are in order to assess conformity to them.

PROCESS APPROACH TO AUDITING
If, during stage 1, it is clear that an organization has adopted a standard-based approach to system design and documentation, an auditor should recognize that a process approach has not been applied as required of the standard. A stage 1 finding should result.  That way, stage 2 does not proceed until planned arrangements have been sensibly and effectively documented as processes.  The stage 2 auditor’s job is then much easier, the audits are more effective, and the audit results are more reliable than those resulting from a standard-based approach to auditing.

Using a process approach to auditing effectively, stage 2 audits do not proceed until management’s planned arrangements are known to satisfy the standard’s requirements at stage 1. Auditors arriving for stage 2 assessment using a process approach are “smart.” They know what management’s planned arrangements are, and they waste no time trying to determine what they are.

They arrive at stage 2 asking questions based upon management’s planned arrangements, couched in organizational terms rather than being couched in terms of ISO 9001.  So auditees actually understand the questions.  Using a process approach, stage 2 audits are process-centric, not ISO-centric.

Auditors applying a process approach to auditing during stage 2 effectively assess conformity to management’s planned arrangements that were previously verified to satisfy ISO 9001 requirements at stage 1. At stage 2, they focus upon conformity to an organization’s own processing requirements.  They find conformity to the standard when conformity to the organization’s processing requirements has been verified.

CONFORMITY MATRICES
A conformity matrix is a valuable tool to a process-based auditor. A conformity matrix reveals how an organization has elected to operate (in conformity with ISO 9001 requirements). Using a standard-based approach prior to ISO 9001:2000, a one-to-one correspondence was expected between ISO 9001 clauses and quality management system (QMS) procedures. The resulting conformity matrix would look like this:

Today, the same mind-set persists, except only 6 procedures are often viewed as being needed to define any organization’s QMS.

On the other hand, using a process approach, procedures are developed to correspond to, you guessed it, processes.  A conformity matrix conveying this arrangement shows which requirements apply to which processes. Bear in mind that knowing where the X’s go depends upon the organization and how it has defined its system. It can’t be known before knowing the organization’s planned arrangements for processing. Here’s an example of an organization’s conformity matrix per ISO 9001:2008, reflecting a process approach:

Using the above matrix, procedures bear a one-to-one correspondence to processes, not to requirements. Also notice that one process might address several ISO 9001 requirements, while a single ISO 9001 requirement might also apply to more than one process. Again, the above matrix is a specific example based upon a (theoretical) organization’s defined system.  Matrices differ from organization to organization. While a matrix identifies which requirements apply where, the objective in stage 2 is not to audit to the requirements. Rather, assure management’s planned arrangements satisfactorily address those requirements during stage 1, then assess effective implementation of those planned arrangements during stage 2.

AUDIT OF THE CALIBRATION  PROCESS AT BOB’S MACHINE SHOP
The Calibration process at Bob’s has been in place since Bob started doing commercial work. It is documented in process fashion.

An auditor using a standard-based approach arrives for the stage 2 audit armed with the pertinent requirements of ISO 9001:2008, clause 7.6, Control of monitoring and measuring equipment.  As the standard does not explicitly require a documented procedure here, processing requirements pertaining to calibration activities (described by Bob’s Calibration procedure) are unknown to an auditor focused only upon the “6 required” procedures.

The auditor asks auditees questions based upon the standard, often couched in ISOese:

“How do you . . .

• “Calibrate or verify, ‘or both, at specified intervals, or prior to use . . . ’?” (ISO 9001:2008, 7.6 a.)
•  “Identify ‘equipment needed to provide evidence of conformity of product to determined requirements’?” (ISO 9001:2008, 7.6. general and c)
•  “Safeguard equipment ‘from adjustments that would invalidate’ results’?” (ISO 9001:2008, 7.6 d.)

Auditees should not be expected to know ISO 9001 requirements, unless they are internal auditors. Auditees are supposed to know their procedures and they are supposed to be demonstrably following those procedures.  Questions couched in ISOese needlessly confuse personnel.  However, even when auditors using a standard-based approach rely less upon ISOese, they cannot escape the standard-based audit plan they have effectively implemented. They are auditing to ISO 9001 requirements, not to organization’s internal processing requirements complying with ISO 9001 requirements.

Conversely, an auditor using a process approach arrives and says to Bob, “I understand that you are using the old index card routine that was common in the military for years. That should work fine, it’s a very simple and effective method. Let’s have a look at the index card file . . .”

Stage 2 of a process-based audit assumes the process was adequately vetted against ISO 9001 requirements during stage 1. Some kind of documentation must be available to stage 1 auditors to show organizations’ processing requirements satisfy requirements of the standard.  In Bob’s case, he supplied his Calibration procedure, which describes his planned arrangements for controlling calibration activities and calibrated devices.  During stage 1, the auditor reviewed these planned arrangements against the requirements of 7.6 and found them to be acceptable in meeting requirements.

The purpose of stage 2 is now to determine if these planned arrangements have been effectively implemented.  That means, determine if the process is being performed as planned, while producing expected results.

So, the auditor here asks questions arising from Bob’s own Calibration procedure.  Using a process approach to auditing, checklists are unique to each audit of each process. (Process audits do not use pre-written checklists based upon the standard, nor pre-written checklists based upon something other than the defined process at hand.)

Knowing the procedure (or, having a checklist based upon the procedure itself), the process-based auditor is “smart.” She fundamentally knows the answer to every question she asks before she asks it.  She is not asking questions about proper process performance because she doesn’t know the answer, as her standard-based counterpart must.

She might not need to ask as many questions, either.  Knowing from Bob’s Calibration procedure that all devices under calibration control are identified with calibration stickers, except for ring gages and thread gages.  Stickers won’t stick on these very well.  So these are identified by unique “Cal ID” numbers engraved on a noncritical surface of each such gage.  Cal ID numbers enable calibration status to be determined by accessing respective index cards in the index card file.

Aware of this, the auditor might request, “Show me your thread gage collection.”  Knowing the procedure, she knows what she wants to see and why she wants to see it.  She looks at some thread gages, jots down some numbers, and checks them against the index cards.  She knows conformity when she sees it.

PROCESS AUDITS PROVIDE MORE VALUE
A standard-based approach to auditing results in conclusions merely regarding conformity to ISO 9001 requirements. An audit report resulting from a standard-based approach consists of a clause-by-clause summation of findings or comments. On the other hand, a process approach to auditing produces results that are more interesting and valuable to management, offering conclusions beyond mere conformity to ISO 9001.

During a 3rd-party ISO 9001 audit, management is relying on an objective, external auditors to assess whether working practice follows management’s planned arrangements. If stage 1 auditing is done properly, management is aware that their planned arrangements satisfy the requirements of ISO 9001 before stage 2 begins.

At stage 2, the focus is upon conformity to organizational processing requirements. Management is interested to find out if working practice in fact follows their defined planned arrangements. A standard-based approach to auditing cannot effectively make this determination.

Management should be more interested in the results of a process-based audit than they are in the results of a standard-based audit. Process-based audits tell them something about their operations, not merely about conformity to ISO 9001. So, audits using a process approach provide more value to management.

Bio:

T. D. (“Dan”) Nelson is a quality management consultant, author, and trainer
specializing in the process approach, ISO 9001, and related sector schemes.
Dan has roughly 20 years of experience with ISO 9000 and over 15 years’
experience with the process approach. Dan holds an MA in Business
Administration from the University of Iowa.  Dan can be reached at:

dan@tdnelson.com
720.412.7994