In an ideal world the design of a product or system will make use of perfect knowledge of all the risks and failure mechanisms. The designed product then is built perfectly without any errors or unexpected variation and will simply function as expected for the customer. Wouldn’t that be nice!

Along with perfect manufacturing and perfect materials, the assumption that we have perfect knowledge is the kicker though.

We often do not know enough about

• customer requirements,
• operating environment
• frequency of use,
• impact of design tradeoffs,
• material variability, or
• process variability.

We do know that we do not know everything we need to create a perfect product. Thus we conduct experiments: We test.

TESTING ROLES

Testing or running an experiment provides information. For reliability engineering, anytime we learn about what fails or when failures occur we have the opportunity to design out failure mechanisms. We also have information that is useful to estimate the number of failures that will occur.

Testing takes many forms and addresses different questions. Testing can include

• functional testing – does the product do what we want it to do?,
• environmental testing – will the product work over the range of expected environments?,
• regulatory testing – will governing agencies approve the device?, and
• reliability testing – will the product work long enough or with a low enough failure rate?

Testing refines our knowledge by reducing the area of uncertainty. Well-crafted experiments allow us to shed a little light into those imperfect regions of our knowledge.

WHAT IS PERFECT TESTING?

If we follow this logic that conducing testing provides information, then if we do enough testing we will create perfect information.

Well, no. There are some problems.

First, we have to design the testing. We are straddled with the same imperfect knowledge as with this design challenge. If only we could construct testing that would reveal what we do not understand, then we could learn what we need to know.

Second, testing is commonly done during development on a sample, which is expected to contain information about the population of items not yet created. Beyond the statistical uncertainties resulting from sampling from a population, there are often real differences between what we test and production units. For example, do you box up and ship your products from your distribution center to your lab using all possible routes? Probably not. While moving a sample may not materially change the product, it has occurred on more than one occasion that it is important. What else are you assuming is not important?

Third, are we measuring what we need to measure to detect how what we do not know impacts the design’s performance? We cannot detect something unless we are looking for it. When we do not know what we’re seeking, it is difficult to recognize when a subtle symptom appears.

Fourth, sometimes an important failure mechanism is masked or hidden by other less important failure mechanisms. The simplest example is the difference in the accumulation of damage. In testing, if we attempt to accelerate the damage on a brake pad by using it more often and with higher loads, we may completely miss the damage of corrosion or material degradation. Both degrade braking force and the time to failure, yet without time to degrade during an aggressive test it is not prevalent in the test results. While the brake pad is in use the failure mechanisms may switch places on which causes the eventual failure.

We do not conduct perfect testing.

Is Testing Worth the Effort?

The short answer is “Sometimes.” Testing is worth the effort when done with forethought on what is and isn’t known, when the experiment has clear objectives and the results are useful for upcoming decisions, and when the testing focuses on allowing us to explore what we do not know.

Bio:

Fred Schenkelberg is an experienced reliability engineering and management consultant with his firm FMS Reliability. His passion is working with teams to create cost-effective reliability programs that solve problems, create durable and reliable products, increase customer satisfaction, and reduce warranty costs. If you enjoyed this articles consider subscribing to the ongoing series at Accendo Reliability.