Every failure provides information. It provides time to failure, stress strength relationship, process stability and design margin types of information. In every case. Even failures directly related to human error.

A hardware intermittent failure observed by a firmware engineer should not be dismissed. Rather recorded, explored and examined.

A single intermittent failure, or glitch, may indicate nothing other than just a totally random glitch, or a design error that degrades over time causing 50% of units to fail in first three months.

For complex systems with the expectation of high reliability every failure seen during the prototype phase provides essential information if examined.

Statistics of single failure

Every failure has meaning. If you have 100 prototypes and one fails, that indicates a 1% defect rate. If we only have 10 prototypes, that one failure may indicate a 10% failure rate. And with two prototypes, a 50% failure rate. Sure, more prototypes would be nice along with the budget and time to thoroughly test each prototype.

That often doesn’t happen.

And, even with one prototype, we have failures. We learn something about the design or manufacturing process. The team adjusts and expects to minimize or eliminate the failure mechanism.

Without more samples or failures a single failure may indicate a very rare event or very common event. Detailed failure analysis will help understand the potential and the root cause, and we should know the expected failure rates before going to production.
Thus, every failure, even ones that are easy to explain away require investigation.

Here is another way to look at failures on limited prototypes. The chance that a rare or isolated event causes a failure is very rare. To have a reasonable chance to find a 1% defect rate we need to evaluate over 100 samples. So if only testing 10 samples, it is very likely that failure experiences will occur in 10% or more of the production units.

Do not ignore any failure during the prototype phase especially when hampered with very few prototypes to find the failures.

Indication of Variability

Single failure

A single failure in a complex system indicates that something is not right somewhere in the design or manufacturing process.

It is difficult to know from a single failure is it indicates a nominal behavior, a rare event, or something in-between. We also may not know if the issue is with the nominal performance centering or with the process variability.

Two failures

Two failures with the same component involved provide information on the nature of the problem.

The difference between the two item’s dimensions or performance is the range of the expected variation. Two values that are both below or above the specification indicates a very likely need to adjust the process to center the performance within the specification.

Two failures also indicate this was clearly not a one-time event.

Three or more failures

Three or more failures of an item allow us to calculate the standard deviation of the expected spread of the item dimension or performance.

Of course the more measurements we have available of items that lead to failure and are within specification the better. One tactic is to focus on gathering information of those items that have led a to failure, to gauge the nature and magnitude of the issue.

Indication of Importance

If a failure occurs and shuts down a system or device for a customer, that failure is important. A single failure of a prototype is likewise important — because it is a clear way to avoid failures that a customer may experience.

A prototype failure indicates the design margin or stress-strength relationship is not adequate. Of the one thousand parts in the system, this one failed. It marks that failed item as important. It may not be the only important part related to product reliability and may not even have been suspected as being important, yet it is on the important list now.

Every failure should bring attention to the item that failed, the root cause of the failure and specific ongoing steps to avoid the failures going forward.