James Toney has more than thirty years of experience in the quality profession. His clients have included the U.S. Department of Labor, IRS, Federal Aviation Administration, U.S. Navy, Housing and Urban Development, Pension Benefit Guaranty Corporation, the Federal Deposit Insurance Corporation, and Health and Human Services. Commercial clients included financial, telecommunications, hospitality, educational, and healthcare organizations.

He has provided ISO 9001 assurance on business performance and quality improvement. He has also consulted on COSO and ISO 31000– based risk management issues. For fourteen years he was a Malcolm Baldrige Performance Excellence Framework Examiner primarily in healthcare. He was in the inaugural class of Carnegie Mellon University’s Chief Risk Officer certificate program. He is a Certified Enterprise Risk Manager, a Six Sigma Black Belt, and a Certified Quality Auditor. For eight years he was an adjunct Professor at Marymount University in Arlington Virginia.

1. You were a Baldrige Performance Excellence Framework Award Examiner for fourteen years. Given this experience, how would you assess the effectiveness of the Performance Excellence Framework in promoting “Performance Excellence”?

I have to qualify my response by saying virtually all of my Baldrige-related Examiner experience was in healthcare.

Effectiveness of the Performance Excellence Framework is driven by five factors: (1) Continual refinement of the Framework itself in terms of relevant content, e.g., adding ERM for example.  (2) Promotion of the value of the Framework. The American Health Care Association does excellent work in this area with their members.  (3) Required annual training of Examiners to introduce changes, improve Examiner-to-Examiner consistency particularly in scoring, and reinforce the Criteria associated with the Framework.  Although not often heard, I prefer to refer to the Framework’s Criteria as requirements as this term is unambiguous to me and seems easier to communicate.  (4) Effort applicants put into understanding, making changes and documenting what they have done. And finally, but essential.  (5) Framework effectiveness is driven by Examiner Team members’ experience, knowledge of the Criteria, insight into the business, and ability to craft targeted action-oriented feedback for an applicant.

Without a large contingent of motivated Examiner volunteers, quality award programs could have never reached the level of success they have achieved.

Every organization that I was fortunate enough to visit as a member of a Site Visit Team was a better organization for having “embarked on the journey.”  Almost all organizations’ applications that I reviewed that had acted on previous feedback was a better organization.

When I say organization, I am not referring to bricks and mortar, but to the people who develop new knowledge and skills from the Baldrige Criteria.  Examiners often comment on an organization’s maturity, which has always amused me because they are really commenting on the management teams’ skills.

My assessment is that the Baldrige Performance Excellence Framework effectively promotes “performance excellence” for those willing to commit to adopting and investing in it as a management framework.  However, it is not and never has been a quick-fix solution.

2. In 2015, the Framework added Enterprise Risk Management (ERM) to the evaluation criteria. From your experience, how effective has the Performance Excellence Criteria been in encouraging the private sector to adopt ERM?

Up to 2019, my observation was that the Performance Excellence Criteria ability to encourage the private sector to adopt ERM has been spotty at best.  ERM was explicitly added to the 2017-2018 Criteria so there probably was not enough time for organizations to recognize and respond to ERM through the Baldrige application for two reasons.

First, there is a time lag between addition of ERM in the 2017-2018 Criteria and effective Examiner review and applicant response to feedback in follow-on applications.  Some Quality Award granting organizations continued to use the 2017-2018 criteria in 2019.  Second, few applicant organizations and even Examiners were really familiar with Enterprise risk Management (ERM).

The chance that an applicant or an Examiner was knowledgeable of The Committee of Sponsoring Organizations of the Treadway Commission (COSO) Enterprise Risk Management — Integrating with Strategy and Performance (2017) framework, which was published in September 2017 – or even the ISO 31000 Risk management – Guidelines (2009 or 2018 versions), was very low at the time.

COSO 2017 explicit linkage of risk to strategic objectives suggest an opportunity to explicitly discuss risk in the Strategic Planning Category of the Criteria, and this could be easily integrated into a strategic planning process.  Some like to say, if you have strategic objectives, you have risk.

The ISO 31000:2009 version provided clearer figures depicting a risk management framework and risk management process than the 2018 version.  Both the risk framework and process are easily adaptable by an organization using the Baldrige Framework.

The Baldrige Criteria mention of risk has been largely confined to the higher Multiple Criteria Level.  The nature of this Level suggests a higher performing organization.  Those beginning a Baldrige journey would most likely not address or poorly address risk and in particular ERM.

Some hospitals I am aware of have a risk manager, but the focus is typically responding to litigation, minimizing adverse publicity, and settling before a case reaches trial.

From my research and experience, healthcare delivery organizations have not practiced risk management as one might find in high reliability organization such as a commercial air carrier where risk and hazard identification and mitigation and safety assurance are essential.

In summary, we should expect to begin seeing more emphasis on ERM within the Baldrige community, and this might be accelerated by the COVID-19 pandemic.

3. Given your practical and academic perspective, how would you assess the state of the quality profession today?

The Quality Profession as we have known it is clearly beyond its heyday, maybe 25 or more years beyond it.  The decade of the 1980s launched the quality juggernaut that captured the interest of Americans and launched my career.  If you drove an automobile in that decade, you were concerned about quality.

Let’s turn on the way-back machine:  in 1980 NBC broadcast a program in prime time called “If Japan Can, Why Can’t We?”  In 1984 Congress designated October as National Quality Month.    In 1988, the United States Secretary of Defense announced adoption of Total Quality Management.  Around that same time, the United States Navy promoted its own quality initiative – Total Quality Leadership. And Motorola engaged in and refined “Six Sigma” in the 1980s, culminating with establishment of Motorola University in 1992 to promote their methods.

The beginning of significant change for the Quality Profession was, in my opinion, the publication of Reengineering the Corporation: A Manifesto for Business Revolution, by Michael Hammer and James Champy, in 1993.  Suddenly, reengineering captivated the business world with the promise of big changes, quickly.

I remember searching the university library periodical reference systems for “TQM” and “Reengineering” in the 1993-1995 timeframe.  There was no doubt then that quality was being eclipsed by reengineering.  The rise of reengineering and decline of quality was evident in the declining number of TQM-related articles and increasing reengineering articles.

Inertia was in play though, as ASQ membership peaked in 1996 to approximately 135,000. In 1997, the American Society for Quality Control (ASQC) changed its name to the American Society for Quality (ASQ).

And, let’s not forget the emergence of the commercial internet and associated technology in the 1990s that changed the world.

So, here we are just about to begin the third decade of the 21^st Century.  Where are the business drivers focused on quality like those of the 1980s?  I do not know of any.  We are in an information economy now and a focus could be assuring decision quality data.

The need for some quality function will always exist, but in what name and form is the question.

4. Looking forward, what changes will quality professionals need to make in their skill sets to remain competitive in the job market?

Quality professionals will need to change their thinking about their profession.  They will have to make the adjustment from being a quality professional to one that will include but not be entirely devoted to quality, within the broad functions of governance, compliance, risk and perhaps data science.  One has to offer in demand skills to have a job much less a career, and to remain competitive that means learning new skills.

Let’s look at some examples.  Governance might infer Baldrige experience and knowledge, yet few of the Baldrige Examiners I met had knowledge of contemporary risk practices.  Compliance infers audit yet many current quality professionals may not have audit training or experience.  Although some did have experience with the Joint Commission Accreditation Standards.

Risk offers career opportunity, but to be competitive one needs certificates of competency, and experience.  This includes various Risk Frameworks but to be truly competitive one needs knowledge of specific business sectors such as finance or technology. Quality ties in to risk by serving as the means to assure the risk management and mitigation function is working as planned.

How many current quality professionals are competent with software such as Python, R, and Tableau or the National Institute of Standards and Technologies (NIST) risk and security frameworks for IT?  Not many, and that is a problem for those who are current quality professionals who want to stay employed or for those who aspire to be a quality professional.

Quality will be a secondary add-on skill set in the future, not a career in itself.  The future profession will require multiple skills, e.g., those associated with governance, risk, compliance, and IT/Cyber security.

Perhaps the most critical skill will be that for data analysis to extract format, summarize, graph, and analyze data.  A recent experience required access, extraction, and manipulation for analysis and on-going process monitoring – which would not have been successful without a Tableau expert on the team.