Maximizing Business Performance with Explainable AI
Artificial Intelligence (AI) is hot. Businesses in every industry are busy developing strategies to incorporate AI into their operations. For the most part, that is a smart move. The potential advantages businesses might gain from AI are simply too great to ignore. They are so great that they dwarf any potential downsides that might come with being an early adopter of today’s still-developing AI technology.
However, even today, there is a right and a wrong way for businesses to go about adopting AI. The wrong way, as plenty of companies have already learned, is to rely on so-called black-box AI solutions. The name describes AI technology that is particularly opaque, meaning that users and stakeholders cannot see its inputs or its inner workings. Such solutions carry unacceptably high risks to businesses and can easily harm both their bottom line and reputation.
The right way for businesses to adopt AI is to make the conscious choice to embrace explainable artificial intelligence (XAI). These are AI solutions that allow end users to see why and how they arrive at their outputs. XAI solutions give businesses the same tremendous upside that comes from black-box AI solutions but without some of the major risks those products entail. To elaborate, this article covers the risks of black-box AI, why XAI solutions are a much better option, and what businesses need to do to go all-in on XAI.
The Problem with Black-Box AI
In the earliest days of commercial AI development, almost every AI-powered product on the market had black-box AI under its hood. This happened because the companies building the products needed to protect their proprietary algorithms from theft by competitors. The problem for the businesses using the products was that there was no way to audit the inner workings of the AI to make sure it was doing what its developers claimed it would do.
As a result, plenty of early AI adopters had a rocky first experience with the technology. For example, Jupiter Hospital in Florida was one of the earliest adopters of IBM’s Watson for Oncology software. Within months of beginning its use, doctors at the hospital noticed that the system had a frighteningly high error rate. It made medication recommendations that, if followed, could have killed patients. It misdiagnosed patients and produced erroneous treatment plans. The system was so bad that one exasperated doctor at the hospital called the system something that we won’t repeat here. The hospital discontinued its use shortly thereafter.
The issue with Watson for Oncology, as is the case with many black-box AI systems, was that it relied on flawed training data. Its developers used hypothetical scenarios rather than real patient data to inform the AI’s decision-making processes. That introduced biases into the system and made the AI blind to a myriad of real-world situations it never encountered in its curated data. Additionally, since the doctors using the system had no way to see how it arrived at its conclusions, they could not spot the flaws to correct them.
The Business Risks of Black-Box AI
The example above is but one of countless black-box AI failures. It is such a common occurrence that we could write an entire library’s worth of articles detailing each one. Instead of doing that, we’ve distilled some of the most pertinent risks posed to businesses. The business risks of black-box AI include:
- Improper decisions based on erroneous data or logic.
- Reputational damage from AI errors affecting customers.
- Legal and regulatory penalties arising from flawed AI decision-making.
- AI-induced inefficiency and delays in rectifying problems due to poor operational visibility.
- Systemic vulnerabilities that go overlooked in the risk management process.
In short, black-box AI systems and products create major blind spots for businesses in a variety of key areas. When things go awry, the business can suffer dire consequences.
The Explainable AI Solution
Explainable AI, by contrast, does not create blind spots for businesses. It is transparent and does not attempt to obscure its functionality. That means stakeholders at every level have the information needed to audit how XAI systems function. To make certain such systems function as intended, it is possible to track their outcomes back to their origins. Therefore, stakeholders can understand how an XAI system works, which gives them confidence in its outputs. It is an approach that not only eliminates many, if not all, of the downsides of black-box AI but also comes with some massive benefits, including:
- Reducing incidences and impacts of model bias
- Reducing errors, particularly recurring ones
- Providing clear accountability for decision-making
- Reducing compliance and regulatory burdens
- Increasing trust and confidence among employees and customers
In short, XAI uses a more human-like decision-making process—one that is both understandable and does not resist efforts at investigating and improving it. That makes XAI perfect for most business applications, but specifically, ones that come with high stakes. For example, XAI works especially well in financial workflows. It can dramatically speed up operations while still allowing human experts the opportunity to check outputs for errors and correct them. It is also a good fit for customer-facing operations since it allows operators to monitor and adjust for any perceived or real biases in its outputs.
The Challenges of Explainable AI
Even though XAI offers significant advantages over black-box AI systems and products, it is not perfect. It does not manage to eliminate every possible problem that businesses might encounter while integrating and using AI. For example, XAI systems are still complex and difficult to comprehend, even though stakeholders can see all their moving parts. That means reaping the benefits of XAI’s transparency still hinges on a business’s staff having the skill to exploit it.
Also, the transparency of XAI systems can pose some data privacy challenges, too. If an XAI system processes data that includes sensitive or personally identifiable information (PII), for example, businesses then must exercise strict control over who has access to their AI systems. By default, XAI might expose such data to anyone looking into its operations.
Lastly, XAI offers businesses the ability to discover biases in their algorithms, but this will not happen by default. It requires the careful and constant work of data teams that monitor data inputs, algorithmic operations, and conclusions looking for evidence of systemic bias. If the people looking for bias have their own biases, those will still end up reflected in the outputs of an XAI system.
How Businesses Can Prepare for Success With XAI
The good news is there are best practices that businesses can use to address some of the challenges that come with XAI. They can also help any business improve the likelihood of succeeding in its XAI initiatives.
The first step is to form an overarching governance committee to control all aspects of the business’s XAI rollout. The committee’s central task is to set standards of explainability that all XAI products the business uses must meet. To make sure those standards are realistic, the committee should gather input from the employees and managers whose departments would interact with prospective XAI solutions. In short, those stakeholders can inform the committee of the kind of process visibility an XAI solution must exhibit for it to remain understandable and auditable from start to finish.
The next step is for the business to invest in the right training to provide its employees with the knowledge they need to make use of XAI tools. After all, XAI is not worth much if the people using it cannot understand it. Such training should include instruction on the use of XAI as well as its mechanics. That will ensure employees know how to make the best possible use of the tools at their disposal as well as how to spot potential problems with the XAI systems they are using.
Lastly, businesses should take a measured approach to XAI adoption. They should resist the urge to adopt new XAI tools unless there is a clear business use case and the prerequisites above are already satisfied. Failure to do so can turn the use of XAI into a liability, just as it would have been with a black-box solution.
Moving Forward into the XAI Future
When done well, the adoption of XAI should prove advantageous to the vast majority of businesses, regardless of industry. It is worth pointing out that patience and care are essential to getting everything right. That is where Outsource IT comes in. We offer comprehensive IT consulting services to help businesses develop various IT initiatives and execute them on time and on budget. Our experienced IT experts can be your organization’s biggest ally as you venture into the realm of cutting-edge technology. To learn more, contact an Outsource IT account manager today.
