Understanding the intended use of artificial intelligence: How to get what we want out of AI algorithms

In a Wednesday morning session, “Data is Dirty: What AI Algorithms Actually Do and Why We Need Explainable Artificial Intelligence,” Robert Thomson from the United States Military Academy took Virtual Analytics Conference attendees down the path of AI algorithms. Despite being driven by science, they are still undoubtedly flawed, but why? Humans are behind the algorithms, and humans bring bias, whether intentional or not.

Thomson argued that AI algorithms are still fallible in unpredictable ways, and Explainable Artificial Intelligence (XAI) frameworks are needed to make this right. He explained that they should be required to not only understand why AI algorithms make decisions, but also to work with human operators so these explanations provide the right quantity and quality of information.

In essence, these algorithms and those who write them need to understand the expertise and goals of the user for the algorithms to gain widespread adoption in mission-critical environments.

Algorithms that look at historical data, fairness and fair AI are a challenge. Thomson warned that if you don’t understand what your algorithm is doing, you could be missing something.

Humans are bias, so it comes down to what data analysts are doing with their data.

Thomson presented several techniques and a set of requirements for robust XAI systems. He described a use-case for cognitive models to act as a bridge between humans and AI. Thomson noted that the biggest concern with AI is it is either amoral or biased, but it’s not just the algorithm itself, there is also bias in the data. Thomson’s main point is that these algorithms can’t be used outside their intended domain.

This work illustrates the need for XAI so people know what goes into the algorithm and what comes out, and to verify the algorithm is working as intended.

Thomson said the goal is to make algorithm models more explainable.

“I don't think the AI is all that well understood yet, at least in the case of neural networks and reinforcement learning-based approaches. I agree that understanding humans is still the final frontier, but in aggregate we do understand a lot about human behavior, while predicting individual humans can be very challenging,” said Thomson.

In conclusion, Thomson noted it is about integrating XAI into the mental workload.