Transparency and the Future of Artificial Intelligence

The problem is not that artificial intelligence (AI) can make foolish errors; it is that sometimes AI can seem really smart. This makes people think there is some underlying ability that doesn’t really exist. To take AI to the next level, we’ll need to look inside to consider why AI is sometimes really smart . . . and sometimes not. 

The Issue of Transparency

To understand AI transparency, one needs to have basic information about how today’s AI works. Take the example of an image classifier, which is intended to look at images of dogs and wolves and identify which is which. To make such a system, we first create a training set of thousands of images that are tagged as being either dogs or wolves. Much of this sample set is fed to the selected AI algorithm, and most such algorithms have many design options and parameters that are tweaked until some semblance of correct results are achieved. We validate the AI by submitting other samples from the training set to verify that the results match the tags. From this, we can calculate an error rate and say that for our application, AI is working with some degree of confidence. The degree of confidence required depends on the application. For AI making life-and-death decisions, we want a very high confidence. For categorizing dogs and wolves, we might not care as much. Developers often go through many iterations of tweaking parameters to reduce the error rate and improve confidence.

It is easy to see where problems can be introduced. Errors can be made in the tagging of the sample set. The sample set may not be representative of the decision space as a whole. Selecting options and tweaking parameters may not be optimal. The AI algorithm itself is a deterministic system. That is, given a specific set of inputs, you’ll get a specific set of outputs (except some algorithms incorporate a small degree of randomness, which further clouds the issue). So, the algorithm itself is seldom at fault, but the selection of the algorithm can be.

Historically, the transparency issue is that once an AI system is working, developers don’t know what’s going on inside. It’s essentially a black box with inputs and outputs. Developers can see that outputs usually correspond correctly to inputs, but they don’t know why. Then when some erroneous output occurs, they can’t know about that either. This makes the system difficult to analyze. 

Breaking the Opacity Barrier

Enter new programs that can analyze the inner workings of AI to explain what’s going on. In their 2016 paper, “Why Should I Trust You?” Explaining the Predictions of Any Classifier,” Marco Tulio Ribeiro, Sameer Singh and Carlos Guestrin present some surprising results. When a husky dog was misidentified as a wolf, they probed the AI and learned that the reason for the miscategorization was the snow in the image. Many of the wolf pictures in the training set contained snow, so the AI developed a bias that pictures with snow were likely to be wolves.

One might think that probing the internals of the AI would be the way to go, but that approach remains elusive. Instead, a program such as the open-source LIME used in this example makes small changes to the input image to see what changes alter the output. Changing areas of snow to something else changed the result, while changing areas of the dog didn’t. That means that the AI’s decision must have been based on the snow.

If we wanted to create an AI that analyzed symptoms to determine whether a patient suffered from the coronavirus disease COVID-19 and its likely severity, we could collect patient data, build a model and run the AI. With a program like LIME, we could discover that having a fever is significant, but the patient’s height is likely irrelevant. But what about a patient’s weight? Perhaps overweight people are more likely to have underlying conditions that could contribute to the severity of their illness. This leads us to the ethical issues of AI transparency. If AI tells us that African Americans are disproportionately likely to die of COVID-19, is that racial bias? If we use that information to test and treat one racial group more carefully, that’s likely a good thing. If we use identical information to charge one racial group more money or deny health insurance, that’s not. Ethical issues with AI and its lack of transparency often revolve around how results are used and misunderstandings about the limitations of the results.

The key thing many AI professionals are learning from improved AI transparency is that AI is often making decisions for reasons we didn’t expect, and it is not making decisions for reasons of intelligence. 

The Present State of AI

Where does that leave us? We’re learning that today’s AI looks more like sophisticated statistical analysis and less like actual intelligence. What will we need to add to get to genuine intelligence? How do we get AI to understand that the presence of snow may or may not mean it’s a wolf, but the presence of children (but no cage) make it very likely to be a dog? We know that being African American makes one disproportionately susceptible to coronavirus for one set of reasons, but disproportionately susceptible to sickle cell anemia (which is genetic) for entirely different reasons.

In 1988, distinguished roboticist Hans Moravec wrote his famous paradox that, loosely paraphrased, says that the simpler an intelligence problem seems, the more difficult it is to implement. Saying something is “so simple any three-year-old can do it” is the absolute kiss of death. Consider that any child can learn a language from scratch, understand the passage of time, know the basic physics that round things roll, square things don’t, all objects fall downward and not upward, understand cause-and-effect, and so on. Further, children can learn with a small sample set of untagged and ambiguous or erroneous information. It’s unrealistic to expect that AI will match these abilities with their current algorithms. 

Context and a Multisensory Approach

The future of AI relies on bridging the gap between our superhuman narrow AI and the common sense of any three-year-old. The child has the advantage of learning everything in the context of everything else they learn. She knows from playing with blocks that things fall down, not up, that a stack must be built before it can fall down, and that blocks will never stack themselves. This is in contrast to a dog that might spontaneously do many things. AI has none of this context; images of dogs or blocks are just different arrangements of pixels.

The child can see and hear things, touch things, smell and taste them. Further, she can move things around and move herself to get a different point of view. A program that is specifically image-based or one that is primarily word-based (like IBM’s Jeopardy-playing Watson) will not have the context of a “thing” that exists in reality, is more-or-less permanent, and is susceptible to basic laws of physics.

Getting this basic context will come from multisensory systems and robots. AI transparency is saying that some of the limitations of today’s AI requires robotics for this contextual understanding to emerge. Does that mean that the concept of general intelligence in a static supercomputer is impossible? Not at all. An important distinction between future AI and children is that the AI intelligence can be copied. It’s obvious that we can clone the learning of one robot to another. But a robot’s experience and understanding (should that ever occur) can also be transferred to a supercomputer that doesn’t have robotic capabilities. In the same way that when you close your eyes you still understand what it means to be able to see, a supercomputer can understand the meaning of vision if it has imported robotic vision experiences. That means the supercomputer of the future will be able to scan the images and text of the Internet and be able to put it all into the context of real-world experiences previously acquired by other robotic systems.

Improving AI transparency will lead to many improvements to AI systems, as we will be better able to identify and eliminate biases in our training sets, helping us to select and tweak AI algorithms. Longer term, however, AI transparency is highlighting the limitations of today’s AI approaches, opening the door to new algorithms and approaches to create true intelligence in the future.