Explainable AI
Recent years have witnessed a surging interest in AI and its applications in various fields have generated fruitful outcomes: computer vision for face recognition and self-driving cars, natural language processing for chatbots, and user profiling for recommender systems. Most of these breakthroughs are made possible by machine learning, a statistical framework that allows an algorithm to learn from big data and thus makes it a more intelligent agent for a specific task. Despite its empirical success, machine learning as a methodology suffers from critical interpretability issues: while the trained intelligent agent performs fairly well in most cases, its underlying logic remains unintelligible to researchers.
This lack of interpretability results in two problems: the lack of reliability and fairness. As long as the working logic of AI remains non-transparent to us, it is impossible for humans to predict when it will fail and this could be a fatal flaw in life-critical scenarios like self-driving. A 100 percent accurate self-driving AI sounds unrealistic because human beings can go wrong as well. What is truly unacceptable is a system that is agnostic of its tendency to make mistakes. Another resultant problem is that the result of AI analysis always demands further human interventions because its data-based decisions could involve unwanted social biases. A facial recognition system might predicate that skin color is associated with the crime rate. Though it seems plausible to AI due to statistical observations, it is unacceptable to social warfare.
Paradigms underlying these problems fall within the so-called explainable AI field, which is now widely acknowledged as a crucial feature for the practical deployment of AI models. The ultimate goal is to establish responsible artificial intelligence, a methodology, and also implementation of methods with explainability, accountability, and fairness, while still maintaining a high level of performance. A natural idea is to construct a new model aiming to explain a given machine learning model, which could be accomplished straightforwardly by analyzing the inner structure of the off-the-shelf model with mathematical tools. Just as the biologists trying to understand the human brain by decomposing it into multiple functional units, we may establish technical tools to figure out the specific role each module plays in an AI system. Once we can recognize certain typical patterns, we can explain how the system accomplishes certain goals by coordinating the submodules and further manage to construct a more complicated system by assembling small pieces together. This methodology is easy to handle because we decompose the construction of an AI system to two decoupled phases, training and interpreting. However, how we balance the trade-off between the performance of these two phases remains a challenge.
A more radical approach to make AI explainable is to train it in a sophisticated way from the very beginning, which is often achieved by teaching it common sense. The advantage of this approach is that we can differentiate different levels of transparency that a machine learning model can feature by itself, and come up with models that are already explainable by design. However, how we should define and represent common sense remains an intriguing yet challenging problem.