What is Machine Learning?

Machine Learning and Artificial Intelligence

Machine learning is a form of artificial intelligence, and broadly refers to using a training algorithm to develop a model that can make a prediction. The model is trained, it makes a prediction, if the prediction isn't accurate enough, it's trained again. So on so forth until you achieve the desired result. Machine learning is the focus of the current wave of Artificial Intelligence, and the theory supports just about every AI you may have heard about, from Neural Networks to Random Forests.

Artificial intelligence is used in many industries all over the world to help make complex decisions. Specifically, these tools are called Decision Support Systems, and not all of them rely on machine learning, but many do. Machine Learning Decision Support Systems, or MLDSSs, are the focus of the XAI industry at the moment. XAI researchers apply the theory behind Expert System Explanations onto the technical challenge of MLDSSs.

Types of Models

One of the main struggles with XAI is the fact that MLDSS models can range wildly. The technical differences between the different models makes it difficult to implement a consistent framework. Generally, different ML models can be split into two different XAI camps: high interpretability and low interpretability.

High interpretability models are relatively easy to construct explanations for. They consist of models like decision trees, knn models, etc. These models lend themselves well to explanations, because their technical makeup isn't super complicated. Highly interpretable models fall into one branch of XAI, known generally as transparent models. Transparency is just one of the goals of XAI, and can be broken down into three parts.

• Simulability: The ability of a model to be simulated by a person, or thought of by a person. How easy is it for people to understand, and possibly recreate, the logic of the model? For example, expert systems have great simulability, as they are just long protocols of rules.
• Decomposition: The ability of a model to be broken down into disparate parts. How easy can the different parts and factors of the model be separated and explained? Does removing or altering one aspect of the model alter the output, and is that change easy to express?

• Algorithmic transparency: The ability of a model to be entirely explained through mathematical analysis methods. Can a person clearly follow all the steps and processes of the model?

Obviously, all of these factors vary from user to user. An ML engineer is going to find certain models much more simulable than an HR representative attempting to hire someone. The goal of XAI is to take these concepts and use them to make already interpretable models more understandable by end users. This will then lead to greater utility, particularly for laypeople, and improve the user experience.

While transparency is an important goal in XAI, a bulk of the research has been focused on models with low interpretability. Not only is crafting an explanation for these models much more difficult, but the accuracy of the models tends to rise as interpretability lowers. One of the most powerful models for a wide variety of ML tasks is the Neural Network, and it's infamous for its so-called "black-box" problem.

To put it simply, ML engineers build the training algorithm for the neural network. They have control over training weights, and how large the network is, the inputs of the data, and the requested output. But the actual decision-making process of the network is a bit of a mystery. It's called a black box because we know what goes into the model and what comes out, but the inside is anyone's guess. Highly interpretable models are referred to as white boxes in contrast to the black box's transparency issues. Creating explanations for black-box models is thus much more difficult than creating explanations for white-box models. These explanations are referred to as Post-Hoc Explanations, because they are generated after the prediction has been made, and are typically specific to the particular model and input data.

Post-Hoc Explanations are much more complicated than Transparency-based explanations. They can vary from model-specific explanations, which are tailored to specific ML models, to model-agnostic explanations, which can be less helpful but run on many types of models. 

Post-hoc explanations can also vary in terms of when they occur, and what they're explaining. Some are considered local explanations, designed for a specific model in the instant it's run, with the specific input data. Some are global explanations, which explain the model as a whole, how it works and the parts that make it up (Du, 2020). 

Explanations can be textual, or visual. They can explain in human language how the model came to it's decisions, or they can simply give a confidence score of how accurate the model may be. Some post-hoc explanations rely on building a proxy-model that is more understandable to humans, others will produce a separate example to help users contrast and contextualize the decision (Mosqueira Rey, 2022, Confalonieri 2021). 

On the whole, the industry hasn't decided on the best modality for explanations. It doesn't help that MLDSSs can be such different models, and are used in vastly different fields by people with enormous ranges of expertise. The consensus thus far is that there simply isn't a one-size-fits-all answer, and that users need to be considered more in the design process.