Memory Maps to Understand Models

Dharmesh Tailor

University of Amsterdam

What do models know and how? Answering this question requires exploratory analyses comparing many models, but existing techniques are specialized to specific models and analyses. We present memory maps as a general tool to understand a wide-range of models by visualizing their sensitivity to data. Memory maps are extensions of residual-leverage plots where the two criteria are modified by easy-to-compute dual parameters obtained with a Bayesian viewpoint. The new criteria are used to understand a model's memory through a 2D scatter plot where tail regions often contain examples with high prediction-error or uncertainty. All sorts of models can be analyzed this way, including not only those arising in kernel methods, Bayesian methods, and deep learning, but also the ones obtained during training. We show use cases to compare data-characterization methods in deep learning, analyze training trajectories, and diagnose overfitting.