DEXTER: Diffusion-Guided EXplanations with TExtual Reasoning for Vision Models
By: Simone Carnemolla , Matteo Pennisi , Sarinda Samarasinghe and more
Potential Business Impact:
Explains how AI sees things without seeing real examples.
Understanding and explaining the behavior of machine learning models is essential for building transparent and trustworthy AI systems. We introduce DEXTER, a data-free framework that employs diffusion models and large language models to generate global, textual explanations of visual classifiers. DEXTER operates by optimizing text prompts to synthesize class-conditional images that strongly activate a target classifier. These synthetic samples are then used to elicit detailed natural language reports that describe class-specific decision patterns and biases. Unlike prior work, DEXTER enables natural language explanation about a classifier's decision process without access to training data or ground-truth labels. We demonstrate DEXTER's flexibility across three tasks-activation maximization, slice discovery and debiasing, and bias explanation-each illustrating its ability to uncover the internal mechanisms of visual classifiers. Quantitative and qualitative evaluations, including a user study, show that DEXTER produces accurate, interpretable outputs. Experiments on ImageNet, Waterbirds, CelebA, and FairFaces confirm that DEXTER outperforms existing approaches in global model explanation and class-level bias reporting. Code is available at https://github.com/perceivelab/dexter.
Similar Papers
Zero-Shot Textual Explanations via Translating Decision-Critical Features
CV and Pattern Recognition
Explains why computers see what they see.
DEXTER-LLM: Dynamic and Explainable Coordination of Multi-Robot Systems in Unknown Environments via Large Language Models
Robotics
Robots plan and do new jobs themselves.
A Novel Framework for Automated Explain Vision Model Using Vision-Language Models
CV and Pattern Recognition
Shows how computer "eyes" make mistakes.