TRAD: Task-agnostic Representation of the Activation Space in Deep Neural Networks

Abstract

The activation space of deep neural networks is studied for multiple purposes, e.g., fairness and robust explanation, and downstream tasks, e.g., detection of adversarial attacks and synthesized content. Given the size and heterogeneous nature of deep neural networks (DNNs), finding a generalizable and efficient representation of activations in DNNs is crucial. Empirical p-values have been used to quantify the relative strength of an observed node activation compared to activations created by already-known inputs. Nonetheless, retaining previous inputs for these calculations results in increasing memory resource consumption and privacy concerns. To this end, we propose TRAD: Task-agnostic Representation of Activations in DNNs using node-specific histograms to compute p-values of observed activations without retaining already-known inputs. TRAD demonstrates promising potential when validated with multiple network architectures across various downstream tasks and compared with the kernel density estimates, and brute-force empirical baseline. TRAD reduces memory usage by 30% with faster p-value computing time, while maintaining state-of-the-art detection power in downstream tasks.