|Michael Tsang||University of Southern California|
|Hanpeng Liu||University of Southern California|
|Sanjay Purushotham||University of Maryland Baltimore County|
|Yan Liu||DiDi/University of Southern California|
Neural networks are known to model statistical interactions, but they entangle the interactions at intermediate hidden layers for shared representation learning.
Neural networks are known to model statistical interactions, but they entangle the interactions at intermediate hidden layers for shared representation learning. We propose a framework, Neural Interaction Transparency (NIT), that disentangles the shared learning across different interactions to obtain their intrinsic lower-order and interpretable structure. This is done through a novel regularizer that directly penalizes interaction order. We show that disentangling interactions reduces a feedforward neural network to a generalized additive model with interactions, which can lead to transparent models that perform comparably to the state-of-the-art models. NIT is also flexible and efficient; it can learn generalized additive models with maximum $K$-order interactions by training only $O(1)$ models.