Robust Neural Networks for Causal Invariant Features Extraction

Most machine learning approaches exploit correlational relationships in a training data set to predict a target variable. When these correlations are spurious or unreliable, this hampers the ability to generalize learned models to new environments. In contrast, models exploiting causal relationships between features and the outcome generalize better across environments. In this paper, we posit that these robust causal relationships can be identified by finding features that, when conditioned upon, render the outcome invariant across environments—that is, when the outcome is independent of the environment given a set of selected features with lower dimensions. We propose a neural network architecture for this task, comparing it with several existing approaches to exploit the causal invariant property, with a discussion on their motivations in a unified framework. Empirically, we perform a simulated experiment to demonstrate and compare the performance of the proposed method to the existing approaches. Finally, we measure its efficacy in a real world data set for advertisement click prediction.