Empirical Risk Minimization is an Incomplete Inductive Principle

Empirical Risk Minimization (ERM) only utilizes the loss function defined for the task and is completely agnostic about sampling distributions. Thus it only covers half of the story. Furthermore, ERM is equivalent to Bayesian decision theory with a particular choice of prior.