Comparison of approaches

• Empirical Risk Minimization (ERM)

  • "Traditional" approach: find model that was most likely to produce the observed data
  • Minimize error on training data

• Structural Risk Minimization (SRM)

  • Minimize error of overfitting (by maximizing margin between classes)
  • Find the model that is least likely to be wrong upon generalisation

Notes: