Why this matters
Anyone can train a model that looks brilliant on the data it learned from. The real question is how it performs on cases it has never seen, because that is what happens once it is in use. Employers list "evaluating models" and "testing and validating models" because a badly evaluated model can approve bad loans, miss sick patients or waste a company's money, while everyone believes it works.
Overfitting
Overfitting is when a model memorises its training data, including its noise and accidents, instead of learning the general pattern. Think of a student who memorises last year's exam answers word for word. They score full marks on that paper and fail a new one. An overfit model does the same. The opposite problem, underfitting, is a model too simple to capture the pattern at all. In books and interviews you will also hear these called high variance (overfitting) and high bias (underfitting). This bias is a different idea from the bias number inside a model and from unfair bias.
The signal of overfitting is a big gap: excellent scores on training data, much worse scores on test data.
Train, validation and test
- The training set is what the model learns from.
- The validation set is used to compare models and settings while you work.