Home / K-Fold Cross-Validation

K-Fold Cross-Validation

Partitioning mechanics, variance reduction, and cross-validation procedures

K-fold Cross Validation

Datasets are usually partitioned into training set and testing set to make sure the testing set is unseen by the training set.

Ideally, the more data for each side, the better. But since they are partitioned from the same pile, they compete.

Process:

  1. Divide the dataset into K sets evenly
  2. For each iteration (fold), choose one set to be the testing set and the remaining k-1 sets becomes the training set
  3. Pit them against each other.
  4. Repeat for a total of k times, changing the pick each time.
  5. Average the scores from all the validations