2.4.3.   Random Subsampling

Random sub­sampling, which is also known as Monte Carlo crossvalidation [19], as multiple holdout or as repeated evaluation set [20], is based on randomly splitting the data into subsets, whereby the size of the subsets is defined by the user [21]. The random partitioning of the data can be repeated arbitrarily often. In contrast to a full crossvalidation procedure, random subsampling has been shown to be asymptotically consistent [17] resulting in more pessimistic predictions of the test data compared with crossvalidation. The predictions of the test data give a realistic estimation of the predictions of external validation data [22].