10.4.   Robustness and Comparison with Martens' Uncertainty Test

In the field of calibration by the PLS, a method called Martens' Uncertainty Test [32],[33],[41] has gained increasing popularity during the last two years. This test uses a jackknifing procedure with many submodels to identify non-significant variables. Thereby a statistics is setup for the regression coefficients of all variables and then those variables are eliminated, which are identified as being non-significant according to this statistics. The genetic algorithm frame­work and the parallel growing network framework introduced in this work use a similar method of selecting only significant variables but work the other way round. Instead of eliminating variables, the frameworks add variables according to a ranking, which was established by many submodels in a previous step, until the prediction does not significantly improve determined by a subsampling process. The frameworks are generally more conservative in terms of selecting variables compared with Martens' Uncertainty Test. The significance determined by the subsampling process in the second step of the frameworks can also be used to access the uncertainties respectively the robustness of the predictions. Thereby the standard deviations for the predictions of the subsampled test data by the different submodels are calculated during the subsampling process [267]. For example, the uncertainties of the predictions of the refrigerant data for the parallel growing neural networks framework (4^th row of table 4) were estimated as 0.17% for R22 and 0.14% for R134a in terms of subsampled standard deviations. For the evaluation of the same data by the genetic algorithm framework, the standard deviations are also low with 0.11% for R22 and 0.18% for R134a. The ternary mixtures of the alcohols measured by SPR showed uncertainties of 0.27% for methanol, 0.32% for ethanol and 0.34% for 1-propanol evaluated by the parallel growing neural network framework respectively 0.21% for ethanol, 0.25% for ethanol and 0.30% for 1-propanol evaluated by the genetic algorithm framework. Thus, the calibrations by the frameworks can be considered as generally being quite robust.