ValidateModel

Location: ..\cubatch\Common\ValidateModel.m

function Validation = ValidateModel(FitFun,PredFun,X,Y,Factors,Method,PreProcess,Options,varargin);

Description:
Validate a model (either in regression or decomposition) and saves the results in ModelOut.validation.
With respect to FitModel some further parameters are necessary and their number depend on the type (Method) of validation chosen.
 

Inputs:
FitFun: function fitting the desired model to the data. It can be a string (for MatLab© 5.3 or superior) or  a function handle (for MatLab© 6.x or superior).
PredFun: function computing the predictions for the desired model. It can be a string (for MatLab© 5.3 or superior) or  a function handle (for MatLab© 6.x or superior).FitFun and  have specific requirements with respect to input and output
X: double array relative to the calibration set
Y: double array relative to the calibration set (it must be empty if it is a decomposition method) 
Factors: number of factors to extract (it can be a single number or a vector depending on the desired model)

Method:
'loo' => Leave One Out
'nboo' => Naïve Bootstrap
'rboo' => Residuals Bootstrap
'test' => Test Set Validation

PreProcess: structure with two fields: 'modx' and 'mody'; each of them is also a structure with two fields: 'cen' and 'scal'.
'modx' defines the preprocessing relative to X, while 'mody' refers to Y.
The two fields 'cen' and 'scal' are vectors with the same number of elements as the modes of the relative array (e.g. if X is 3-way PreProcess.modx.cen and PreProcess.modx.scal must be 3 x 1 (or 1 x 3) vectors. For more information on the values that these two vectors can be given check the help of nprocess (n-way toolbox).

Options: if it is set to 1 RecFac and PermFac are applied to remove the permutational indeterminacy between a model and all the replicates

varargin: additional inputs (e.g. Options, Constraints, etc) handled directly by FitFun and PredFun.
Depending on the Method the first one or two elements must be used for additional validation parameters:
  1. Leave One Out: no specific requirements are made on varargin
  2. BootStrap: the first element of varargin must be the number of replicates.
  3. Test Set Validation: varargin{1} is the index, within X, of the samples/batches (horizontal slabs) that compose the test set. varargin{2} are the model parameters, i.e. the structure ModelOut.model.

 


Outputs:
Validation_Results: the results of the computations are stored in a structure equal to ModelOut.validation. Depending on the chosen method some of the parameters (such as Validation.xfactors{1:n} or Validation.core) are concatenated along an additional mode.

E.g. Leave One Out Validation of a PARAFAC model for a 3-way array leads to Validation.xfactors{1:n} elements that are 3-way arrays.

In particular, in the Leave One Out validation, for some of the parameters a further slab along this additional mode is added, holding the predictions for the left out samples/batches.
This means that Validation.xfactors{1} for a PARAFAC model with F factors, validated via 'loo' on an I x J x K array, has dimensions I x F x (I + 1); i.e. I segments with only one sample excluded plus one slab with the predictions.  Validation.xfactors{2} and {3} have sizes of respectively J x F x I and K x F x I

The Explained Variance and the RMSE in validation are not computed when BootStrap is applied.


 Called by:
Model_nPLS1\Calculate, Model_PARAFAC\Calculate

Subroutines:
Internal: Validate_Decomposition, Validate_Regression
External: definemodelout, facperm, preprocess, recfac

Author:
Giorgio Tomasi
Royal Agricultural and Veterinary University
MLI, LMT, Chemometrics group
Rolighedsvej 30
DK-1958 Frederiksberg C
Danmark

Last modified: 02-Nov-2002 18:59:46

Contact: Giorgio Tomasi, gt@kvl.dk

References

Validate_Decomposition

function Validation_Results = Validate_Decomposition(FitFun,PredFun,X,Factors,Method,PreProcess,Options,varargin)

Description:
Validates a model which decomposes the X in absence of a Y of predicted variables.

Inputs:
The same as ValidateModel

Outputs:
The same as ValidateModel. Only the fields referring to the X array are filled in.

Subroutines:
Internal:    CleanX
External:    DefineModelOut, MergeResults, NProcess (n-way toolbox)

Author:
Giorgio Tomasi
Royal Agricultural and Veterinary University
MLI, LMT, Chemometrics group
Rolighedsvej 30
DK-1958 Frederiksberg C
Danmark

Last modified: 05-gen-03

Contact: Giorgio Tomasi, gt@kvl.dk

References

Validate_Regression
function Validation_Results = Validate_Regression(FitFun,PredFun,X,Factors,Method,PreProcess,Options,varargin) 

Description:
Validates a regression model having X as the predictor matrix and Y as predicted variables.

Inputs:
The same as ValidateModel

Outputs:
The same as ValidateModel.

Subroutines:
Internal:    CleanX
External:    DefineModelOut, MergeResults, NProcess (n-way toolbox)

Author:
Giorgio Tomasi
Royal Agricultural and Veterinary University
MLI, LMT, Chemometrics group
Rolighedsvej 30
DK-1958 Frederiksberg C
Danmark

Last modified: 05-gen-03

Contact: Giorgio Tomasi, gt@kvl.dk

References

CleanX
 

function [Flag,IndX1,IndX2,...IndXn,...,IndY1,IndY2,...IndYm] = CleanX(X,Y);

Description:
Extracts the indexes on X corresponding to slabs that are not constituted only by missing values.
In case a Y is provided the horizontal slabs (i.e. the samples/batches) that have only missing values either in X or in Y are absent from these indexes.  Flag is 1 in case any of the slabs had to be removed. Otherwise it is 0.

Inputs:
X: n-way array (where n >= 2)
Y: m-way array (where m >= 2)
The first mode of X and Y must have the same dimensions in the first mode.

Outputs:
Flag: is one in presence of at least one slab in either X or Y containing only missing values
 IndX1,IndX2,...IndXn: are the indexes in the n-dimensions of X.
 IndY1,IndY2,...IndYm: are the indexes in the m-dimensions of Y.

Subroutines:
Internal:    none
External:    nshape (n-way toolbox)

Author:
Giorgio Tomasi
Royal Agricultural and Veterinary University
MLI, LMT, Chemometrics group
Rolighedsvej 30
DK-1958 Frederiksberg C
Danmark

Last modified: 05-gen-03

Contact: Giorgio Tomasi, gt@kvl.dk

References

MergeResults


function Validation_Results_Out = MergeResults(Validation_Results_In,Model_Parameters,NRep,u,v,i,N_HorSlabs,Options,PreProcess,Centre_Par_Y,Scale_Par_Y,Parameters_Reference)

Description:
Copy the Results in the appropriate mode in Validation_Results. This function is called only in case several replicates of the same model are produced by the validation procedure (that is: Leave One Out and Bootstrap).

Inputs:
Validation_Results_In:  ModelOut.validation structure, empty or with the results of the previous replicate in Validation.

Model_Parameters: results for the new replicate in Validation

NRep: total number of replicates; for 'loo' it is equal to I, while for '*boo' is fixed by the user.

u: cell vector of doubles; u{i} holds the indexes for Validation_Results_In, in the sample/batches mode, for the i-th replicate (they are general and applicable to 'xfactors', 'yfactors', etc)

v: cell vector of doubles; v{i} holds the indexes for Model_Parameters, in the sample/batches mode, for the i-th replicate (they are general and applicable to 'xfactors', 'yfactors', etc). In other words it contains the indexes of the horizontal slabs (the number of dimensions irrelevant) of a certain field of Model_Parameters  to be copied  to the u{i} horizontal slabs of the corresponding field of Validation_Results_In.


Outputs:
The same as ValidateModel.

Subroutines:
Internal:    CleanX
External:    RecFac, FacPerm, NProcess (n-way toolbox)

Author:
Giorgio Tomasi
Royal Agricultural and Veterinary University
MLI, LMT, Chemometrics group
Rolighedsvej 30
DK-1958 Frederiksberg C
Danmark

Last modified: 05-gen-03

Contact: Giorgio Tomasi, gt@kvl.dk

References