Russian Chemometric Society (Excel Tutorials - 001)

I´m practicing the Excel tutorials from the Russian Chemometric Society, they are really interesting.

Visit their Web page from time to time, it is really interesting.

Estoy practicando los Tutoriales de Excel de la "Russian Chemometric Society" , son realmente interesantes.Podéis sacar mucha información de su página Web.

Should I adjust the slope?

I add a new video “Should I adjust the slope”, where a new part of script is added to the monitor function.  I don´t recommend adjusting the slope, but there are circumstances where it is necessary:

Suppose you have an equation, but not the calibration set from which it was developed, in that case is necessary to adjust the slope and the intercept in some cases, in others just the bias (See the video “Should I adjust the Bias”). We have to take into account that the calibration was developed with the samples scanned in another instrument and with (probably) another reference method, so adjust the slope and intercept is a way to standardize one instrument to the other or one reference method to the other.

To adjust the Bias and the Slope/Intercept, we should have a representative set with samples including all the variance represented in the calibration set. If not any of these adjustments should be done.

If we run the monitor and observe a slope problem in the slope/intercept or bias we have to check the instrument for problems and the reference method as well.

We have to see all the statistics together in order to find an answer.

To see if an adjustment of the slope is necessary, we have to see (with a T-test) if there are significant references that the calculated slope is different from 1.

We have to calculate the residual standard deviation:

, but the predicted values (y hat) are changed by the predicted value corrected with the slope and intercept.

ISO 12099 describes how to get a “t observed” from this RSD, and we compare it with the value obtained from tables for a probability of α =0.05.

I have added these calculations to R script.

These monitor function can be configures the way we prefer. You can take ideas from the ISO12099, or from an interesting article you can download from the Web:
See post: "Some considerations about NIR Spectroscopy"

Improving Monitor statistics with the "UECLs"

New Monitor Video:
Improving Monitor statistics with the "UECLs"

F-test to find UECLs

I have fixed the link to the video "Removing Y outliers from the validation set" and it´s time to see what could be the next step to the function. As we know the RMSEP is the sum of the explained (BIAS) and unexplained error (SEP).

We get also the SEP, so we know the unexplained error, and we can compare bouth in order to see if the Bias is significant, so the adjustment of the Bias is necessary but only temporally until we find the source (if possible) of this explained error. If it is a new source of variance which should be included in the calibration we will develop the model again.

Now I´m working in another function which tell us if the unexplained error (SEP) is out of limits, or what is called the unexplained error confidence limits (UECLs).

This is a  F-test (ratio of two variances): Validation Set and Calibration Set.

In the Calibration Set we have to decide if to choose the SEC (too optimistic) or the SECV (CV error) which is more realistic.

Of course degrees of freedom will be different for bough:

Nv-1 for the validation set

Nc-P-1 for the calibration set

(N =number of samples, P=number of terms).

I will come soon with this, finding if the function is already developed in this case in R (please add your comments), or I have to develop it.

The function will say if a certain SEP value, with a certain number of samples can be accepted if the value exceeds a certain percentage of the SEC or SECV.

Removing "Y" outliers from the "Validation Set"

This is a new video about how to monitor and interpret statistics and graphics for validation.
Removing "Y" outliers from Validation Set

Previous videos about the Monitor function  for validation:
Should I adjust the Bias?
Monitor: Adding "RER" and "RPD" statistics

Other posts with part of the script:
Monitoring some statistics with "R"
Practicing Script with “ R”: Monitor
Improving script_001: “Monitor”
Improving script_002: “Monitor”
Monitor: Removing zero values from the data set.