Looking to the Residuals Plot.

Even if an application recommend us to make a type of adjustment, we have to look always to the plots.

In this case due to 3 samples (with prediction higher than 30) which are bellow the zero residual line, the application recommend us to adjust the slope, 

but it is clearly seen that for the rest of the samples the problem is a clear Bias problem. So we remove these samples with a prediction higher than 30 from the original data frame:

pass1<-subset(pass1$Table1,pass1$Table1[,3]<30)

and now the application will recommend the adjustment of the Bias. It is clear that we need more samples of this type in the plot to see clearly if we have a 

problem in the prediction of the protein content in this case.

The red dots are the sample values without bias adjustment and the yellow ones the samples with bias adjustments. The distribution of the yellow dots is fine 

and represents the random noise.

It is clear that more work must be done on the calibration or in the analysis of 

the reference methods in order to improve the predictions.

Validation Samples  = 85 
Reference Mean  = 21.7 
Predicted Mean  = 20.1 
RMSEP           : 2.35 (Current Error)
Bias            : 1.59 (Sugested Bias correction)
SEP             : 1.74 (Random error once Bias is corrected)

 

Monitor function 27-11-2014

Once developed a model with the math treatments we consider adequate, and using the Calibration Samples for Instrument 1 (following with the Shootout_2002 tutorial), the idea is to check if that model performs fine with Instrument 2 for exactly the same samples. A bias is expected, because even being the same model of instrument, differences in the hardware components, optics, alignments,…, apart from some others are the cause of this bias.

Some time ago I develop a function to monitor, to plot and obtain the statistics necessaries to take some decisions as if the bias or slope should be adjusted, to check for outliers of high residuals,.....

 

It is clear that we have 5 outliers of high residuals that the literature about this set considers that must be removed, so the error RMSEP will decrease.
Due to the high RMSEP of the model used for this monitor (RMSEP=4.33 using 4 terms and with all the samples), the bias must be quite high in order to be consider that it should be adjusted. This error is more than 3 times the Lab error.

So the statistics are:

monitor14(monit.tr2[,2],monit.tr2[,1],155,4,0.95,4.33)

Where 0.95 is the confidence interval and 4.33 the CV error of the model using 4 terms

N Validation Samples  = 155 
N Calibration Samples = 155 
N Calibration Terms   = 4 
------------------------------------- 
RMSEP    : 4.942 
Bias     : -2.509 
SEP      : 4.272 
UECLs    : 4.951 
***SEP is bellow BCLs (O.K)***
Corr     : 0.9811 
RSQ      : 0.9626 
Slope    : 0.9813 
Intercept: 6.122 
RER      : 19.9   Fair 
RPD      : 5.146   Good 
BCL(+/-): 0.6778 
***Bias adjustment in not necessary***
Residual Std Dev is : 4.266 
***Slope adjustment in not necessary***

We can see how the SEP (error corrected by the bias) is similar to the error of 

the model, so a bias adjustment will help to transfer the model from instrument 1

to Instrument 2.

I will remove the 5 samples and come back with the results.

Why this bias? (Process)

I had a new set of samples for validation (process system), as I said in other posts, this is an exciting moment: Will the  calibration performs well?

My first reaction is to look to the plots (Lab vs Predicted). In this case I had this surprising plot:

There are some samples marked as x which have a bias. These are 12 samples (one for batch of flour production), taken randomly during the batch production. It is clear that something happened during the process, and that produced a bias, giving 3 points more of protein during that period.

I expected to have the same problem for moisture, ..., it was not so much, but all these samples has also a bias , being the residuals (Lab - Predicted) positive for all of them.

Same for ash, again these samples are apart from the others.

It is clear that something is happening with these samples. We have to take a look to the spectra.

To be continued.....

¿Por que este Bias? - Why this Bias?

¿Porque tengo este Bias?  ¿What is the reason for this Bias?
Estamos analizando muestras en rutina, el equipo nos empieza a dar mas altos, de una manera sistemática, los resultados de alguno de los parámetros. Puede que nos demos cuenta de ello de una manera rápida o que nos demos cuenta mas tarde, después de haber contrastado algunos muestras pasadas por el NIR, por los métodos de referencia.
¿Que ha pasado?.
We realize that from certain sample or date, we start having a Bias in our results. What is happening from that moment?. We can organize our sample set by date or any other form in order to find the reason.
Organizando las muestras por fecha, nos podemos dar cuenta de que ha sido a partir de una determinada fecha y hora. ¿Que ha pasado en ese momento?. What is happening?.

Bias (temperature/temperatura)

Cualquier cambio en alguna de las variables de la propia muestra que no esté representado en la calibración (en este caso la temperatura), dará lugar a un posible Bias (error sistemático). En este caso la calibración fue realizada en la harina (directamente en el proceso) durante los meses de otoño, invierno, y parte de la primavera. Con la llegada del tiempo más caluroso se empezaron a observar desviaciones en la predicción (daba de menos). Se analizaron espectros del proceso durante varios días y se incorporaron a la base de datos junto al resto de datos de todo el año y se recalibró. Se dispuso también de un conjunto de muestras separadas (de diferentes lotes) para validar. La validación y posterior muestra en rutina mostró que la calibración ya se ajustaba a un error más normal.

Con esta operación y con otras que se vayan desarrollando aportando nueva variabilidad las calibraciones se van volviendo más robustas.

La temperatura causa variaciones en los puentes de hidrogeno de las moléculas, siendo el espectro NIR muy sensible a los mismos.

Any change in some of the variables of the simple itselft, which are not represented in the calibration (in this case the temperature), will cause probably a Bias (systematic error). In this case is a flour calibration (on line) done with samples taken in Autum, Winter and part of the Spring. With the arrival of higher temperatures a Bias in the prediction was observed (lower values). New samples were acquired for several days with this type of weather and were merged in the database for doing a new calibration. Post validation with new samples confirm that the Bias was corrected doing the calibration more robust.