Transferencia de NIR System II a DS Systems

Soy meal Protein bands

I merged in this case the Excel Plots with the Win ISI spectra to get this picture. We can see in black the raw  and second derivative spectra of a 47% protein soy meal sample. In green the are 2ª derivative spectra of soya hulls and soya beans. All this spectra can help us to understand better this products when developing calibrations.

 Click on the picture to enlarge it.

Removing redundant samples

Sometimes we accumulate huge amounts of spectra in a CAL file, and most of them is redundant and maybe that not helps to improve the calibration. One thing important when we have a lot of spectra is that we have more when we can select and to fill our hypercube in the best way. The option "Select Samples from a Spectra File" in Win ISI help us to select samples which fill the hypercube  with the samples at similar distances between them, so all of them are important in the calculation in the centroid, and there are not groups which makes that the centroid is more closer to them. The number of PCs is also reduced and as we can see comparing these two plots (left: without redundant samples, right: with redundant samples) and maybe we increase the variance explained for the PC terms respect to some constituents in the case we want to develop PC regression. So no discard to use this option to improve the calibrations.

 

Comtrade Webpage

These days I am doing an Online Course of  Python and we are practicing extracting data,  to work with, from a very interesting web page that I want to share with the readers of this blog.
http://comtrade.un.org/data/
From this web page you can extract very interesting information about your country, about from or to where, we import  or exports commodities.
For example in this case I want to see from which countries we imported computers and to which countries we exported computers in 2015:

You can preview or download the csv file and to work with it  R or Python for grouping and practice statistics.
The first row is the result for all the countries in the World. So we have to look to the next rows to see from which countries we import or to which countries we export computers:
In the case of imports:

and in the case of exports:

Hope you find it interesting, and share things that have surprise you from your Country.

Looking to the plots (Validation for W in flour)

When we check a model with a validation set, what we normally look is to the standard error of prediction (RMSEP) and the RSQ. 

After we check if we have a bias and to the standard error of prediction corrected by bias. Maybe we are happy with the results if we see that are similar to the calibration statistics or maybe not so happy and think that the calibration does not work. 

It is important to check if we have outliers and to remove those samples that could increase the errors, but if they are not clear outliers, they must stay. What we can do is if the error is bigger is some parts of the XY plot.

Maybe the calibration is not so fine for certain range, but it works fine for other range. This way we can take some conclusion about where to include more samples to improve the calibration. 

Histograms will help you with this as well, but also the X-Y plot of reference versus predicted. In the case of flour there are different types according to the W parameter.

In the X-Y plot I can see with an independent validation set that the calibration is performing well for flour between 200 and 270 of W, with a RMSEP of 9. This tells me that the calibration is working fine for this type of flour used normally for pizza products.

The calibration does not work for soft flour (low W) or hard flour (high W). You have to decide how to improve it or to separate the flour product into 3 products in order to improve the predictions.

Look always to the plots and try to find conclusions about the data.