Spatial Mapping of Moisture Content in Tomato Fruits using
Hyperspectral Imaging and Artificial Neural Networks
Kaveh Mollazade1*, Mahmoud Omid1, Fardin Akhlaghian Tab2, Sayed Saeid
Mohtasebi1, Manuela Zude3
1
Department of Agricultural Machinery Engineering, Faculty of Agricultural Engineering and
Technology, University of Tehran, P.O. Box 4111, Karaj 31587-77871, Iran
2
Department of Computer Engineering and Information Technology, Faculty of Engineering,
University of Kurdistan, Sanandaj, Iran
3
Leibniz Institute for Agricultural Engineering Potsdam-Bornim (ATB), Max-Eyth-Allee 100,
14469 Potsdam, Germany
*Corresponding author. E-mail: [email protected]
Abstract
The current study evaluates the potential of hyperspectral imaging combined with artificial
neural networks to predict the moisture content in tomato fruit and to obtain spatial
distribution maps. A total of 192 tomato samples, Solanum lycopersicum 'Pannovy', were
harvested in the third and fourths stage of ripening according to the OECD color chart.
Samples were immediately transmitted to the laboratory and stored at 15°C and 92% rH.
Measurements were carried out after 1, 8, 15, 22, and 30 days of storage. After acquisition of
hyperspectral images of 40 tomatoes on each date, moisture content of samples was
calculated after oven drying (105°C). The electromagnetic spectrum between 400 to 1000
nm was recorded at 495 passbands by a hyperspectral imaging system. After applying
preprocessing operations and removal of data in saturation, valid data sets were collected
manually from the region of interest. Spectral dimensionality was reduced by selecting the
wavelengths in which high correlation exists between the spectral data and fruit moisture
content. A multilayer percepteron neural network has been trained with training dataset to
predict the moisture content of samples. To prepare the spatial prediction maps, tomato
samples were separated from the background. After that, 2D images were unfolded to
vectors and the intensity values of each pixel in the selected wavelengths were fed into the
fully trained neural network. The output matrix, containing the predicted values of moisture
content in every pixel, was folded back to form a 2D matrix with the same spatial dimension
of the hyperspectral images. Finally, the spatial distribution of moisture content was
displayed as a color map, where colors represent different values of predicted attribute.
Results proof the feasibility of the method for characterizing the spatial distribution of an
attribute in horticultural produce.
Key words: Food technology, nondestructive test, quality evaluation, spectral imaging.
1. Introduction
Development of analytical techniques for monitoring the quality of tomato fruit is important,
since it is one of the most widely produced fruits in the world with production about 146 MT in
2010 (FAO, 2010). During the last decades, several non-destructive tests have been
developed for monitoring internal and external qualitative factors of agro-food materials.
Regarding the tomato fruit, researchers have investigated several non-invasive methods
based on engineering properties of fruit to evaluate its qualitative parameters. Raman
spectroscopy (Mohammad-Nikbakht, 2009), near infrared spectroscopy (Floers et al., 2009;
Xie et al., 2008; Pedro and Ferreira, 2007), bio-speckle imaging (Romero et al., 2009), and
multispectral imaging (Hahn, 2002) are a sample of optical techniques used to monitor
variation of soluble solids content (SSC), acidity, firmness, and hue angle in the tomato fruit.
Sorting and grading of fruits is one of the most important applications of non-destructive tests
in agriculture. Conventional sorting/grading machines consider fruit attributes such as size,
shape, and colour, while monitoring the internal quality characteristics like moisture content
(MC), SSC, acidity, nutritional content, etc. is important in advanced quality evaluation. Most
of optical techniques provide information just in a limited section of fruit. To obtain spatially
resolved information of the sample, it is necessary to develop a spatial distribution map for
each quality attribute of interest. Hyperspectral imaging, as a new advanced technique,
provides both spectra and spatial information of samples by integrating the principles of
spectroscopic and imaging technologies in a system. The hyperspectral image is a 3D cube,
normally called hypercube, in which the spectral information is provided for each pixel in the
image. By analysing the spectral information in each pixel and finding the predicted value for
each quality factor of fruit, spatial distribution maps can be found for regions of interest.
Kamruzzaman et al. (2011) developed an image processing based algorithm for
discrimination of lamb muscles based on processing the hyperspectral images obtained in
the spectral range of 900-1700 nm. The multivariate analysis showed the overall accuracy of
100% for discriminating three types of lamb muscles. In another research, ElMasry et al.
(2011) reported the potential use of hyperspectral imaging to predict water holding capacity
in fresh beef. They provided some spatial distribution maps of drip loss in beef meat resulting
from a calibrated partial least square regression (PLSR) model. These maps help to know
the influence of some post-slaughtering processing regimes on meat.
The objectives of the current study were to:
1. Determine important wavelengths to monitor MC changes in tomato fruit.
2. Establish a model for predicting MC in tomato fruit based on artificial neural networks
(ANNs) and fruit spectral signatures in the wavelength range of 400-1000 nm.; and
3. Make spatial distribution maps for MC using image processing algorithms developed by
calibrated ANN model.
2. Methodology
2.1. Sample preparation
A total of 192 defects free 'Pannovy' tomato fruits, grown in a research greenhouse of
Humboldt University of Berlin, were harvested at two stages of maturity, the third and fourths,
according to the OECD color chart. After transportation to the laboratory and numbering, the
samples were stored at 15°C and 92% rH. The day after the storage experiment started was
defined as the beginning of experiments. From the start of the storage experiment samples
were withdrawn at regular intervals of one week during a period of 30 days. Prior to the
experiments the samples were equilibrated at laboratory temperature (21±1ºC) for at least
2h, to avoid effects of storage temperature on the measurements.
2.2. Data acquisition
The hypercubes were acquired in the spectral range of 400-1000 nm (spectral resolution of
1.215 nm) using a laboratory hyperspectral imaging system composed of an illumination unit,
a spectrograph (ImSpector V10E, Specim Co., Finland) coupled with a standard C-mount
zoom lens, a CCD camera (pco.oem uc, PCO Co., Germany), and a sample holder unit. The
spectrograph disperses the incoming line of light into the spectral and spatial matrices and
then projects them onto the CCD. The hyperspectral imaging system was controlled using a
PC with software developed in LabView 8.6 (HyBiS 1.0 datalog, ATB, Germany).
Immediately after acquisition of spectral images of 40 tomatoes on each date, MC of
samples was determined using oven drying method at 105°C.
2.3. Preprocessing of hyperspectral images
Several preprocessing operations were carried out in order to improve the quality of acquired
hypercubes. Images were binned (by 2×2 pixel windows) in spatial direction to provide
images with spatial dimension of 696×512 pixels with 495 spectral bands from 400-1000 nm.
This operation leads to improve the signal to noise ratio. After that hyperspectral images
were subjected to correction treat by white and dark references. Observations showed that
the light reflectance from different sections of fruit is not the same so that in the central
sections of each sample maximum reflectance is occurred, while the minimum values were
recorded in the marginal sections. To reduce the effect of fruit curvature on the intensity of
light reflectance, spectral information was first collected from the 40 points of each sample
with a specific pattern as shown in the Fig. 1. The spectra were smoothed by the SavitzkyGolay filter to remove random noises and then were subjected to mean normalization
operation in order to reduce the effect of fruit curvature on the spectra scaling. Finally, the
average of preprocessed 40 spectra was considered as the spectral signature of each
sample (Fig. 2).
FIGURE 1: The pattern of collecting the spectra from the 40 points in different sections of
samples, also the removed saturation area due to specular reflection is visible.
FIGURE 2: A sample of spectral signature of tomato fruits in different levels of MC after
preprocessing process
2.4. Selection of optimal wavelengths
Eliminating wavelengths having no discrimination power on the quality evaluation of agrofood materials and finding the sensitive wavelengths is a key step in processing the
hyperspectral images. This process leads to reduce the data dimensionality while most
important information is preserved in the new data space. Two factors influence on the
selection of sensitive wavelength; the behaviour of spectral responses of the fruits under
study and differences among them. Leardi (2000) reported that genetic algorithm (GA) is a
powerful tool for wavelength selection in chemometrics as it showed good results for different
kinds of dataset. Hence, in the current study GA was used for wavelength selection. The
initial parameters of GA were set as those proposed by Leardi (2000). Since GA is a feature
weighing technique, the output of algorithm was wavelengths listed in order of importance.
To find the optimum number of wavelengths, it was necessary to use a function
approximation technique, like support vector machines (SVMs), to evaluate the power of
GA’s wavelength list in prediction of MC when it is increased form the 1st to 200th. As shown
in Fig. 3 the best MC prediction occurred when first ten wavelengths are used as the input
vector to the SVM. Finally, the best wavelengths for prediction of MC in tomato fruit were
determined as 718.5, 718, 697, 660, 717.5, 697.5, 741.5, 719, 660.5, and 742 nm.
FIGURE 3: Result of MC prediction by SVM when number of wavelengths selected by GA
technique changed from 1 to 200
2.5. Development of ANN model
ANN is an information-processing system that has certain performance characteristics in the
same way as the biological nervous system. An ANN is typically composed of a large
number of parallel and distributed processing units, called neurons. The multilayer
perceptron (MLP) with the backpropagation learning strategy is a mostly used topology to
model the engineering problems. Furthermore, “trial and error” is a common way to adjust
the network structure. Using this strategy, different type of neurons ordering is considered
and finally, that structure in which the maximum correlation between desired output and
network output exist is selected. In the current study, the number of neurons in the hidden
layers was changed from 2 to 20 by the step of 2 neurons. Correlation coefficient of testing
dataset was used as a criterion to compare the performance of different topologies. Results
showed that network with topology 10-12-2-1 is the best model for prediction of MC in tomato
fruit as its correlation coefficient and mean square error were 0.773 and 0.0356, respectively.
3. Results
3.1. Segmentation the regions of interest
As shown in the Fig. 4-A, specular reflection of the illumination source at the tomato surface
leads to produce regions with high-intensity values in the hyperspectral images. These
regions act like a mirror and lead to saturation of CCD. These pixels were eliminated by
thresholding over the red (R>225) and green (G>220) channels of image. This process was
repeated for segmenting the rest of fruit from the background (R>25 and G<45). Result of
threshoulding was a binary mask (Fig. 4-B) used to segment the regions of interest (Fig. 4C).
3.2. Visualizing spatial distribution of MC
To create spatial distribution maps, the hypercube of each sample was unfolded to a matrix
in which the first column contains the spatial information of each pixel and the rest of
columns carry the spectral information. Then, the matrix was simplified by holding the ten
optimal wavelengths selected by GA (refer to Section 2.4) and removing the rest of spectral
data. In parallel, the mask image (refer to Section 3.1) of each sample was converted to a
matrix containing two columns. The first column contains spatial data and the second
contains the binary values of mask image. The second column of the mask matrix was then
multiplied to the matrix of spectral data. The new matrix, containing the spectral data in the
segmented regions of fruit, was used as the input vector to the fully trained neural network
described in the Section 2.5. The output of ANN was a vector containing the predicted value
of MC for each pixel. The spatial distribution map of MC for tomato fruit was created by
merging the output of ANN and the vector of spatial data of pixels (Fig. 4-D).
FIGURE 4: A to C. Segmentation the region of interest from the background and saturated
sections. D. Prediction maps of MC values generated from ANN model. Color bar shows the
scale of MC values. The value in the right side of prediction maps shows the average MC
values of each sample.
4. Conclusion
To predict MC in tomato fruit, a systematic approach was developed using hyperspectral
imaging in conjunction with ANN as well as GA to find the optimal wavelengths. Developed
prediction maps showed a slight change in the spatial distribution of MC for the fruits under
study. In order to have comprehensive information for quality evaluation of fruits, it is advised
to create spatial distribution maps for several qualitative characteristics (MC, acidity,
elasticity, etc.) and merge them to a specific map by considering specific weights for each
characteristic.
Acknowledgements
Part of this work was supported by the ICT-AGRI project “3D-Mosaic – Advanced Monitoring
of Tree Crops for Optimized Management – How to Cope with Variability in Soil and Plant
Properties?“ which is funded by the German Federal Ministry of Food, Agriculture and
Consumer Protection under the guidelines of the European Commission's ERA-NET scheme
under the 7th Framework Programme for Research. Supports form Iran’s national elites
foundation (INEF) is also acknowledged.
Reference list
ElMasry, G., Sun, D. W., & Allen, P. (2011). Non-destructive determination of water-holding
capacity in fresh beef by using NIR hyperspectral imaging. Food Research International, 44,
2624–2633.
FAO. (2010). FAOSTAT database. http://faostat.fao.org (accessed: April 21, 2012).
Flores, K., Sanchez, M. T., Perez-Martin, D., Guerrero, J. E., & Garrido-Varo, A. (2009).
Feasibility in NIRS instruments for predicting internal quality in intact tomato. Journal of Food
Engineering, 91, 311-318.
Hahn, F. (2002). Multi-spectral prediction of unripe tomatoes. Biosystems Engineering, 81(2),
147-155.
Kamruzzaman, M., ElMasry, G., Sun, D. W., & Allen, P. (2011). Application of NIR
hyperspectral imaging for discrimination of lamb muscles. Journal of Food Engineering, 104,
332–340.
Leardi, R. (2000). Application of genetic algorithm–PLS for feature selection in spectral data
sets. Journal of Chemometrics, 14, 643-655.
Mohammad-Nikbakht, A. (2009). Model determination for intelligent and non-destructive
prediction the qualitative properties of tomato using Raman spectroscopy. PhD dissertation.
Department of Mechanical Engineering of Agricultural Machinery. Tarbiat Modares
University. Tehran, Iran.
Pedro, A. M. K., & Ferreira, M. M. C. (2007). Simultaneously calibrating solids, sugars and
acidity of tomato products using PLS2 and NIR spectroscopy. Analytica Chimica, 595, 221227.
Romero, G. G., Martinez, C. C., Alanis, E. E., Salazar, G. A., Broglia, V. G., & Alvarez, L.
(2009). Bio-speckle activity applied to the assessment of tomato fruit ripening. Biosystems
Engineering, 103, 116-119.
Xie, L., Ying, Y., Lin, H., Zhou, Y., & Niu, X. (2008). Nondestructive determination of soluble
solids content and pH in tomato juice using NIR transmittance spectroscopy. Sensing and
Instrumentation for Food Quality and Safety, 2(2), 111-115.