Research Article
Volume 1 Issue 4 - 2015
Physical Modelling of Nikon Coolpix Camera RGB Responses for Application in Non-Destructive Leaf Chlorophyll Imaging
^{1}Department of Bioscience Engineering, University of Antwerp, Belgium
^{2}Applied Physics, Eindhoven University of Technology, Netherlands
^{3}Department of Organismic and Evolutionary Biology, Harvard University, USA
^{4}Department of Biology, University of Antwerp, Belgium
^{2}Applied Physics, Eindhoven University of Technology, Netherlands
^{3}Department of Organismic and Evolutionary Biology, Harvard University, USA
^{4}Department of Biology, University of Antwerp, Belgium
*Corresponding Author: Frank Veroustraete, Department of Bioscience Engineering Antwerpen, University of Antwerp, Belgium.
Received: March 03, 2015; Published: June 30, 2015
Citation: Frank Veroustraete., et al. “Physical Modelling of Nikon Coolpix Camera RGB Responses for Application in Non-Destructive Leaf Chlorophyll Imaging”. EC Agriculture 1.4 (2015): 223-242.
Abstract
This paper describes computer aided leaf digital image analysis based on leaf reflectance imaging. It is based on a fast, non-destructive measurement technique of leaf chlorophyll content imaging based on measurements of leaf reflectance. The validity of the method is demonstrated by a direct comparison of conventional chlorophyll extraction of both leaf chlorophyll pigments a, b with reflectance based estimates of leaf chlorophyll a, b and total chlorophyll imagery. The leaves of the species selected for this paper are characterized by heterogeneous chlorophyll distributions. The application of the software developed for image analysis at the leaf spatial level allows revealing the morphological structures which determine the spatial variation of leaf chlorophylls. The technique opens the path towards FERET type taxonomy.
Keywords: Camera physical modeling; RGB bands; leaf chlorophyll imaging; chlorophyll spatial statistics; FERET type taxonomy
Abbreviations:B: Blue spectral band; CCD: Charge Coupled Device; (Chl): Leaf chlorophyll concentration; DN: Digital Number; DMSO: Dimethyl-sulphoxide; EMS: Electromagnetic Spectrum; G: Green Spectral band; HZ: Leaf cross vein sampling pattern; K: Cornus sp; L: Tilia sp; LIBERTY: Leaf Incorporating Biochemistry Exhibiting Reflectance and Transmittance Yields; LVA: Leaf Vein Asymmetry; M: Zea mays L; MP: Mega Pixels; NDVI: Normalized Difference Vegetation Index; NIR: Near-Infrared Band; PAR: Photosynthetic Active Radiation; PROSPECT: A Model of Leaf Optical Properties Spectra; R: Red Spectral band; ρ_{l}: Leaf reflectance; REP: Red Edge Position; RGB: Red - Green - Blue spectral bands of a (commercial) true colour CCD camera; ROI: Region of Interest; RTF: Radiative Transfer Model; SDP: Species Discriminative Power; STD: Standard Deviation; VI: Vegetation Index; VT: Leaf parallel vein sampling pattern; <x>_{n}: Mean value of x with the number of samples equalling n
Introduction
Plant physiologists, agronomists as well as farmers quite often require the examination of leaf pigmentation to describe physiological processes or chlorosis patterns, spatially resolved at leaf level. Since chlorophyll is rarely spatially homogeneously distributed, quantitative as well as spatial comparisons between different leaf information layers or even statistical analysis of a single leaf information layer are required. However this type of information is quite rare in the literature, even though functional relationships between different physiological processes at leaf level may be revealed much more straightforward when the spatial distribution of chlorophyll content (Chl) is known at leaf level.
Data acquired and analytical approaches applied in plant leaf morphological descriptions are analogous to those used in the remote sensing of for example landscapes. The methods used in landscape analysis [14], [22] are well suited to analyze spatially explicit (Chl) at the leaf level. Clearly, important leaf information layers are leaf (Chl_{a}), (Chl_{b}) and (Chl_{tot}). They constitute important characteristics of plant leaves and critical input variables for modeling purposes, especially so, when in remote sensing, one models scaling from leaf to canopy level. In that case (Chl) is a very stringently required input variable [8].
Leaf colour is an indicator of (Chl), which in its turn is closely related with the nutrient status of a plant, more specifically and most importantly with nitrogen content. Within that framework, leaf colour is an important indicator of plant endogenous or exogenous factors, impacting on leaf (Chl). This is typically the case for factors like mutual shading, phenology and senescence, viral, bacterial or mould infections, species specificity of (Chl) and hydrological status [16].
A non-destructive (Chl) imaging method to quantitatively determine leaf (Chl) is therefore useful for applications related to before mentioned processes. When using leaf colour as an attribute for (Chl) quantification, the estimation method is limited to the Photosynthetic Active Radiation (PAR) part of the electromagnetic spectrum (EMS: 400-700 ηm). Different research tracks have been followed to estimate (Chl) of leaves with optical non-destructive methods [3], [12], [25], [10], [19], [23], [24]. Methods hitherto developed for leaf (Chl) quantification, are typically based on R (Red) or NIR (Near-Infrared) reflectance or absorbance measurements.
A well known method based on hyperspectral leaf reflectance signatures, is the method based on the red edge position (REP). The REP is the inflection point of a leaf reflectance hyperspectral signature, typically between 680 and 740 ηm [1], [5], [9], [17], [20]. Only a small subset of investigators did follow the research track developing a (Chl) quantification method based on the measurement of leaf reflectance in the PAR part of the EMS. Most authors developed methods based on the so-called spectral indices or vegetation indices (VI's), of which the most well-known as well as oldest one is the Normalized Difference Vegetation Index (NDVI).
An advantage of methods based on the PAR of the EMS - which is somewhat underrated - is that use of the PAR spectral region wavelengths offers the possibility to use commercially cheap and abundantly available digital RGB (Red-Green-Blue) true colour cameras. They offer the possibility to apply leaf (Chl) spatial imaging techniques for leaf level research, since these cameras include CCD arrays as well. These CCD's allow performing leaf imaging at very high spatial resolutions (10 to 20 µm). This type of imagery in its turn enables to perform spatial and statistical analysis of leaf (Chl) (spatial and frequency distributions) at leaf level.
For example [15] developed a field method to estimate spatially explicit leaf (Chl) using a RGB video camera and image processing software. Unfortunately, these authors did not calibrate the imagery acquired by the camera to enable expression of the imagery as a reflectance. They neither determined the transfer function to estimate leaf (Chl) from leaf reflectance. Hence the repeatability and especially the quantitative accuracy of their method remain uncertain. Evidently, (Chl) was not determined either.
In the work presented in this paper, the LIBERTY [7] and PROSPECT [13] leaf RTF models are applied to determine the transfer functions for the estimation of (Chl_{a}), (Chl_{b}) and (Chl_{tot}) for three plant species, e.g., Tilia sp., Cornus sp. and Zea mays L. A commercial RGB camera was used to determine calibrated leaf reflectances for the camera RGB bands. The transfer functions obtained with the LIBERTY and PROSPECT leaf RTF models were validated using destructive sampling of (Chl_{a}), (Chl_{b}) and (Chl_{tot}) for the leaves of the species cited earlier. The results of this methodological development are presented and discussed in the following chapters.
Materials and Methods
Determination of leaf reflectance
In this chapter, the approach to determine leaf reflectance (ρ_{l}) as well as the measuring protocol is described. Digital imagery of leaves for three plant species has been acquired according to a method which makes use of a consumer electronics digital camera (NIKON Coolpix) and a Spectralon^{®} panel used as a calibration reference target for reflectance measurements. A single leaf dataset consists of a digital image acquisition of the adaxial side of a leaf as well as a digital image acquisition of the Spectralon^{®} reference panel (Figure 1). Both acquisitions are made under the same illumination conditions.
In this chapter, the approach to determine leaf reflectance (ρ_{l}) as well as the measuring protocol is described. Digital imagery of leaves for three plant species has been acquired according to a method which makes use of a consumer electronics digital camera (NIKON Coolpix) and a Spectralon^{®} panel used as a calibration reference target for reflectance measurements. A single leaf dataset consists of a digital image acquisition of the adaxial side of a leaf as well as a digital image acquisition of the Spectralon^{®} reference panel (Figure 1). Both acquisitions are made under the same illumination conditions.
The camera used is a 10 MP digital camera (Nikon Coolpix 5000^{®}, Nikon Corp.). Three plant species, Tilia sp., Cornus sp. and Zea mays L. are used for leaf (Chl) determinations. A large enough number of leaves per species (6 or more) are selected for statistical purposes. Destructive measurements include the measurement of (Chl_{a}), (Chl_{b}) and (Chl_{tot}), fresh and dry weight and water content of the different leaves sampled from the species mentioned. The surface area of the individual leaves is determined with a copy paper replica of the leaves. By weighing the replica and a reference surface area of 10 × 10 cm of the same material, the leaf surface area can be determined accurately within a margin of 1% absolute error.
A raw image dataset consists of leaf digital number (DN) imagery in lossless tiff image format. Once acquired, it is transferred from the camera to a laptop equipped with IDL/ENVI^{©} 3.6 image processing software (Research Systems Inc., Boulder, Colorado). To enable the next step, e.g., the conversion of image (DN) to leaf reflectance, a raw leaf image is converted, into an R, G and B 1 byte (DN) image. In a next step, calibrated reflectance is calculated, by dividing the R, G and B one byte digital numbers (DN_{l, λ}) by a Spectralon^{®} reference panel one byte digital number (DN_{r, λ}). The spectral signature of the Spectralon^{®} reference panel between 250 and 2500 ηm is illustrated in Figure 1.
R, G and B calibrated spectral reflectances for a leaf image pixel are calculated using equation 1:
(1)
ρ_{l,λ} (-) and ρ_{r,λ} (-) are the leaf and reference target reflectance’s respectively for band λ, with λ indicating the R, G or B spectral band of the camera. (DN_{l,λ}) and (DN_{r,λ}) are the digital numbers (range: 0–255) of leaf pixel reflectance (ρ_{l,λ}) and the reflectance of the Spectralon^{®} reference panel (ρ_{ρ,λ}), respectively. The average R, G en B responses (DN_{r,λ}) of six measurements of the Spectralon^{®} reference panel, are taken as reference values to calculate the R, G and B leaf reflectance’s (ρ_{l,λ}). The Spectralon^{®} results are presented in Table 1.
R | G | B | R | G | B | |
<DNHZ>_{n=6} | <DNHZ>_{n=6} | <DNHZ>_{n=6} | <DNVT>_{n=6} | <DNVT>_{n=6} | <DNVT>n=6 | |
<DN> | 218 | 175 | 141 | 218 | 175 | 141 |
STD(DN)_{tot} | 3 | 2 | 3 | 3 | 2 | 3 |
STD(DN)_{tot} (%) | 1.3 | 1.3 | 2.2 | 1.3 | 1.3 | 2.2 |
Table 1: Summary of the Spectralon^{®} reference panel digital numbers [DN_{r, λ}] executed with six consecutive measurements of the reference panel in a cross pattern (HZ and VT). It is elicited, that the horizontal and vertical average digital numbers acquired with the Nikon camera are identical. The same is elicited for the standard deviations. This suggests that a spatially homogeneous response for the reflectance of the Spectralon → reference panel is obtained. The relative standard deviations are satisfactory low between 1 to 2.5%.
When for a leaf, the spectral DN ratio (see equation 1) is multiplied with the reference panel reflectance, the R, G and B calibrated reflectances according to equation 1, are obtained, and expressed in (%). Typically, calibrated leaf reflectance can then be studied in function of leaf (Chl) (µg.cm^{-2}). Leaf (Chl) descriptive statistics are calculated using IDL/ENVI^{©} and a descriptive statistics add-in of Microsoft Excel^{©}.
Figure 2: Overview of the leaf dataset and leaf weights; The symbol K stands for Cornus sp., the symbol L stands for Tilia sp. and the symbol M stands for Zea mays L. the symbols enclosed between <> e.g.; <K_{i}>, <L_{i}> and <M_{i}>, represent the average values of each species specific dataset.
In Figure 2 and 4, Zea mays L, leaves have been selected for their strongly increasing fresh weight and (Chl) gradients. Water content is the highest for Zea mays L. as illustrated in Figure 3. As mentioned, Zea mays L. leaves have been selected for their increasing (Chl). Selecting as such, a large enough range of (Chl) is obtained for this plant species. This is illustrated by Figure 4, which shows the (Chl) obtained by destructive sampling applying the dimethyl sulphoxide (DMSO) method [11].
Dataset description
This chapter gives a description of the leaf dataset used in the study presented in this paper. For each plant species e.g. Tilia sp., Cornus sp. and Zea mays L., basic variables are determined e.g., fresh weight, dry weight and water and (Chl). The DMSO (Chl) extraction method was used [11], as recommended by [2] and [18].
This chapter gives a description of the leaf dataset used in the study presented in this paper. For each plant species e.g. Tilia sp., Cornus sp. and Zea mays L., basic variables are determined e.g., fresh weight, dry weight and water and (Chl). The DMSO (Chl) extraction method was used [11], as recommended by [2] and [18].
Figure 3: Water contents of the leaf dataset. Zea Mays L. has the highest leaf water content. <K_{i}>, <L_{i}> and <M_{i}> represent average values for each species specific dataset.
Figure 4: Overview of the leaf [Chl] dataset. For the symbols see Figure 1. <K_{i}>, <L_{i}> and <M_{i}> represent average values for each species specific dataset.
Figure 5 illustrates a series of Zea mays L. leaves where (Chl_{tot}) increases gradually from left to right. The leaves illustrated by Figure 5 have been used to establish a destructive (Chl) gradient dataset for Zea mays L. as illustrated in Figure 4 (M_{i}).
Figure 5: Photographs of Zea mays L. leaves ordered according to increasing [Chl_{tot}] (increasing from left to right).
A conspicuous feature of the Zea mays L. dataset is that with increasing (Chl_{a}), (Chl_{a})/(Chl_{b}) increases significantly as well, as illustrated in Figure 6.
Figure 6: Increase in the [Chl_{a}]/[Chl_{b}] ratio with increasing [Chl_{a}], destructively measured for Zea mays L. leaves.
The ratio of (Chl_{a})/(Chl_{b}) to (Chl_{b}) in the chloroplast is normally 3:1. It is known that (Chl_{a})/(Chl_{b}) is higher under intense light conditions than under low irradiance conditions. More (Chl_{b}) relative to (Chl_{a}) is observed in shade leaves. The advantage of a higher (Chl_{b}) is that it absorbs light at different wavelengths than (Chl_{a}) and hence, extends the spectral range of light which can be absorbed for use by the light reactions of photosynthesis. In the case shown in Figure 6, Zea mays L. leaves have been selected and sampled from the top to the bottom of the about 2.5m high Zea mays L plants. For this dataset the row distances of the Zea mays L. plants is about 50 cm. This results in a gradient of increased shading towards the bottom of the maize canopy. In our case the maize plant rows were oriented parallel with an east - west axis. We suggest that this explains the results in Figure 6 pretty well.
A landmark in this paper is that significant relationships have been established, calibrated and validated for leaf reflectance in the R band (ρ_{R}) of a digital RGB camera and destructively determined (Chl_{a}), (Chl_{b}) and (Chltot). This relationship or transfer function is valid for the three plant species cited earlier and hence seems to be generic. The relationships remain significant even when the leaf (Chl) datasets of the three cited plant species are pooled. To enable the establishment of a transfer function with a maximal accuracy in the R band, a part of the leaf as large and homogeneous as possible is defined as the leaf ROI (Region of Interest) using image processing software (IDL/ENVI^{®}). For this leaf ROI (ROI_{l}), the relationship between leaf adaxial reflectance (ρ_{a}) and destructively measured (Chl) has been investigated. Transfer functions were optimized applying leaf RTF models. Destructively measured datasets are used as input in the RTF's to establish the transfer functions for quantitative imaging of (Chl_{a}), (Chl_{b}) and (Chltot) and the determination of spatial (Chl) frequency distributions for leaves of Tilia sp., Cornus sp. and Zea mays L.
Till now, (Chl) reflectometry with plant leaves has seldom been applied for chlorophyll quantitative imaging. In the next chapters the leaf (Chl) imaging technique is described, results presented and discussed as well as conclusions drawn.
Results and Discussion
Leaf reflectance sampling results
When sampling the reflectance of leaf adaxial surfaces the measuring protocol with Spectralon^{®} as described earlier is applied. Figure 7, illustrates ρ_{l} for the R, G and B bands for the HZ and VT sampling patterns (Cornus sp. leaves). The graphs are the mean of 5 leaf measurements.
When sampling the reflectance of leaf adaxial surfaces the measuring protocol with Spectralon^{®} as described earlier is applied. Figure 7, illustrates ρ_{l} for the R, G and B bands for the HZ and VT sampling patterns (Cornus sp. leaves). The graphs are the mean of 5 leaf measurements.
Figure 7: Leaf reflectance (expressed as a fraction) in the horizontal <ρ_{HZ}> and vertical <ρ_{VT}> directions for an average of 5 Cornus sp. leaves. The reflectance in the G band is the highest while that for the B band is the lowest. Moreover the ρ_{HZ} vs. ρ_{VT} measurements have a different magnitude. The vertical measurements which are taken parallel with the leaf veins are higher than the horizontal measurements, which cut across the leaf veins. Since the veins have a lower [Chl] than the inter-vein tissues, the cross-vein-cut sampling <ρ_{VT}> leads to higher reflectance values in all bands relative to the parallel vein sampling configuration <ρ_{HZ}>.
Cornus sp. | <ρ_{R,HZ}> | <ρ_{G,HZ}> | <ρ_{B,HZ}> | <ρ_{R,VT}> | <ρ_{G,VT}> | <ρ_{B,VT}> |
<ρ> | 0.401 | 0.487 | 0.238 | 0.475 | 0.531 | 0.369 |
STD_{<ρ>} | 0.10 | 0.09 | 0.15 | 0.09 | 0.08 | 0.17 |
STD_{<ρ>} [%] | 24 | 18 | 63 | 19 | 14 | 46 |
Tilia sp | ||||||
<ρ> | 0.464 | 0.526 | 0.286 | 0.484 | 0.532 | 0.344 |
STD_{<ρ>} | 0.08 | 0.08 | 0.17 | 0.29 | 0.06 | 0.15 |
STD_{<ρ>} [%] | 17 | 15 | 61 | 11 | 11 | 44 |
Zea mays L. | ||||||
<ρ> | 0.449 | 0.512 | 0.267 | 0.505 | 0.548 | 0.383 |
STD_{<ρ>} | 0.12 | 0.10 | 0.13 | 0.10 | 0.08 | 0.17 |
STD_{<ρ>} [%] | 27 | 19 | 48 | 19 | 15 | 43 |
Table 2: Average reflectance for Cornus sp., Tilia sp. and Zea mays L. with a HZ <ρ_{λ,HZ}> and a VT<ρ_{λ,ςT}> sampling pattern. For the three species and for both sampling patterns, the highest reflectance is obtained for the G band, then the R band and finally the B band. Again (as observed in Figure 7) the <ρ_{λ,HZ}> vs. <ρ_{λ,ςT}> measurements have a different magnitude. The vertical measurements (taken parallel with the leaf veins) are higher than the horizontal measurements, which cut through the veins. Since the veins have a lower [Chl] than the inter-vein tissues, the cross-vein-cut sampling <ρ_{λ,ςT}> leads to higher reflectance values in all bands relative to the parallel vein sampling configuration <ρ_{λ,HZ}>. This observation is valid for all species.
Table 2 summarizes the average reflectances for Cornus sp., Tilia sp. and Zea mays L. with a HZ <ρ_{λ,HZ}> and a VT<ρ_{λ,VT}> sampling pattern for leaf reflectance. For the three species and for both sampling patterns, the highest reflectance is obtained for the G band, then the R band and finally the B band. Again (as observed in Figure 7) the <ρ_{λ,HZ}> vs. <ρ_{λ,VT}> measurements have a different magnitude. The <ρ_{λ,VT}> measurements (taken parallel with the leaf veins) are higher than the <ρ_{λ,HZ}> measurements, which cut through the veins. Since the veins have a lower (Chl) than the inter-vein tissues, the cross-vein-cut sampling <ρ_{λ,VT}> leads to higher reflectance values in all bands relative to the parallel vein sampling configuration <ρ_{λ,HZ}>. This observation is valid for all three species. While the standard deviations of the R and G bands are quite comparable in magnitude, those of the B band are significantly higher than those of the R and G bands. Again this observation is valid for the three species. This observation might be linked with a higher photon scattering of blue light in leaves as opposed to R and G light. This could mean that STD is linked with scattering and/or absorption of photons in the vicinity of the chlorophyll a and b chromophores, which is highest for the B band, followed by the R band and lowest for the G band. This is true for all species tested.
Leaf vein asymmetry
Table 3, summarizes the results obtained for spectral Leaf Vein Asymmetry (LVA_{λ}) for the three plant species. LVA_{λ} per spectral band is defined by equation 2.
(2)
Table 3, summarizes the results obtained for spectral Leaf Vein Asymmetry (LVA_{λ}) for the three plant species. LVA_{λ} per spectral band is defined by equation 2.
(2)
With:
LVA_{λ}: Leaf Vein Asymmetry for spectral band λ (-).
ρ_{λ},VT: Parallel vein (or VT) spectral reflectance (-).
ρ_{λ},HZ: Cross vein (or HZ) spectral reflectance (-).
LVA_{λ}: Leaf Vein Asymmetry for spectral band λ (-).
ρ_{λ},VT: Parallel vein (or VT) spectral reflectance (-).
ρ_{λ},HZ: Cross vein (or HZ) spectral reflectance (-).
Table 3 summarizes the results obtained with the LVA (-) for the three plant species and for the R, G and B bands of the camera.
LVA_{R} | LVA_{G} | LVA_{B} | |
Cornus sp. | 8.4 | 4.3 | 21.6 |
Tilia sp | 2.1 | 0.6 | 9.2 |
Zea mays L. | 5.9 | 3.4 | 17.8 |
Table 3: LVA for the R, G and B bands and for Cornus sp., Tilia sp. and Zea mays L.
LVA per band is maximal for the B band decreasing over the R band to a minimal value for the G band. Between species the LVA is quite different as well. Cornus sp. has the highest LVA followed by Zea Mays L., while the lowest LVA is observed for Tilia sp. This observation is consistently observed for all spectral bands. It seems that the LVA is strongly species as well as wavelength dependent. The wavelength dependency of the LVA seems to be related to tissue (Chl), which is understandable, since (Chl) is much lower in the vein than in the inter-vein tissues.
Relationship between leaf reflectance and (Chl)
Figure 8 illustrates the relationship between leaf spectral reflectance and destructively measured (Chl_{a}) and (Chl_{b}) concentrations. In a first representation, results of spectral reflectance in function of leaf (Chl_{tot}) are shown for Zea mays L. for respectively the R, G and B bands. Quite conspicuously, only the R band shows a change in function of leaf (Chl_{tot}). The reflectance in the B and G bands is independent of (Chl_{tot}), hence Zea mays L. leaf spectral reflectance shows a dependency of (Chl_{tot}) in the R band only.
Figure 8 illustrates the relationship between leaf spectral reflectance and destructively measured (Chl_{a}) and (Chl_{b}) concentrations. In a first representation, results of spectral reflectance in function of leaf (Chl_{tot}) are shown for Zea mays L. for respectively the R, G and B bands. Quite conspicuously, only the R band shows a change in function of leaf (Chl_{tot}). The reflectance in the B and G bands is independent of (Chl_{tot}), hence Zea mays L. leaf spectral reflectance shows a dependency of (Chl_{tot}) in the R band only.
Figure 8: Relationship between leaf spectral reflectance in the R (red dots), G (green dots) and B (blue dots) channels (ρ_{R}, ρ_{G}, ρ_{B}) and destructively measured total [Chl_{tot}] for Zea mays L.
Tiliaas well as Cornus sp. elicit a relatively small range in (Chl_{tot}) compared to Zea mays L. This results in an incomplete representation of the leaf spectral reflectance (ρ_{l,λ}) in function of leaf total (Chl_{tot}) for both species. This is illustrated in Figures 9 and 10.
Figure 9: Relationship between leaf spectral reflectance in the R (red dots), G (green dots) and B (blue dots) channels (ρ_{r,t}, ρ_{g,t}, ρ_{b,t}) and destructively measured total [Chl] for Tilia sp.
Figure 10: Relationship between leaf spectral reflectance in the R (red dots), G (green dots) and B (blue dots) channels (ρ_{r,c}, ρ_{g,c}, ρ_{b,c}) and destructively measured total [Chl] for Cornus sp.
The limited (Chl_{tot}) range explains the lack of an increase in reflectance for the R band with low (Chl_{tot}). Hence, both species have been left out in the characterization of the (Chl_{tot}) versus leaf spectral reflectance study except when species (Chl_{tot}) have been pooled.
Pooled species relationships between R band leaf reflectance and (Chl)
Figure 11 represents the pooled relationship for all species, between the R band and destructively measured (Chl_{a}), (Chl_{b}) and (Chl_{tot}). In contrast with the approach where leaf reflectance is plotted for each species separately, it is elicited that when species results are pooled, useful multispecies transfer functions are obtained between R reflectance and (Chl_{a}), (Chl_{b}) and (Chl_{tot}). Especially the relationship between leaf R reflectance and (Chl_{b}) is strong. In contrast, species reflectance results pooled for the G and B bands, do not elicit useful relationships to determine (Chl_{a}), (Chl_{b}) and (Chl_{tot}) (results not shown). In this chapter (Chl) contents are ranked according to a logarithmic scale (see Figure 11), which leads to more useful linear transfer functions.
Figure 11 represents the pooled relationship for all species, between the R band and destructively measured (Chl_{a}), (Chl_{b}) and (Chl_{tot}). In contrast with the approach where leaf reflectance is plotted for each species separately, it is elicited that when species results are pooled, useful multispecies transfer functions are obtained between R reflectance and (Chl_{a}), (Chl_{b}) and (Chl_{tot}). Especially the relationship between leaf R reflectance and (Chl_{b}) is strong. In contrast, species reflectance results pooled for the G and B bands, do not elicit useful relationships to determine (Chl_{a}), (Chl_{b}) and (Chl_{tot}) (results not shown). In this chapter (Chl) contents are ranked according to a logarithmic scale (see Figure 11), which leads to more useful linear transfer functions.
Figure 11: Relationship between pooled species leaf adaxial reflectance for the R band and destructively measured [Chl_{a}], [Chl_{b}] and [Chl_{tot}] concentrations.
The transfer functions in Figure 11, relate ρR for all species with destructively measured leaf (Chla), (Chlb) and (Chltot). The regression functions are represented by equations (3), (4) and (5):
ρ_{r} = – 0.135ln(Chl_{tot}) + 0.6043 (3)
ρ_{r} = – 0.105ln(Chl_{a}) + 0.5276 (4)
ρ_{r} = – 0.101ln(Chl_{b}) + 0.5297 (5)
ρ_{r} = – 0.135ln(Chl_{tot}) + 0.6043 (3)
ρ_{r} = – 0.105ln(Chl_{a}) + 0.5276 (4)
ρ_{r} = – 0.101ln(Chl_{b}) + 0.5297 (5)
With:
ρ_{r}: Red reflectance (-)
(Chl_{a}): Chlorophyll a content (µg/m³)
(Chl_{b}): Chlorophyll b content (µg/m³)
(Chl_{tot}): Chlorophyll total content (µg/m³)
ρ_{r}: Red reflectance (-)
(Chl_{a}): Chlorophyll a content (µg/m³)
(Chl_{b}): Chlorophyll b content (µg/m³)
(Chl_{tot}): Chlorophyll total content (µg/m³)
Spectral Indices
In classical remote sensing, spectral indices are frequently applied. The most well known index is the NDVI, which has been applied in literally hundreds of methods and applications. Typical for the NDVI is its dependence on R reflectance (besides NIR reflectance), and hence – at leaf level – its relationship with (Chl). When using a RGB camera, evidently one cannot use the NDVI since the NIR band is not present for this camera type. Nevertheless linear combinations of the R, G and B reflectance’s can be tested for their usefulness in (Chl) estimation.
In classical remote sensing, spectral indices are frequently applied. The most well known index is the NDVI, which has been applied in literally hundreds of methods and applications. Typical for the NDVI is its dependence on R reflectance (besides NIR reflectance), and hence – at leaf level – its relationship with (Chl). When using a RGB camera, evidently one cannot use the NDVI since the NIR band is not present for this camera type. Nevertheless linear combinations of the R, G and B reflectance’s can be tested for their usefulness in (Chl) estimation.
Of the many combinations possible between the R, G and B channels of a digital camera the ρ_{R}/ρ_{G} ratio gives the most optimal statistical results (results not shown). The relationship is illustrated in Figure 10. The resulting transfer function is represented by equation (6):
ρ_{r}/ρ_{g} = – 0.153ln(Chl_{tot}) + 0.6411 (6)
ρ_{r}/ρ_{g} = – 0.153ln(Chl_{tot}) + 0.6411 (6)
With:
ρ_{r}: Red reflectance (-)
ρ_{g}: Green reflectance (-)
(Chl_{tot}): Total chlorophyll content (µg/m³)
ρ_{r}: Red reflectance (-)
ρ_{g}: Green reflectance (-)
(Chl_{tot}): Total chlorophyll content (µg/m³)
The fit between ρ_{r}/ρ_{g} and (Chl_{tot}) according to equation (6) is however less optimal with a much lower regression coefficient than that between redρr and (Chl_{tot}) (compare Figures 11 and 12).
Figure 12: Relationship between the ρ_{r}/ρ_{g} ratio and destructively measured [Chl_{tot}] for all species pooled.
Transfer function determination using radiative transfer modelling
To corroborate the findings described earlier, it is required to simulate the transfer function represented by equation (3) using leaf radiative transfer (RTF) models. Recently, several leaf RTF models have been published of which some take leaf dorsi-ventral asymmetry into account. In this work we opted to use the LIBERTY and PROSPECT leaf radiative transfer models developed by respectively (7) and (13) since our reflectance measurements are limited to the adaxial leaf side only.
To corroborate the findings described earlier, it is required to simulate the transfer function represented by equation (3) using leaf radiative transfer (RTF) models. Recently, several leaf RTF models have been published of which some take leaf dorsi-ventral asymmetry into account. In this work we opted to use the LIBERTY and PROSPECT leaf radiative transfer models developed by respectively (7) and (13) since our reflectance measurements are limited to the adaxial leaf side only.
Figure 13: Relationship between ρ_{r}, ρ_{g} and ρ_{b} and destructively measured [Chl_{tot}] for all species pooled, compared with two transfer functions simulated with the LIBERTY and PROSPECT leaf radiative transfer models. P in the legends represents the PROSPECT model and L represents the LIBERTY model. The coloured lines are plots based on RTF simulations. The black line is the regression line based on destructive measurements.
Figure 13 summarizes the results of the comparison between ρ_{R}, ρ_{G} and ρ_{B} and destructively measured (Chl_{tot}) for all species pooled and for the LIBERTY and PROSPECT models. Evidently the parameterization of the transfer function is crucial to obtain a good correspondence with the transfer function obtained using destructive (Chl_{tot}) data (Figure 9, equation 1). The input parameters used in the LIBERTY and PROSPECT model simulations are given in Tables 4 and 5 respectively.
Average leaf cell diameter [µm] | 40 |
Intercellular air space determinant | 0.1 |
leaf thickness [mm] | 2 |
Linear baseline absorption | 0.0005 |
Albino leaf visible absorption | 2 |
Leaf [Chl] content [mg/m²] | variable |
Leaf water content [g/m²] | 200 |
Lignin Cellulose [g/m²] | 40 |
Nitrogen content [g/m²] | 1 |
Table 4: Variable input dataset to simulate the transfer function in Figure 13, applying the LIBERTY model.
Number of cell layers [n] | 5 |
Leaf [Chltot] [µg/cm²] | variable |
Leaf water thickness [cm] | 0.015 |
Dry Matter [µg/cm²] | 0.01 |
Table 5: Variable dataset input to simulate the transfer function in Figure 13, applying the PROSPECT model.
The RTF transfer functions coincide quite well for all spectral bands. The RTF transfer function with the highest coefficient of determination for all bands originates from the LIBERTY simulations. Once more, the regression line for destructive (Chl_{tot}) measurements and the R band reflectance has the highest coefficient of determination compared to the other two spectral bands G and B.
Based on these results, a generic multispecies transfer function between calibrated leaf adaxial R reflectance (ρ_{R}) and leaf (Chl_{tot}) can be defined (see equation 7):
(Chl_{tot}) = e(– (^{ρ}r– 0.5532)/0.103) (7)
With:
ρ_{r}: Red reflectance (-)
(Chl_{tot}): Total per pixel leaf chlorophyll content (µg/m²)
ρ_{r}: Red reflectance (-)
(Chl_{tot}): Total per pixel leaf chlorophyll content (µg/m²)
Equation (7) can be put forward as the most accurate multispecies transfer function, for the purpose of per pixel quantitative imaging of leaf (Chl_{tot}). Figure 14 illustrates the transformation of a Tilia sp. leaf true colour RGB image (digital raw 3 byte (DN) RGB image format, left panel) into a (Chl_{tot}) image (right panel).
Figure 14: A Tilia sp. leaf, processed - per pixel - with IDL/ENVI^{®} from a raw 3 byte RGB image (left) into a per pixel [Chl_{tot}] image (right) using calibration function (1) and transfer function (7). [Chl_{tot}] values vary between 0.01 (light green) to 15 (μg/cm²) (dark green).
In Figure 14, (Chl_{tot}) values vary between 0.01 (µg/cm²), (light green) and 15 (µg/cm²) (dark green) in the right panel. (Chl_{tot}) variability is clearly related with leaf vein structure as can be seen from this Figure when transformed into a (Chl_{tot}) image. This variability is sheer invisible in the raw RGB image (left pane in Figure 14). Once a leaf raw image has been transformed into a leaf (Chl_{tot}) image, it is possible to perform statistical analysis and image segmentation as described in the following chapters.
Leaf Chlorophyll image analysis
Descriptive Statistics and species discriminative power (SDP)
This chapter demonstrates the basic statistics which can be calculated once a leaf raw image is converted into a quantitative (Chl_{tot}) image. Figure 15, illustrates histograms and cumulative histograms for the three species under study. Table 6 summarizes the descriptive statistics.
Descriptive Statistics and species discriminative power (SDP)
This chapter demonstrates the basic statistics which can be calculated once a leaf raw image is converted into a quantitative (Chl_{tot}) image. Figure 15, illustrates histograms and cumulative histograms for the three species under study. Table 6 summarizes the descriptive statistics.
The leaf mean value of the Tilia sp. (Chl_{tot}) distribution is 5.54 (µg/cm²), with a standard error of 0.15. The median value is 3.79 (µg/cm²). The descriptive statistics demonstrate that the (Chl_{tot}) variability (sample variance) for Tilia sp. at the leaf level is quite significant, even though - with the naked eye (Figure 14, RGB raw image) - the leaf (Chl_{tot}) appears homogeneous.
For Cornus sp. the leaf mean value (Chl_{tot}) is 3.54 (µg/cm²) with a standard error of 0.12. The sample variance is about half that of Tilia sp.
For Zea mays L. the leaf mean (Chl_{tot}) value is 2.77 (µg/cm²) with a standard error of 0.07. The sample variance is about six times less than that of Tilia sp.
Parameter | Tilia | Cornus sp | Zea mays L. 8 |
Mean [µg/cm²] | 5.54 | 3.54 | 2.77 |
Standard Error (µg/cm²) | 0.15 | 0.12 | 0.07 |
Median [µg/cm²] | 3.79 | 2.30 | 2.06 |
Mode [µg/cm²] | 4.46 | 2.19 | 1.70 |
Standard Deviation µg/cm²) | 5.57 | 4.16 | 2.37 |
Sample Variance (µg/cm²) | 31.02 | 17.31 | 5.62 |
Kurtosis [-] | 10.93 | 24.73 | 20.97 |
Skewness [-] | 2.88 | 4.26 | 3.82 |
Range [µg/cm²] | 51.03 | 42.16 | 23.48 |
Minimum [µg/cm²] | 0.48 | 0.30 | 0.30 |
Maximum [µg/cm²] | 51.51 | 42.46 | 23.78 |
Confidence Level (95.0%) | 0.29 | 0.24 | 0.13 |
Table 6: Descriptive statistics of optical estimates for a whole leaf. For Zea mays L., number 8 signifies that leaves with the highest concentration levels have been taken into account. The number 1 signifies that leaves with the lowest concentrations have been considered.
The two most important statistical variables besides mean values and variability indicators, are kurtosis and skewness. It can be noticed from Table 6 that the species discriminative power (SDP) of the combined three variables mean, kurtosis and skewness is very strong. Kurtosis is a measure of the "peakedness" of the probability distribution of a real-valued random variable, although some sources are insistent that heavy tails, and not peakedness, is what is really being measured by kurtosis. Higher kurtosis means more of the variance is the result of infrequent extreme deviations, as opposed to frequent modestly sized deviations. In Table 6 we can observe that kurtosis is high with the lowest (Chl_{tot}). Hence extreme deviations of (Chl_{tot}) occur more frequently with low (Chl_{tot}).
Skewness on the other hand, is a measure of the asymmetry of the probability distribution of a real-valued random variable. Skewness values can be positive or negative, or even undefined. Qualitatively, a negative skewness indicates that the tail on the left side of the probability density function is longer than the right side and the bulk of the values (possibly including the median) lie to the right of the mean. A positive skewness indicates that the tail on the right side is longer than the left side and the bulk of the values lie to the left of the mean. A zero value indicates that the values are relatively evenly distributed on both sides of the mean, typically but not necessarily implying a symmetric distribution. With skewness as well, we can observe that it is high with the lowest (Chl_{tot}). Hence asymmetric (Chl_{tot}) distributions occur again more frequently at low (Chl_{tot}). The cited statistical variables are a very strong tool enabling the development of a species discriminating algorithm based on leaf (Chl_{tot}) image statistical patterns.
Leaf (Chl_{tot}) spatial segmentation based on decision tree application
A decision tree classifier performs multistage classifications by using a series of binary decisions to sort pixels into (Chl_{tot}) classes (in this paper). Each decision divides the pixels in a set of images into two classes based on a logical expression. Each new class can again be divided into two more classes based on a second expression. The result of a decision tree sequence is a segmented (Chl_{tot}) image. This approach has been applied on the (Chl_{tot}) imagery of the leaves of Cornus sp., Tilia sp. and Zea mays L. using the decision tree as illustrated in Figure 16.
A decision tree classifier performs multistage classifications by using a series of binary decisions to sort pixels into (Chl_{tot}) classes (in this paper). Each decision divides the pixels in a set of images into two classes based on a logical expression. Each new class can again be divided into two more classes based on a second expression. The result of a decision tree sequence is a segmented (Chl_{tot}) image. This approach has been applied on the (Chl_{tot}) imagery of the leaves of Cornus sp., Tilia sp. and Zea mays L. using the decision tree as illustrated in Figure 16.
Figure 15: Tilia sp., Cornus sp. and Zea mays L. leaf [Chl_{tot}] histograms obtained with IDL/ENVI^{®} based on 50 [Chl_{tot}] frequency classes. It can be observed that Tilia leaves elicit a larger spread in [Chl_{tot}] than Cornus sp. and Zea mays L. The red line illustrates the cumulative frequency. For Zea mays L., number 8 means that leaves with the highest concentration levels have been taken into account.
Figure 16: Decision tree expressed in [Chl_{tot}] to segment leaf [Chl_{tot}] imagery from Tilia sp., Cornus sp. and Zea mays L. The decision tree leads to five [Chl_{tot}] classes, e.g. 0 to 1, 1 to 2, 2 to 5, 5 to 20 and larger than 20 (μg/cm²).
The decision tree classifier leads to segmented chlorophyll imagery as illustrated in Figure 17.
Figure 17: Example of segmented imagery for Tilia sp., Cornus sp. and Zea Mays L. For all imagery the leaf top is upward. The colour legend equals the one depicted in the decision tree in Figure 16.
A first observation is that with this technique the heterogeneous adaxial spatial distribution of (Chl_{tot}) is visualized quite accurately. Secondly the spatial pattern of leaf adaxial (Chl_{tot}) seems to be strongly species dependent. With Zea mays L. the central vein contrasts strongly in (Chl_{tot}) with the neighboring tissues. For the C3 species Tilia and Cornus, (Chl_{tot}) is higher the closer the tissue is located towards a leaf vein in contrast with Zea mays L. Last but not least parallel veins are typical for C4 species as can be seen above for Zea mays L in contrast with the C3 species Tilia and Cornus.
In conclusion, quite a few leaf attributes can be visualized and quantified by spatial segmentation of leaf (Chl_{tot}) using a relatively simple decision tree classifier.
Conclusion
In this paper it is demonstrated that it is perfectly feasible to perform low cost quantitative and non-destructive (Chl) imaging as well as spatial and statistical analysis with a commercial CCD camera at leaf level. Digital imagery of leaves of different plant species can be acquired using a consumer electronics digital camera (NIKON Coolpix) and a Spectralon^{®} panel as calibration reference target to produce reflectance imagery for the RGB bands of the camera. With this technique, (Chl_{tot}) content of leaves can be estimated with a standard error between 0.07 to 0.15 µg/cm² using R band reflectance as expressed by equation 7 in chapter 3.6.
Leaf RTF modeling was applied to corroborate the results obtained, especially the validity of a generic (Chl_{tot}) transfer function. Based on the results of model application, a generic multispecies transfer function converting calibrated leaf adaxial ρ_{R} to (Chl_{tot}) has been defined. Hence, quantified leaf vein asymmetry and other leaf morphological structures are enabled by statistical methods and variable extraction. These can be applied in an automated mode, enabling the identification of plant species using specific leaf ROI’s. Needless to state, that this approach is perfectly complementary to the one-dimensional hyperspectral measurements of leaf pigmentation, since it measures leaf morphological attributes and the spatial distribution of pigmentation instead of pigmentation itself.
Image analysis for routine measurement of intact leaf (Chl) clearly offers multiple advantages over less recent techniques. Traditional (Chl) extraction not only destroys tissue prior to analysis, but involves many steps in sample preparation, which increases the analysis time and increases the variability of the results [4], [26]. Non-invasive measurements have distinct advantages over conventional methods [6], but sometimes require complex, non-linear mathematical and statistical analysis before (Chl) estimates can be accomplished. In the case presented in this paper, relatively simple log-linear transfer functions have been obtained and applied.
Commercial (portable) (Chl) meters have a limited spatial resolution or are non-imaging instruments (which measure chlorophyll absorbance, not reflectance). They are not very reliable for thick leaves and measure small sample sizes of a few cm at most [26], which is a disadvantage for their application with large leaves and/or heterogeneous (Chl) distributions. Moreover spatial analysis is excluded as well as (Chltot) segmentation.
In the applications described in this paper, the camera setup spatial resolution is high enough to make quantitative assessments of (Chl) heterogeneity, in function of plant species. Image analysis, permits non-destructive, non-invasive and quantitative measurement of (Chl) for entire leaves, within a fraction of the time compared to invasive methods. The method demonstrates sufficient sensitivity to detect very small differences in leaf (Chl), yet each measurement is quickly accomplished through simple, routine sampling once calibration has been performed.
A wide range of leaf sizes can be analyzed, the maximum size being determined by the camera fore-optic focal length and the distance between camera and specimen. The image acquisition and analysis system has the added value of application in many other laboratory and field measurements. For example, the described procedure can be directly applied for the quantification of leaf necrosis or chlorosis during the development of a disease or physiological disorder, leaf or other surface area, root growth, flower petal pigmentation in genetic studies, or in vitro cell growth [21].
The pixel size for a leaf of 5 cm length, a camera aperture of 8° and a CCD of 10 MP is 7 µm. This allows for a detailed analysis of (Chl) spatial patterns at leaf level. For example leaf vein asymmetry can be quantified and is elicited to be strongly species dependent. This opens the door to plant and tree taxonomies based on leaf imagery and leaf vein asymmetry and/or other morphological attributes, which hitherto require a lot of manual intervention and manipulation. Evidently a non-calibrated true colour digital photographs of a leaf also enable a taxonomist to identify a plant species, but not on a quantitative basis, but based on leaf type recognition and experience.
Last but not least the findings in chapters 3.6.1. and 3.6.2. lead us to conclude that there is a strong potential for species identification based on leaf level (Chl_{tot}) imaging leading to a new application realm in botany taxonomy, comparable with the FERET evaluation methodology for face-recognition algorithms [27]. Applications are manifold and hence the statistical variables and the patterns unveiled by segmentation are definitely a topic for further research and development towards automated leaf (Chl_{tot}) based plant species recognition.
Acknowledgements
The authors wish to thank the University of Antwerp to make resources available for the students Koen Hufkens, Bert Gielen en Filip Colson that performed the experimental work described in this paper. We would also like to acknowledge the editorial work performed by the anonymous reviewers, which enhanced the scientific quality of this research paper significantly.
Conflict of Interest
No conflict of financial or any other type of interest can be reported.
Bibliography
- Barber J., et al. “Fundamental relationships between plant spectra and geobotanical stress phenomena, Final contractor report” (1981) NAS5-23738, NASA, Greenbelt, MD, USA.
- Barnes JD., et al. “A reappraisal of the use of DMSO for the extraction and determination of chlorophylls a and b in lichens and higher plants”. Environmental and Experimental Botany 32.2 (1992): 85-100.
- Benedict HM and R Swidler. “Non-destructive method for estimating Chlorophyll content of leaves”. Science 133.3469 (1961): 2015-2016.
- Bruinsma J. “A comment on the spectrophotometric determination of Chlorophyll”. Biochimica et Biophysica Acta 52.3 (1961): 576-578.
- Curran PJ., et al. “The effect of a red leaf pigment on the relationship between red edge and Chlorophyll concentration”. Remote Sensing Environment 35.1 (1991): 69-76.
- Daley PF. “Chlorophyll fluorescence analysis and imaging in plant stress and disease”. Proceedings of the 6thInternational Congress on Plant Pathology 17.2 (1995): 167-173.
- Dawson T P.,et al. ”LIBERTY - Modelling the Effects of Leaf Biochemical Concentration on Reflectance Spectra”. Remote Sensing of Environment 65.1 (1998): 50-60.
- Delegido J., et al. “Chlorophyll content mapping over the city of Valencia based on urban vegetation using the hyperspectral index NAOC”. Ecological Indicators 40 (2014): 34-42.
- Ferns DC., et al. 1984. “Application of high resolution spectroradiometry to vegetation”. Photogrammetric engineering and remote sensing 50 (1984): 1725-1735.
- Hardwick K and NR Baker. “In vivo measurement of Chlorophyll content of leaves”. New phytologist. 72 (1973): 51-54.
- Hiscox JD and G F Israelstam. “A method for the extraction of chlorophyll from leaf tissue without maceration”. Canadian Journal of Botany 57.12 (1979): 1332–1334.
- Inada K., et al. “Studies on a method for determining the deepness of green colour and chlorophyll content of intact crop leaves and its practical application”. Proceedings of Crop Science Society, Japan, 32 (1964): 301-308.
- Jacquemoud S and Baret F. “PROSPECT: a model of leaf optical properties spectra”. Remote Sensing of Environment 34.2 (1990): 75-91.
- Jensen JR. “Introductory digital image processing: A remote sensing perspective, 3^{rd} Edition (2005)”. Upper Saddle River, Prentice Hall, New Jersey.
- Kawashima S and M Nakatani. “An algorithm for estimating Chlorophyll content in leaves using a video camera”. Annals of botany 81.1 (1998): 49-54.
- Lillesaeter O. “Spectral reflectance of partly transmitting leaves: laboratory measurements and mathematical modeling”. Remote Sensing Environment 12.3 (1982): 247-254.
- Miller JR., et al. “Seasonal patterns in leaf reflectance red-edge characteristics”. International journal of remote sensing 12.7 (1991): 1509-1523.
- Monje OA and B Bugbee. “Inherent limitations of non-destructive Chlorophyll meters: a comparison of two types of meters”. Hort Science 27.1 (1992): 69-71.
- Richardson AD., et al. “An evaluation of noninvasive methods to estimate foliar Chlorophyll content”. New Phytologist 153.1 (2002): 185-194.
- Salisbury JW., et al. “Significance of non-isotropic scattering from vegetation for geobotanical remote sensing”. International journal of remote sensing 8.7 (1987): 997-1009.
- Spomer LA ., et al. “Rapid, non-destructive measurement of Chlorophyll content in leaves with nonuniform Chlorophyll distribution”. Photosynthesis Research 16.3 (1988): 277-284.
- Swain PH and SM Davis. “Remote sensing: the quantitative approach” McGraw-Hill International Book Co., New-York, NY. (1978).
- Takano Y and Tsunoda S. “Light reflection, transmission, and absorption rates of rice leaves in relation to their Chlorophyll and nitrogen content”. Tokohu Journal of Agricultural Research 21 (1970): 111-117.
- Thomas JR and Gaussman HW. “Leaf reflectance vs. leaf Chlorophyll and carotenoid concentration for eight crops”. American Society of Agronomy69.5 (1977): 799-802.
- Wallihan EF. “Portable reflectance meter for estimating Chlorophyll concentration in leaves”. American Society of Agronomy 65.4 (1973): 659-662.
- Yadava UL. “A rapid and non-destructive method to determine Chlorophyll in intact leaves”. HortScience 21 (1986): 1449-1450.
- Phillips PJ.,et al. “The FERET evaluation methodology for face-recognition algorithms”. IEEE Transactions on Pattern Analysis and Machine Intelligence 22.10 (2000): 1090-1104.
Copyright: © 2015 Frank Veroustraete., et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.