An Eﬀective Pomegranate Fruit Classiﬁcation Based On CNN-LSTM Deep Learning Models

Objectives: To employ a deep learning technique that would sort the fruits into normal and abnormal based on the features such as fruit colour, number of fruit spots, and shape of the fruit. Methods: A combined CNN LSTM deep learning model is applied to classify a set of 6519 fruits into two classes namely normal and abnormal. The dataset is an excel ﬁle with each record consisting of 9 features. Convolutional Neural Networks (CNN) are applied for deep feature extraction and Long-Short Term Memory (LSTM) is used to detect the class based on extracted features. Findings: The proposed system achieved an accuracy of 98.17%, speciﬁcity of 98.65%, sensitivity of 97.77%, and an F1-score of 98.39%. Novelty: The sensitivity of disease detection was less with lesser availability of enhanced detection methods for detecting disease in earlier stages. The issue with these various existing algorithms is that the accuracy was reduced since some sources of errors were not eliminated. Deep Learning delivers methodologies, approaches, and functionalities that can help to resolve analytic and predictive analysis accurately.


Introduction
Pomegranate (1) is one of the major fruits produced in India. According to International Trade Centre India stands first in the production of pomegranates worldwide. Approximately 5 percent of the fruits produced in our country are exported every year. Pomegranate export earns a considerable foreign exchange for our country but not much research found to be carried out on pomegranate fruit quality classification. It is therefore essential to classify the fruits into normal and abnormal accurately post-yield given marketing and export. The presence of disease in the fruit can be easily detected by external features like the colour of the fruit, the lesions or black spots, the weight of the fruit, the plant stand and so on. There are several studies carried out to detect diseases in fruits using Convolutional Neural Network Some of the work related to the application of CNN are given below.
Tomato leaf disease detection using CNN (2) employed 3 convolutional layers. The hyperparameters of this model are 10000 epochs each with a batch size of 64, a learning rate of 0.001 and a dropout rate of 0.5. This model has generated a testing accuracy of 91.2%. In a VGGNet CNN Model developed to detect diseases in sugarcane (3) having 40 epochs gives a training accuracy of 95.40%. (4) Classified and graded okra using ResNet50 CNN model and it has produced a classification accuracy of 99.17%. An attempt to detect diseases in maize using CNN (5) resulted in a validation accuracy of 99.28%. The deep learning approach in detecting diseases in the fruit guava (6) is carried out by Farhan and Rubaiya shows a testing accuracy of 95.61. The data size of the system is 10000 and the number of epochs for training the dataset is 50. Another CNN LSTM model application for apple disease detection (7) was able to classify good and bad apples with an accuracy of 99.2%. The experimental analysis of the ANN classifier algorithm combined with the SVM classifier algorithm in citrus fruit disease detection (8) has shown a classification accuracy of 88.96%. Plant disease Detection using Deep Learning (9) has achieved a testing accuracy of 98% with hyperparameters of epoch numbers 25 and a batch size of 32. The learning rate of the CNN Model in this research is 1e-1. It uses the activation layer Relu. Another research for plant disease detection using tensor flow (10) was able to detect diseases such as bacterial blight, Cercospora, anthracnose with a testing accuracy of 90%.
Similarly in an approach to classify mango leaves infected by anthracnose disease (11) using CNN has made a classification with the accuracy of 97.13%. A deep convolutional neural network training model (12) to identify 14 crop species and 26 diseases achieved an accuracy of 99.35%. Fruit recognition from images using deep learning (13) had recognized different types of fruits such as apple pomegranate etc. with an accuracy of 98%Fruit recognition using images is carried out using CNN (14) and obtained an accuracy of 100%. There are very few works done in detecting diseases in pomegranate fruit and leaves using deep neural networks and CNN. Some of the related work is listed below. A recent CNN Model developed to detect pomegranate plant leaf disease (15) using CNN has achieved a 100% accuracy in classifying a good fruit and obtained an accuracy of 85.71% in detecting bacterial blight disease. In another work to detect diseases in pomegranate using CNN (16) where an input of 60 images are given a good fruit detected with 88% accuracy and an infected fruit detected with an average accuracy of 79%. A work to detect bacterial blight disease in pomegranate fruit (17) using CNN has a classification accuracy of 90%. A multi class SVM (18) with K-means clustering for classification of diseases gives a 90% of accuracy. (19) Pomegranate fruits are classified as healthy and diseased using the SVM classifier and ANN and it resulted in a classification accuracy of 80% and 92.65% respectively.
It can be observed from various researches carried on to detect plant or fruit diseases using deep learning, CNN approach has proven to produce effective accuracy results The present study describes the accuracy obtained from machine-based models to classify pomegranate into two classes normal and abnormal. Healthy fruits are referred to as normal and diseased fruits are referred to as abnormal. The data is collected by observing the important features of fruits that quickly exhibits the quality of fruit and is recorded. Disease prediction in the fruit is connected to many factors such as weight of the fruit, number of spots on the fruit, fruit shape, the plant stand and defoliation in the tree. The classification of pomegranate fruits is carried out by a classifier model that was trained on the training data to predict the class label of new testing data. The novelty that our present work provides is feature extraction task is done using CNN as in previous researches but the LSTM model is combined with CNN to classify fruits which is considerably improving the accuracy of classification. The main advantage of using LSTM for classification is that it uses several parameters such as learning rate, input bias, output bias which minimizes the complexity of weight update.

Related Theory and Fundamentals
The following section introduces CNN and LSTM used in this study.

Convolutional Neural Network (CNN)
CNN (20) is a type of deep neural network which mimics a biological neural network. It is suitable for pattern recognition and voice recognition. Being an efficient tool in machine learning it is popular in classification dataset. CNN is ideal for processing 2D images. CNN is made up of several layers which include the Input layer, Convolutional layer, pooling layer fully connected layer, ReLu, Drop out and classification layers as shown in Figure 1. The functions of these layers are briefed below.
Input Layer: This is the first layer of CNN. In this layer pre-processed input image is given to the network. Convolutional Layer: It is the basis of the CNN model. It is responsible for extracting feature maps by using pixel matrices of images. This layer produces a new image matrix or feature map. The operation of the convolution layer is given as Where I denote input matrix, K denotes a 2D filter, F represents the output of a 2D feature map.
https://www.indjst.org/ Rectified Linear Unit (ReLu): This layer comes after the convolution layer. It consists of an activation function that will output the input directly if it is positive otherwise it will output zero. This layer is also called as non-linearity layer. It is mathematically expressed as Pooling Layer: This layer reduces the dimension of the feature maps generated by the convolutional layer. Therefore, it reduces the number of parameters to learn and the amount of computation. Fully Connected Layer: This layer summarizes the results of the previous layers and uses them to classify the image into a label. The neurons in this layer are fully connected to all areas of the previous layer.
Dropout Layer: This layer helps prevent the model from over fitting. Dropout can be implemented in any layers but not in the output layer. This layer uses a hyperparameter called dropout that randomly sets the outgoing edges of hidden units to 0 at each update of the training phase. This layer is responsible for the random deletion of some nodes in the network.
Classification Layer: This layer comes after the fully connected layer and here is where the classification takes place.

Long Short Term Memory (LSTM)
Long Short-Term Memory network, or LSTM (21) for short, is a category under recurrent neural networks that provides stateof-the-art results on several complicated prediction problems. It is used to solve the problem of short term memory using gates which helps in the proper flow of information. The long term memory in LSTM is called cell state and the cell state is modified by three gates namely input gate, forget gate and output gate. The architecture of the LSTM model is shown in Figure 2. The principle of the input gate of LSTM is described below Where, W i denotes weight matrices b i denotes input gate bias C t =memory information C t = tanh output Forget Gate: This gate decides which bits of the cell state are useful given both the previous hidden state and new input data. The principle of the output gate is given below Where,W f represents the weight matrix b f is the offset σ is the sigmoid function Output Gate: The output gate decides the new hidden state based on the newly updated cell state, previously hidden state and the new input data. The output gate principle is shown below where W o denote output gate's weighted matrices b o =LSTM bias

CNN-LSTM Model
The CNN model described above is capable of handling a single image transforming it from input pixels to a vector representation. But we need to repeat this process for multiple images and allow the LSTM to build up an internal state by updating weights. Hence our proposed work uses a single CNN model and a sequence of LSTM models one for each time step.
Here CNN Model is applied to each input image and the output is input to the LSTM as a single time step.

Proposed Methodology
In the current study a deep model is proposed that is based on deep features extracted using CNN and LSTM network. The deep features are extracted from fully connected layers. The extracted deep features are sent as input to the LSTM layer. After LSTM layer a fully connected layer, a softmax layer and a classification layer are used that would sort the images to normal and abnormal which are represented with class labels 0 and 1 respectively. In the current study we extracted deep features by samples to work through before updating the deep network parameters. The structure of the proposed work is shown in Figure 3 https://www.indjst.org/

Result and Discussions
This section describes the dataset, experimental works and the results achieved in detail.

Dataset Acquisition
Due to the non-availability of the benchmark dataset on pomegranate fruits, we have obtained the data by visiting pomegranate farms in and around Bangalore. Twelve farms are visited and images of fruits are captured using the DSLR and iPhone camera. The images were captured in the daylight. Some images are downloaded from Kaggle Dataset and Cofilab Dataset. Some representative image samples from the dataset are shown in Figure 4. As given in Table 1 the dataset includes a total of 6519 images belonging to two different classes namely healthy and diseased. The dataset contains normal and abnormal fruits in the ratio 2:1 respectively which is a balanced dataset.

Experimental Setup
The proposed model was implemented using the Python toolbox. The experiment was conducted using a MacBook pro with an 8-core CPU with 4 performance cores and 4 efficiency cores,8-core GPU,16-Neural engine and 8GB of unified memory. During the experiments, 62% of the dataset was used for training and the remaining 38% used for testing the proposed model.

Significance of hyperparameters
Hyperparameters (22) are those that directly control the behaviour pattern of the training Algorithm and have a significant impact on the performance of the model being trained. A CNN Model with finely tuned hyperparameters can yield the best results. The most important hyperparameters are the number of epochs, learning rate and drop out.

Number of Epochs
An epoch (23) represents the number of passes taken by the machine learning algorithm to complete the entire training dataset. Datasets are usually grouped into batches when the amount of data is very large. If the batch size is the whole training dataset then the number of epochs is the number of iterations. The number of epochs needed for a model to run is decided based on several parameters that are related to both the dataset used and the purpose of the model, and where there have been efforts to turn this process into an algorithm, often a deep understanding of the data itself is indispensable. The number of epochs used in our model is 50.

Learning Rate
The magnitude of the weights is continuously updated during the training. This process of weight update is called learningrate (24) . Thus, learning-rate is a hyperparameter used in the training of neural networks and its value ranges between 0 .0 and 1.0. The learning rate of the current model is 0.001.

Activation Function
ReLU is an acronym of Rectified Linear Unit and is a kind of activation function (25) . Mathematically, it is expressed as Y = max (0, x). Visually, it looks as in Figure 5. ReLU is the most commonly used activation function in neural networks, especially in CNNs. It will output the input directly if it is positive, otherwise, it will output zero. The ReLu activation function enables models to learn quickly and perform better by overcoming the problem of vanishing gradient.

Performance Evaluation
The confusion matrix (27) or error matrix is an error specific Sensitivity: It is defined as the proportion of real positive cases that are correctly predicted positive. It is given as: Specificity: It is represented as the proportion of actual negative, which got predicted as the negative. It can be represented as Recall: It indicates the proportion of actual positives which are identified correctly. It can be determined by F-score: It is a measure of model accuracy on the applied dataset. It can be equated as follows.
Precision: It is defined as the proportion of predicted positives that are correctly real positives. It is given as: Accuracy: It is the most institutional performance that measures the correctly predicted observation to the total observations. It defines the ability to distinguish between normal and abnormal cases. It is given as: False positives and false negatives correspond to type-I and type II error respectively. The confusion matrix obtained for our model is shown in Figure 6.
https://www.indjst.org/ The performance of the model is further explored by the computation of precision, recall and F-score. These values are shown in the classification report Table 3. The accuracy of the model is obtained as 98%.  Figure 7 shows the accuracy curve for the given model. The model loss curve is shown in Figure 8.  Table 4 we have shown a comparative analysis of our work with previous research carried out in fruit quality classification using machine learning algorithms. It can be observed from the comparison table that CNN LSTM produces more accuracy in the classification process and also it can be noted that considering more data size and trials results in higher accuracy in assessing the quality of fruits and classifying it as normal and abnormal. Pomegranate Disease Classification. (28) Ada Boost Ensemble Algorithm 92. 9 3 Diagnosis of Pomegranate Plant diseases (29) Back Propogation Algorithm 90 4

Conclusion and Future Work
The current study implements a CNN LSTM model using Python to detect the diseased pomegranate and classify them into normal and abnormal. The fruit classification accuracy was found to be 98.17%. This accuracy is considerably high compared to the accuracy obtained using the classifiers such as SVM, ANN and Back Propagation Algorithm. But the proposed model is limited with only two classes so as for future enhancement of this research is concerned, we aim to develop a CNN LSTM model with multi-way classification to predict four common pomegranate diseases such as anthracnose, bacterial blight, Cercospora fruit spot and heart rot.