Siamese Neural Networks for Kinship Prediction: A Deep Convolutional Neural Network Approach

Objective: This study worked out kinship veriﬁcation, which is a laborious problem in scientiﬁc discipline and pattern discovery. It has many applications, such as: ﬁnding missing children, identify family and non family member. The aim is to kinship prediction using similarity computation to identify kin and non-kin based on image dataset. Method: To measure the similarity score of the proposed method on primary 96-family dataset, considers 410 images and 77,887 diﬀerent pairs. The data was split into 80% for training and 20% for testing. This study proposed Siamese Deep Convolutional Neural Network model with deep algorithm viz., ResNet and VGGNet with Adam optimizer to verify the kinship for the four kinship relations of father-son, father-daughter, mother-son and mother-daughter. Findings: It is observed that, the proposed model gives better perform and with 72.73% average similarity score. Novelty: Experimental results on the primary kinship datasets showed the superior performance of the proposed methods over state-of-the-art kinship veriﬁcation methods and human ability in our kinship veriﬁcation task. In the future, this model can be applicable for kinship veriﬁcation in society.


Introduction
In recent years the territory of image processing has developed due to which kinship verification has also come to light.Relatively new branch of Biometrics developed such as kinship recognition (1) .The use of human face images in identifying relationship is a strenuous research problem in biometrics and Computer Vision (2) .Also, human facial image analysis has passionate lots of interests from image processing and computer vision community for a long time, e.g., face recognition, gender, detection face detection, landmark, facial attributes perception age (3) .Relationship verification means verifying if two people are biologically related by measuring their similarity (4) .Kinship analysis can be classified as: verification, classification and recognition.The goal of relationship verification is to identify whether two human faces are related by blood https://www.indjst.org/(i.e., kin or non-kin), the purpose of family classification is to identify the family of the subject.Families are created using the faces of all but one family member and also used for testing with an omitted member and kinship recognition refer as kinship detection, to identify difference between genetically close such as kin and non-kin.The solution of this type of problem, in computer vision, a established such method to determine the exact kin relation using two human face images.Those methods predict which class such as father-daughter (FD), mother-daughter (MD), father-son (FS), mother-sun (MS), brother-brother (BB), brother-sister (BS), sister-sister (SS) etc. (5) In practice, it has many applications such as forensic analysis finding missing children, social work analysis and builds a family tree from a photo album.Among the chances found in the study of kinship verification, the manifest similarity of two human who have no such relationship and the case when human have blood alliance and don't have a same aspects.In addition, effect of the different conditions of images captured in the society so for that purpose kinship verification is very important.
The remaining part of this paper is organized as follows: Section II introduces related work in recent years.Section III dealt with proposed method.Section IV validates the performance of our method by Comprehensive experiments and gives the experimental analysis.Section V conclusion of paper in our research, and we also discuss the challenges in this area and scope for further study.

Related Work
Many research works have been conducted on facial kinship verification, in (6) authors were introduced deep Siamese Neural Networks for facial expression recognition in the wild and evaluated algorithm using AffectNet, FER2013, and Compound Facial Expressions of Emotion (CFEE) datasets.In (7) the authors, NRML metric learning, SVM classifier were used to generate the discriminative feature vector to verify to kinship relations on KinFaceW-I and KinFaceW-II databases and measure the similarity score.In their study, (8) authors investigated the fusion of CNN classifiers for the image-based kinship verification problem, namely EFNet and LFNet.They also designed the fusion CNN classifier with ResNet-50, ResNet-101, and DenseNet, evaluating performance on the FIW dataset In (9) author proposed a family-aware CNN classifier for the visual kinship verification problem, and the experiment yielded good results on the FIW dataset using deep face models such as ResFace-101, SphereFace, and VGGFace.In (10) author was worked on Feature Extraction and classification for decision (kin or non-kin) and proposed ALEXNET, SVM and KNN model by new deep neural network mdeol like Google net, Resnet-50, Inceptionv3, but the best results were received by Alexnet deep neural network on the KinFaceW-I and KinFaceW-II database.
In (11) author proposed an approach involving face preprocessing, feature extraction and selection, and kinship verification on five different databases, namely Cornell, UB KinFace, Family 101, KinFace W-I, and KinFace W-II.They achieved good similarity score using deep feature models such as VGGNet, AlexNet, ResNet, and ImageNet.In (12) author proposed a kinship face generator network in three stages, namely, extracting robust facial features; adversarial scheme and cycle-domain transformation approach achieve promising intuitive results on the FIW dataset.In (13) authors were applied a deep fusion Siamese neural network to the RFIW2021 dataset and obtained effective results using the ResNet50 and SENet50 deep learning.

Siamese Deep Convolutional Neural Network
The Siamese deep convolutional neural network is a specialized architecture designed for various tasks, including image similarity comparison; face verification, and kinship verification.The main idea in the background the Siamese network is to learn a similarity metric that can extent the similarity or dissimilarity between two input images.Merging and symmetry are two important features of Siamese structures for identical ANNs with each other, utilizing the same array of weight coefficients obtained during the pre-training process, and enabling parallel translation of initial descriptions of compared objects into a feature space convenient for their comparison (14) .Workflow of a Siamese deep convolutional neural network such as: in the data collection and preprocessing collect dataset and arrange in pair and In the processing dataset input image is resized to 64×64 and cropping into different regions (left-top, left-bottom, right-top, right-bottom) and applying any necessary data augmentation techniques to increase the model's robustness.With this Network Architecture used like ResNet, VGG, or custom-designed networks can serve as the sub-networks.Loss function is the contrastive loss, which penalizes the model when the predicted similarity for similar items is too low and when the predicted similarity for dissimilar items is too high.Train the Siamese network using the prepared dataset and the contrastive loss function.The network learns to extract discriminative features and produce similarity scores for input pairs.Evaluate the model's performance on a separate validation set to monitor its progress during training and make necessary adjustments.After training, test the model on a hold-out test set to assess its generalization https://www.indjst.org/ to new data and predict the similarity score structure of Siamese neural network shown in Figure 1.

Fig 1. Siamese deep convolutional neural network
The Siamese deep convolutional neural network offers an effective way to learn similarity metrics for various applications.By following this methodology, we can adapt and deploy it for the specific use case, achieving accurate similarity comparisons between pairs of data samples, workflow of this study with Siamese deep convolutional neural network shown in Figure 2 and mathematical model for convolutional neural network is as

.d
Where B i = i th bias.Y i = i th output, X j = j th input, K i j =Kernels of i th output and j th input, and similarity score formula is,

Data set
We have collected primary dataset of 96 family's data visualise in Figures 3 and 4. It includes 95 father images 96 mother images 92 sun images and 127 daughters.In the processing dataset input image is resized to 64×64 and cropping into different regions (left-top, left-bottom, right-top, right-bottom).In this work, we have used python packages and libraries namely tensorflow, sklearn.utils,sklearn.model,_selection,pathlib, cv2, random, numpy, os, matplotlib.pyplot and tensorflow.keras and in this package libraries are layers, losses, optimizers, metrics, Model, resnet, train_test_split and shuffle.Also, with this, we have categorized seven types of relationship pairs into two groups.The first group comprises four types of relations: father-daughter (F-D), mother-daughter (M-D), father-son (F-S), mother-son (M-S).The second group includes sister-sister (S-S), brother-sister (S-B), and brother-brother (B-B), which span relationships across two generations as shown in Figure 5.

Result and Discussion
In this section we have explored our result and compared with results of other methods.Table 1 shows the verification results of different kinship verification algorithms on different dataset.Our method significantly outperforms the state-of-the art method.We have used similarity score as a performance measure in this work.7.5 41.6 49.4 48.8 46.83 CACD MDLN (16) 63.1 66.4 63.2 60.8 63.4 TSKinFace WGEML (17) 66.2 68.22 63.22 68.32 66.54 FIW ResNet+SDMLoss (18) 69.02 68.60 72.28 69.59 68.37 FIW SphereFace (19) 69 Facial images from our primary collected datasets, along with some sample positive and negative pairs are shown in Figures 6  and 7, and the proposed train model exhibits trends in various metrics across epochs.Between epochs 3 and 7, the similarity score rises, followed by a decline from epochs 7 to 13.Subsequently, it displays slight fluctuations, alternating between increases and decreases, ultimately stabilizing from epoch 13 up to epoch 160.Similarly, the loss decreases from 0 to 5 epochs and remains consistent up to epoch 160.Experiments on the primary dataset show that our proposed model achieves an average similarity score is 72.73% as compared to SPP, Sphere-Face, Multi Abstract Fusion etc.These results could inspire researchers to delve further into kinship verification from facial images for real-world scenarios and studying this challenge as part of future work would be beneficial.Furthermore, in the continuation of this research, we intend to place greater emphasis https://www.indjst.org/

Fig 2 .
Fig 2. Flow chart of the study

Fig 5 .
Fig 5. Sample image pairs and their relationships from our data set Figure 8 illustrate the behavior of the validation accuracy and validation loss graphs FD train model and also similar graphs for FS, MD and MS train model.https://www.indjst.org/