Vision based Identification and Classification of Weld Defects in Welding Environments: A Review

This paper is a review for the identification and classification of weld defects in welding environments based on vision. The revolution in research to develop an autonomous system to identify and classify types of weld defects has been attempted for a long time. According to the difficulty in identifying and classifying small flaws in an image of professional skills, they should be formed accordingly. However the identification process takes time depending on the job knowledge, experience, skills and prejudice. The techniques in the identification welding introduced in this paper are compared to each basic stage of development of welding defects classification system in the welding environment. The geometrical parameter and linguistic gray value is used to interpret the characteristics of the data extraction, including size, location, attributes and shape of weld defects. A perfect knowledge of geometry in the weld defect is an important step in assessing the quality of the weld. In welding classification there are several methods to inspect the weld defect term of type of welds, shapes, information of welding defects such as width, location and position used a statistical tools, neural networks, interference fit line profiles diffuse system and according to the average gray level. A good suggestion can be considered in this work are researchers should focus on some new development that work with grayscale profiles as input set for feature extraction which the welds defect area segmentation step it’s not necessary. Further improvement can be taken is uses a CCD camera only to reduce the development cost.


Introduction
The advanced technology needed in the automation of production processes and quality control inspection to solve welding quality problems has not yet been completely resolved.Because of these shortcomings, active research is needed on inspection and quality control.Inspect the quality welding is used in non-destructive testing.In industries it is often used in X-ray test method for radiographic inspection inside the weld metal.
Weld defects used human inspection is a hard and difficult task for many welds defects should be counted and inspected.The human experience and skill of specialized X-ray radiographs testing should be consider in the welds inspection because it affects the particular inspection task in time and human performance.Recently, methods for welds defects detection in X-ray film or a CCD camera automatically have been investigate to improve processing efficiency and quantify the inspection results.
This review paper aims to study and investigate previous work consistent in various methods of vision based identification and classification of welds in the welding environment.In addition some advantages and limitations are discussed in the previous research method according to their results.Figure 1.0 shows the block diagram of the sequence approaching the literature review.

Related Works
An identification and classification system of weld defects using X-ray films and a CCD camera includes an acqui-sition, feature extraction, identification of welds and techniques is applied.The applications that will be studied and researched entries using classifiers and features will be extracted as a value of geometric parameter of the gray and linguistic description segmented radiographs regions.
In [1][2][3][4] proposed automatic system to classify the types of defects in welds in the radiographic image.The system uses radiographic images as input parameter.Digital image processing techniques are used to extract objects that reflect weld defects radiographic images.Feature extraction using the geometric parameter such as shape, size, location, intensity, gray value.The system is able to identify the different types of weld defects such as crack, hole gas, including hydrogen, lack of fusion, porosity, lack of penetration and weld defects categories subset characteristics.
In 5 describe an automatic detection system uses a fuzzy inference based on the adaptation of the network to recognize weld defects on radiographic images.The purpose of the system is to get the best performance of a classifier based on ANFIS.Feature extraction uses geometric parameters such as area, centroid (X and Y), major axis, minor axis, eccentricity, orientation, Euler's number, equivalent diameter, strength, length and position.
The assessment of the level of gas metal arc imperfections of the CCD camera image quality (GMAW) welding and metal inert gas (MIG) abutment surface defects is introduced by Kumar G S 7,8 .The system uses the gray level for identifying and classifying surface defects by calculating the mean value of gray levels then classify the gray value in the different area according to the characteristic distribution of grayscale.The systems are able to classify different types of defects such as solder, insufficient solder, excessive solder and a good weld.
In 9 proposed detection and classification multiclass welding based on support vector machines, neural networks and k-NN.The system uses the selection functions as a second angular momentum, contrast correction, sum of squares, different inverse, average sum, sum of entropy, entropy, difference variance, entropy difference and texture characteristics.In this system, the types of defects represent in the different groups such as non-default (false alarm), wormholes, porosity, linear inclusion, gas pores, lack of fusion and crack.
In 10 has implemented a weld line defect detection based on multiple thresholds and support vector machine.The system focuses on the detection, location and line segmentation defects in X-ray images of multiple weld thresholds to extract image features.A CCD camera with an X-ray source is the acquisition of inputs.The image extraction from three main areas which is the lead plate, base plate and weld are where there are only exits defects in the weld zone.The system is capable to extract the distribution index characteristic point in the weld defects.
An expert vision system for the automatic inspection of gas lines of weld defects from radiographic films introduced by Gadelmawla E. S. 11,12 .The purpose of these systems is to detect and evaluate weld defects radiographic image of gas lines captured by the CCD camera with X-ray scanner.Feature is extract from shape, orientation and location weld defects includes the area, perimeter, width and minimum bounding rectangle.The proposed system is able to identify 11 the welding defects which are hollow lace, porosity, porosity dispersed, melting, incomplete fusion of the root collar, external slag line reduced, slag inclusions, cracks longitudinal, transverse cracks and cracking of base metals and also also classifies the shape of the defects in three categories which are circular, rectangular and irregular.
In 13 describes an automatic real-time detection of weld defects of steel tubes to detect steel tube welding defects.The systems used a CCD camera with the X-ray image and calculate a required level mean gray level and variance value of the detected area.By using this value, the systems can be sort types of weld defects which are slag, blowholes and incomplete penetration.
A new classification approach weld defects from radiogram pictures based on the expectation maximization (EM) algorithm was introduced by Tridi M 14 .EM method uses vector geometrical characteristics consist of perimeter, surface, direction of inertia and elongation.The system can identify the various types of weld defects which are inclusion of gas, crack, lack of penetration and the oxide inclusion.
In [15][16][17] has proposed the pattern recognition estimated flaws accuracy detection in welded joints by a radiographic image.The system is designed to obtain the estimated accuracy, non-linear classifier, best linear and hierarchical nonhierarchical discriminators for classifying defects with the main geometric parameters such as the extraction of welding characteristics.There are many types of defects introduced into the system using linear classification which are vice undercut (UC), lack of penetration (LP), porosity (PO), inclusion (LI) and nonlinear (NLI), slag inclusions (SI), crack (CR), lack of fusion (LF), non-linear slag inclusions (NLSI) and linear slag inclusion (LSI).
A classification of effectively welding defects has been developed using a large database of simulated fault data proposed by Lim and T 18 .The data of the simulated system radiographic image digitized by the X-ray film scanner and a MLP neutral network to classify weld defects using 25 common descriptors then the result will be grouped into different types of defects which is longitudinal crack, incomplete penetration, porosity, cavity, including slag and transverse crack.
In 19 has implemented threshold image used to extract weld defects in industrial radiographic testing.The research is a comparative study of methods based on thresholding histogram of grayscale, 2-D histogram and locally adaptive approach for the welds defects extraction in the radiographic images.Radiography images with AGFA Arcus II scanner (800 dpi, 256 gray levels) is used to determine the data of histogram grayscale, histogram 2-D and locally adaptive.The system will classify the type of weld defect in different groups such as solid inclusion and porosities, lack of penetration, transverse and longitudinal cracks, inclusions cut systemic error.
In 20 has described a type of defect in the application of artificial neural networks for discrimination with an image processing method to show the shortcomings of computer graphics.The system can identify five types of defects which are blowhole (BH), slag inclusion (SI), down (UC), crack (CR) and incomplete penetration (IP) using characteristic values such as position, relationship between the ratio of the horizontal length and perpendicular to the horizontal length of the area, ratio of the length perpendicular to the surface, complexity, coefficient of formal, heywound diameter, average intensity, the scattering intensity and contrast.
Analysis of surface quality with soft computing for automated quality analysis of the external panels surface body was introduced by Klose A [21][22][23] .The system is a new approach based on processing images in 3-D using the CCD camera as the input images.Surface defects and their characterization are performed by the linguistic description of specific appearances.The types of defects can be grouped into different categories such as sink mark, flat area, uneven surface, press mark, draw line, bulge, dent and uneven radius.
In 24 has proposed a framework for the grading automatic classification of malignant tumor fine needle aspiration biopsy tissue (FNA).Malignancy grades can be classified into two groups which are G2 (intermediate) and G3 (high) using the extraction functions such as area, perimeter, convexity, eccentricity measurement and texture.
In 25 has described an online system based on computer vision using the modular framework for the inspection of surface defects in the copper strips.The system utilizes fuzzy clustering algorithms multi-analysis of the characteristics of the copper surface of the image strip captured by the CCD camera.The extraction of the invariant characteristics such as width-length ratio, rectangular degree circular degree and invariant moment mainly used to identify the categories of deferent types of defects that are no defect, scratch, pit and smearing.
A new system can analyze the images of a sequence acquired vehicle body for detecting various types of defects by using the invariant rotation of the local variance and a Bayesian classifier against-all machines support vector introduced by kamani P 26,27 .The system uses the size and shape characteristics such as area, perimeter, aspect ratio, the ratio of thinness, the major and minor axis length, eccentricity and range to classify the different types defects.The defects that can be detected are leaking, chipping, zero feet, hair, and solvent popping pinhole.
In 28 has developed a techniques based on vision and neural network for inspection in casting surface defects and categorization approach.The system is able to classify the defects into blowholes, shrinkage porosity, shrinkage cavity using the geometric properties attributes such as area, perimeter, ferret elongation, compactness, roughness, length, elongation and breadth.
A new automatic defect detection correction technique known as Correction of Defect (CoD) for using two vision sensors as its core work by 29,30 .The system is focuses on the vision algorithm uses on the shape matching properties to identify defects occurs in the works pieces.The operation of the Gaussian smoothing system is applied to perform better image compared to the median filters.The defects occur by collects the data of height (z-coordinate), the length (y-coordinate) and the width (x-coordinate).
Tomography techniques for weld defects with algebraic reconstruction and simultaneous Cimmino (SART) for iterative image reconstruction introduced by 31 .Weld detect as slag inclusions, lack of penetration and burn through can be determined within the system.In 32 uses the Hilbert transform, ultrasound signal can be decomposed into high and low frequency components to identify the nature of defects in welds.Ultrasonic signal can be obtained five different defects such as lack of fusion, root, crack in the side wall and the transverse crack.1.0 show a summary of related works according to identification and classify welding defects.

Analysis of the Performance of the Previous Works
Normally an automatic welding defect has five levels namely image acquisition, preprocessing, welds extraction, segmentation and classification of defects.Image acquisition can be selected from a CCD camera, CCD scanner X-rays or X-ray film it is often damaged by uneven lighting, noise and low contrast.Due to the poor quality of the image acquisition, the image preprocessing is the first step carried out shortcomings in detection.These include the elimination of noise and contrast enhancement.
The literature reviews of the application an automatic inspection of welding defects, there are many algorithms used.Fuzzy k-nearest neighbors, multilayer perceptron neural network classifier and bootstrap method is used to classify the welds defect 1 .Appropriate algorithms based 3D implement in TableCurve have been found for modeling background for background subtraction method to be effective.These algorithms provide reasonable results faster.Based on the bootstrap method, the trained neural network MLP with 108 defects 92.39% accuracy rating (were correctly classified by an average of 25 of the 27 validation errors).On the other hand, fuzzy K-NN reached 91.57% accuracy, which the MLP neural network is slightly worse.However the systems need a reference image to perform comparison sensitive and it is also no solution for determine the optimum value of the parameter K for a given data.A process of trial and error has been made to find the best value of K between the options for testing.
To allow a change in the number of divisions of each of the universe of discourse, a genetic algorithm to modify the first stage of the WM is introduced by 2 .The system uses 7 functions with the 12 characteristics as done in previous research 1 .The results show that the method of acquiring knowledge generates more fuzzy rules for classification accuracy.However the challenges of how to improve the fuzzy expert system approach so that the accuracy and the performance can be improved.
In 3 has embedded in line profile image of a weld and consists of four modules which are pre-processing, curve fitting, anomaly detection profile and post processing are introduced.Each edge image of the approach used value average gray so each profile has the same size approximately.However the systems do not concentrate to reduce the processing time.The inspection system of linear weld can be extracted from digital radiographs then processed the various weld defects with a detection rate of 93.3%, and to identify the false alarm rate of 4.2% success rate.
Meanwhile 4 has implemented by improving the accuracy of the feature selection method with the following sequential search strategy and the strategy of random search.There are two versions of selection algorithms ant colony optimization (ACO) function in random and sequential basis.Four different classifiers were tested, including the nearest mean, k-nearest neighbor, fuzzy k-nearest neighbors and on the basis of the center of the nearest neighbors.The results show four classifiers that produce the mean error of classification is 16.2% for all welding defects low identification data selection function.
In 5 researchers are capable to develop a system to preserve the accuracy of 100% in applied artificial neural network (ANN).The system detects defects candidates of all defects (true positive) observed by the human expert.A set of 86 images IIW/IIS reference collection X-ray was used.However the problem of the regulation that it is difficult to determine the optimal value of the performance index parameter.
In 6 has introduced a fuzzy inference system based adaptation (ANFIS) network for the classification of welding defects.The system reduces the number of features in the input vector from 12 to 4 geometric properties used as input for ANFIS functions.With different combinations of features can improve classification performance and is the most important combination.Therefore, the correlation coefficients for a minimum value of 0.84 are obtained.The accuracy or the proportion of the total number of predictions was well calculated for a value of 82.6%.However if a data set is not balanced (number of samples differ in different classes significantly) the correlation coefficient of a classification does not represent the actual performance of the classification.
There are four LED areas for efficient information retrieval used as characterize the weld nature 7,8 .The use of artificial neural networks (ANN) with back propagation (BP) and ANN with differential evolutionary algorithm (DEA) separately from the system obtained using ANN with BP, the larger welds shows insufficient 100% accuracy and in excessively good without welds (90%).But ANN using DEA is achieved with 95% accuracy greater all kinds of welds 7 .If the same method is applied in an average vector characteristic of 2D gray-levels the performance showed that only 95% accuracy.This control system however based on the image can be enlarged with different types of connections in the welding process for images classification.Meanwhile the welds can be classified into one of four predefined based on neural networks back propagation 8 .The 80 sample welds were used for training and testing.The highest accuracy is 100% for insufficient solder and the lowest is not welding (90%) where the overall accuracy was 95%.
Another approach is to compare a state of the art methods of multi-class classification (Support Vector Machines and Neural Networks) 9 for detection and classification using seven different classes (including nonsegmented defects).The selection of the proposed features of the systems used to limit the processing of these functions which are really important for each different type to avoid duplication of information.The system is the generalization ability of neural networks and SVM classifier high accuracy (about 85% accuracy).However the system requires more images and a set of public data which are used to create the algorithm evaluation.
Meanwhile 10 , proposed two-dimensional wavelet transform method to compare the transform blocks segmented images.The method is based on multiple thresholds, where extract the features and SVM technique for classifying characteristics.The approaches are effective and feasible to segment and locate defects in low contrast images and noisy X-ray images.Although SVM has a large capacity of the classification, it is necessary to implement the functionality to adjust the examples before the SVM classification.Positive examples are those blocks where defects only in the center easily.
In 11 was developed a system to eliminate any loss of image data on a pipeline because of the deteriorating storage film radiographic films on magnetic media for mass storage.The system improves the capture image so that defects appear much clearer and eliminate the loss of image detail that occurs due to the deterioration time of the film, may be the transfer of radiographic films digitized images stored magnetically in a mass storage medium.The system is quite cheap compared to commercial automated inspection systems and eliminates the need for interoperability images by qualified inspectors.
In 12 proposed a system that could reduce the inspection time for defects that are not clear on radiographic films using pass decision algorithms (ADA).The cost of the inspection process using human inspector's adequate knowledge reduces rather than general inspection process specialists.The results of the lack of information system include measures form factor, the rectangularity factor, shape, orientation and location.However other information about welds and welders will store in the database for further analysis such as welder performance evaluation and common defects generated by each welder.
Fuzzy recognition algorithm is easy to understand and a similar level to human vision using the membership representing the value of the variance of gray and middle gray difference 13 .The advantage of this system is the speed detection is reach 3.5 m/s equal to 3-4 frames/s can be achieved which means that the requirement of real-time detection can meet.The system correctly identifies 65 of 66 defects on the tape.Small defect is near the welded black limit is not labeled and two misinterpretations that regions of noise are considered occur at the rate of the misreport is 3% and reporting rate is 1.5%.However it focuses to improve the aspects of imaging and image processing.
The approach to classify weld defects based on EM and FCMI algorithms is introduced in 14 .The result shows that the rate of classification EM much better between EM and FCMI by adding Bayes classifier.Indeed the EM algorithm is very sensitive to the choice of initial parameter values.However the system strongly recommended the need to increase the size of the features vector and the data base in order to identify great classes of weld defects which existed in industry.
Meanwhile 15 , has developed a hierarchical and nonhierarchical linear classifier using a neural network technique for classification of the principal welding defects.Non-hierarchical linear classifiers are already been possible to promise the success rates in the classification of some of the welding defects most commonly found in radiographic testing.However the welding defects as lack of fusion and cracks will not be analyzed yet because of absence of sufficient data for training and simulation of the classifiers.As a result the hierarchical classifier generally achieved 85% of successes compare to non-hierarchical classifier with only 80%.
A statistical technique interference data with random sample (bootstrap) without repositioning was proposed by 16 .The results in this study obtained closer to the true accuracy of classifiers.The most common types of welding defects, in which classes are below defect (UC), lack of fusion (LF) and porosity (PO) have a high classification accuracy for both training data and classes missing fusion (LF), crack (CR) and the inclusion of slag (SI) in low precision rates.However the systems require used more data for less dominant classes such as lack of fusion and cracks but no less important than the other classes.
Other approach to classify the welds defect in radiographic weld joints is introduce by 17 by using nonlinear classifiers of patterns implemented by artificial neural networks.Nonlinear classifiers provide higher performance results for all defect classes that were studied, when four features of weld defects were used.As result in 5 classes case, the classifier reached the maximum performance percentage (99.2%)and the minimum error percent, with the training data of 17/18 neurons.By implemented fourclass case, the maximum performance percent (100.0%)happened for 10 neurons.However the kinds of cracks and lack of fusion are important in view of the welds have not been evaluated by this technique due to insufficient amount of reliable samples that are available.
In 18 , the researchers developed a Multilayer Perceptron neural network (MLP) trained by parameters extracted using simulated images weld defects.The method of approach solves the problem of limited real samples for classifications are neural network failures.Using this classification system simulated with real defects training samples gave the highest accuracy of 97.96%.Although the systems use a number of descriptors in order to optimize the current job does not produce a significant improvement in the results.
In 19 has a comparative study of the methods of thresholding with nine different methods Otsu threshold, Kittler, Kapur, Tsai, local joint entropy, relative entropy, Niblack and Sauvola.Niblack method has a problem in his lies in the textures of the backlight is treated as objects with low contrast.To overcome this problem, the method Sauvola can be applied.As the conclusion Kapur method is best methods based on the histogram 1-D but the images of non-uniform background intensity and Niblack Sauvola methods are recommended.Process for local adaptation in Sauvola method proves the strongest tool to be emerging.
Region growing method (RGM) proposed by 20 , shows that the defect image with unclear boundary is detected and detection of extra images by unevenness of film is held back.The classification of weld defect to discrimination automatic radiographic testing of welds using artificial neural network.In an experiment of 27 defects, 25 defects are judged rightly.Therefore, the right discrimination ratio is 92.6 % approximately.
Detection of surface defects for assessing the quality of car body panels developed by 21 .In this system a new approach was presented based on 3-D image processing.There have four method applied which are naive bayes, decision tree, multi layer perceptrons and NEFCLASS.Naive Bayes classifier parameters basic shape has not learning, but can be improved by often selecting an optimal subset of features.In an experiment NEFCLASS offers the best compromise between accuracy of results and the transparency of the foreground.However the systems need to predict a quantitative analysis of the severity of a differential form is to be launched into the actions in this way.
To improve the detection of surface defects three approaches to decision tree NEFCLASS and the formation of mixed fuzzy rules are applied 22 .Improvement was obtained in the accuracy, clearly showed that the training consistency and reviewing decisions of experts in the design of the classifier are set up is of great importance when the classes' labels are uncertain and may have errors.The accuracy of the test average of decision trees is enhanced by 4.78%, NEFCLASS improved by 5.88%, and the mixed fuzzy rules by 8.38%.Overall accuracy improved compared to the results of previous work 21 .

However the systems have not yet been reliably obtained descriptions of all classes.
A soft computing for the automated analysis of surface quality of the outer body panels' car was introduced by 23 .In the system of linear regression the clearest signs of over fitting, the best results were obtained in the training data, but the standard classifiers in the test records that are not inferior to see.NEFCLASS offers the best compromise between accuracy of results and the transparency of the knowledge learned.However the system requires a more quantitative analysis of the severity of the form deviation is to predict the actions are started in this way.
An automatic classification of malignant tissue fine needle aspiration with neural networks based on Support Vector Machines (SVM) was proposed by 24 .In systems SVMs perform better achieving an error rate of 5.76% which is recorded the lowest error rate.However the error rate is very promising and more research on feature extraction requires preprocessing and levels to achieve better classification rates because it would be to increase the database include more cases of malignancies.The results show the SVM was able to classify the degree of malignancy very well achieve the highest accuracy of 94.24%.
A multi features fuzzy classification algorithms in inspection system of surface defects for copper strip was proposed by 25 .This system has a high speed, effective capacity, self-learning and highly accurate inspection.The defects detection rate can reach 93.74% which meet the needs of industries.However the performance of the systems should be improved in real-time process and defect segmentation and classification algorithm should be optimized.
Meanwhile 26 , has implemented a one-against-all SVM classifier for automatic detection and classifica-tion of car body paint defects.The best performance is one-against-all SVM classifier achieved of the average rate is 98.81% classification.However the system must be need to design the integrated system that can identify the causes of defects painting by recognizing and classifying them.By Applied Bayesian classifier 27 the average accuracy of 96.2% was evaluated for samples in experiments.This classifier requires a small amount of training data to estimate the parameters (means and variances of the variables) are required for the classification.Prior probability can equiprobable classes are calculated by assuming that all classes equally likely a priori or to charge by estimating the probability of class training set.
In 28 proposed pattern recognition on surface casting defects inspection using neutral network.In the casting surface, neural networks are very good tool for pattern recognition problems.This technique can sort the data with arbitrary precision.The results show that the total percentage of correct and incorrect classification in the experiment is 90% to 10%.However the system requires a large number of elements (neurons called).They are particularly suitable for complex decision problems contour on many variables.Table 2.0 show the summary of previous works on the defect inspection systems.

Summary of Identify and Classify Weld Defects in Welding Environments
From the previous works in the identification and classification of welds defect, we have noticed that this issue has been investigated widely in different ways.Many researchers have used radiographic image rather than CCD camera only as the input acquisition integrated with external light source or control the welding environment to reduce the noise.Most of the researchers used geometrical parameter in feature extraction which includes size, location, attribute and shape of the weld defects.Other parameter can be applied to interpret the data such as gray value and linguistic description.Linguistic description is done by human expert where it needs the experience and skill to observe the welds defect linguistic description.Perfect knowledge of the geometry of the welds defect is an important step which is essential to appreciate the quality of the weld.
The other aspect of interest here is how to categories the weld defects.Many of the previous works discuss weld categories in term of type of welds, defect shape, welds flaw and defect information such as width, location and position.There are several methods to inspect the weld defect such as statistical tools, neural networks, fuzzy interferences system, fitted line profiles and average gray level characteristic.Statistical approaches are usually implemented using classifiers such as bayer, decision tree, SMV and NEFCLASS.Figure 2.0 show the taxonomy aspects used for the weld defects inspection in welding environment.

Conclusion
From the literature review, the identification and classification of welds defects in the different categories is not new but how to select the feature extraction needs further investigation because most of the previous researchers focus on the geometrical parameter only.We believe that it may be a good suggestion that some new developments could be achieved by working with gray level profiles as input set for feature extraction, since in this way the segmentation step of the weld defect area is not necessary.
Nevertheless in the classification of weld defects, most of the research is carried out using neural networks.In the approach the researchers make some considerations on the shape of the defects to choose the most relevant features for discriminating the common welding defects.The input images for data acquisition, radiographic images are still the main selection for the researchers compared to CCD camera.

Acknowledgments
The authors are grateful for the support granted by Universiti Teknikal Malaysia Melaka (UTeM) in conducting this research through grant FRGS/2/2014/FKE/01/ F00 238 and Ministry of Higher Education.

Figure 1 .
Figure 1.Block diagram of the sequence approaching the literature review.

Table 1 .
Summary of studies related to identify and classify welding defects

Table 2 .
Summary of previous works on the defect inspection systems