Acoustic emission-based damage localization using wavelet-assisted deep learning

Abstract

Acoustic Emission (AE) has emerged as a popular damage detection and localization tool due to its high performance in identifying minor damage or crack. Due to the high sampling rate, AE sensors result in massive data during long-term monitoring of large-scale civil structures. Analyzing such big data and associated AE parameters (e.g., rise time, amplitude, counts, etc.) becomes time-consuming using traditional feature extraction methods. This paper proposes a 2D convolutional neural network (2D CNN)-based Artificial Intelligence (AI) algorithm combined with time–frequency decomposition techniques to extract the damage information from the measured AE data without using standalone AE parameters. In this paper, Empirical Mode Decomposition (EMD) is employed to extract the intrinsic mode functions (IMFs) from noisy raw AE measurements, where the IMFs serve as the key AE components of the data. Continuous Wavelet Transform (CWT) is then used to obtain the spectrograms of the AE components, serving as the “artificial images” to an AI network. These spectrograms are fed into 2D CNN algorithm to detect and identify the potential location of the damage. The proposed approach is validated using a suite of numerical and experimental studies.

Introduction

Civil structures are prone to damage due to structural ageing, faulty construction, natural disasters, accidental loads or lack of adequate, timely maintenance. Their existing condition can be monitored using sensor-driven structural health monitoring (SHM) strategies to avoid any catastrophic failure [18]. SHM involves data-driven technology to monitor and detect the damage and provide the infrastructure owners opportunities for timely maintenance. Over the last several years, Acoustic Emission (AE) has shown great promise to detect and localize different internal damage patterns such as crack, fatigue, corrosion in civil structures [32]. In general, AE is defined as a transient elastic wave generated as an outcome of a material deformation due to damage initiation and propagation [27]. Depending on the nature of structures and loading conditions, a suite of AE parameters such as duration, signal strength, amplitude, rise time, counts, and energy indices reflect their as-is state and detect any anomalies. However, these parameters are often sensitive to operational conditions, the severity of damage and measurement noise present in the data. In this paper, an improved damage detection and localization technique is developed using a deep learning method augmented with the time–frequency decomposition of AE data.

Over the last several years, AE technique has been applied to detect and localize damage in various structural elements such as beam, slab, wall, and full-scale structures [1, 2, 5, 9, 41]. For example Abouhussien and Hassan [3], proposed an AE monitoring system to evaluate the performance of bonds of reinforcement rebars subjected to corrosion in reinforced concrete structures. The traditional AE parameters were considered to detect, quantify, and classify the bond damage. The delamination defects in concrete containment structures were identified using the AE-based technique [12]. AE parameters obtained from an instrumented concrete wall were considered to detect and classify the damage using a novel clustering and visualization approach. Quy and Kim [26] proposed a method based on AE data to detect and localize the defect in pipeline systems. The time–frequency decomposition method was used to detect the AE event as well as localize the AE source by determining the time difference of arrival. An experimental test was conducted on a pipeline system subjected to internal pressure while collecting AE data using AE sensors. It was concluded that the proposed method could be applied for both offline and online monitoring of ageing pipelines.

In a recent study, Manthei and Plenkes [22] provided a comprehensive review of AE-based structural condition assessment in underground structures. In this research, the AE technique was applied to detect the different levels of deformation in mines. On the other hand, Calabrese and Proverbio [10] provided a review of the AE techniques to detect the damage due to corrosion in various engineering systems. In addition, the study discussed the importance of the AE technique as a damage detection and localization tool for material or systems subjected to corrosion-induced damage. On the other hand, Verstrynge et al. [39] discussed the application of AE-based methods to detect, assess, and identify the severity of damage in masonry structures. The study provided a detailed discussion regarding the current challenges, findings, and future works that can be conducted to improve the performance of AE-based methods as a damage detection tool for masonry structures.

Ma and Du [23] integrated AE parameters into an advanced deep neural network to detect cracks in prestressed concrete elements. A set of experimental and full-scale studies were undertaken to extract the crack signature of the prestressed concrete samples using AE features. In a recent study, Wang et al. [42] developed a new technique based on AE measurements to evaluate the damage in concrete elements. The wavelet packet transform method was utilized to decompose the measured AE signal into a set of frequency bands, followed by calculating the energy of the coefficient in each frequency band. The variation in energy across various coefficients was used to estimate the damage level. On the other hand, Barbosh et al. [8] applied a new feature extraction technique by combining EMD with Shannon Entropy in AE data to detect and localize minor damage. The proposed technique was used to monitor and evaluate the existing condition of the walls in a dam under various operational conditions. However, most of these studies are either based on pattern recognition or signal processing techniques that require a computationally intensive suitable selection of features to identify the severity and location of the damage using AE data.

With the advancement in deep learning (DL)-based Artificial Intelligence techniques, the SHM researchers have explored various Convolutional Neural Networks (CNNs) for crack localization [11, 35]. In these methods, the actual images of the crack can be used as the training data for the identification and classification of cracks without requiring any specific feature selection. Recently, fast Fourier transform (FFT) and various other time–frequency (TF) methods such as short-term Fourier transform (STFT), wavelet transform (WT) [4, 28], empirical WT (EWT) [13, 43] transform (HHT [6, 15] empirical mode decomposition (EMD) [7, 31], and synchro-squeezing transform (SST) [19, 24, 34], have been explored to obtain TF images that were fed into DL techniques in both mechanical and structural systems [38]. explored three different TF methods (i.e., STFT, WT and HHT), and the resulting TF images of bearing diagnostic data were used as the input to the CNN. It was shown that the resizing of TF-images greatly minimized the training time but significantly reduced the classification accuracy [40]. used eight different TF methods, and the TF images were used as the input to AlexNet-based DL architecture. Two bearing datasets with different loads and speeds have been used to validate the proposed method. Whereas STFT provides low-resolution TF images due to the fixed window size, CWT can provide high-resolution images in both time and frequency domain due to its self-adaption of window size [29, 38, 40].

Pandhare et al. [25] used spectrograms as the input to the CNN. The performance of CNN was compared with conventional machine learning methods [14]. used EWT to convert current signals of a three-phase induction motor to 2D grayscale images, which were used in CNN to classify several types of faults, including bearing axis deviation, stator and rotor friction, rotor aluminum end ring break, bearing noise and poor insulation. Sun et al. [36] adopted Multi-SST to analyze time-varying signals for obtaining TF matrices that could accurately reflect the raw signal information. Sparse Feature Coding and Dictionary Learning were then implemented to extract the low dimensional and most discriminative features. Liu et al. [20] proposed a fault diagnosis method based on CNN and transfer learning, where CWT was used to extract the features of the sampled signal data, and the resulting scalograms were used as input to the LeNet-5 CNN architecture. It was shown that the results of CNN converged very fast with reasonable accuracy. Zhang et al. [44] used 2D spectrograms which were used as input to the LeNet-5 CNN structure. It was demonstrated that the proposed method had a high fault-diagnosis accuracy rate and strong classification ability for various health conditions. In another study, Kumar et al. [17] implemented grayscale scalograms of an accelerometer installed on a bearing housing as the input to the CNN. Shao et al. [33] proposed a DL-based multi-signal fault diagnosis method in which the acquired data was converted into scalograms that were used as input to the CNN. However, most of the research associated with TF image-based deep learning has been explored only in mechanical systems; there has been a very limited focus on SHM.

Contrasting machine health monitoring, there has been minimal research conducted in SHM where the TF images were fed into CNN for damage or anomaly detection. Tang et al. [37] proposed a data anomaly detection method based on a 2D CNN. FFT was implemented to generate frequency domain images of the segmented time series, which were analyzed in CNN for data anomaly classification. The results showed that the proposed approach could detect the multipattern anomalies (such as outlier, minor, missing data, trend and drift) of SHM data with high accuracy. Acceleration data of a real long‐span cable‐stayed bridge was used for validation, and the results were compared with a deep neural network-based framework. Li et al. [21] used SST to represent the energies of AE signals in the TF domain. A multi-branch CNN model with Adam parameter optimization algorithm was developed as the feature extractor and classifier to automatically distinguish multiple types of AE waves resulting from crack, impact and operational conditions. The proposed model utilized the TF energy distribution features of AE waves in a rail track and achieved higher accuracy than the traditional CNN models. However, most of these applications were employed directly using the vibration data, which are only suitable for global damage detection and have a limited application with respect to localized damage detection such as crack localization.

This study aims to propose a new AE data-based 2D CNN model to automate the detection and identification of the potential location of the damage. In this paper, an EMD-based TF method is developed to eliminate the presence of noise and extract the key AE components of the measured data obtained from limited AE sensors. Continuous Wavelet Transform (CWT) is then used to extract the spectrograms of the key AE components. The resulting spectrograms form the training and testing database to validate the proposed 2D CNN, which can serve as a damage detection and classification tool without considering the standalone AE parameters and feature extraction. The paper is outlined as follows. A brief background of CWT and 2D CNN is presented first, followed by the proposed method. A suite of numerical and experimental studies is conducted next to show the performance of the proposed method.

Proposed AE-based damage localization method

Continuous wavelet transform

WT provides an improved TF representation of the signals in a multi-resolution framework. Continuous wavelet transform (CWT) is considered one of the powerful signal processing techniques that are applied in various applications such as image compression, noise removal, and pattern recognition [29] CWT of a signal $x(t)$ is defined as:

$${c}_{m}^{r}\left(x\right)={\int }_{-\infty }^{\infty }x\left(t\right){\psi }^{*}(\frac{t-m}{r} )dt$$

(1)

The inverse CWT (ICWT) can be determined as follows:

$$x\left(t\right)=\frac{1}{{C}_{\varphi }} \underset{-\infty }{\overset{\infty }{\iint }}\frac{1}{\left|r\right|}{c}_{m}^{r}\left(y\right){\psi }^{*}\left(\frac{t-m}{r}\right)dm\frac{dr}{{r}^{2}}$$

(2)

where ${C}_{\varphi }$ can be expressed as:

$${C}_{\varphi }={\int }_{-\infty }^{\infty }\frac{|\psi (f){|}^{2}}{|f|} df<\infty$$

(3)

where $r$ and $m$ represent scale and translation of the mother wavelet, respectively. The basis function is called mother wavelet $\psi (t)$, where superscript $(*)$ denotes its complex conjugate. With the appropriate choice of $r$ and $m$, CWT utilizes the shifted and scaled versions of $\psi$ and subsequently estimates its inner product with $x\left(t\right).$ In this paper, CWT is applied to generate spectrograms of key AE components (i.e., IMFs) that are used to feed the CNN model to detect and identify the potential location of the damage. A mixture of sine signals with three different frequencies, including ${f}_{1}$= 1.4 Hz, ${f}_{2}$= 3.5 Hz, and ${f}_{3}$ = 7.0 Hz is considered to show the performance of CWT as shown in Eq. 5. CWT is applied to the mixed signal using two popular mother wavelet basis functions (i.e., Morse and Morlet) to investigate and compare the resolution of extracted CWT spectrograms of both basis functions. Figure 1 (a) shows the Fourier spectrum of the mixed-signal $X(t),$ whereas Fig. 1 (b-c) represents the spectrogram of $X$ obtained from CWT using Morse and Morlet function as a mother wavelet, respectively. It can be seen that the signal contains three frequencies, which appear in the FFT and spectrogram. Moreover, it can be observed in Fig. 1 (b) that the CWT spectrogram of Morse wavelet provides a better resolution compared with the spectrogram of Morlet wavelet, as shown in Fig. 1 (c). Therefore, this study uses the Morse wavelet as the mother wavelet to extract the CWT spectrograms herein.

Fig. 1

(a) Fourier spectrum of the mixed-signal and (b) and (c) its spectrogram obtained from the CWT using Morse and Morlet basis functions, respectively

$${x}_{1}=\mathrm{sin}\left(2\pi {f}_{1}t\right), {x}_{2}=\mathrm{sin}\left(2\pi {f}_{2}t\right), {x}_{3}=\mathrm{sin}\left(2\pi {f}_{3}t\right)$$

(4)

$$X= {x}_{1}+ {x}_{2}+{x}_{3}$$

(5)

Convolutional neural network

A Convolutional Neural Network (CNN), as depicted in Fig. 2, is a Deep Learning algorithm that assigns learnable weights and biases to various features present in 1D and 2D data in order to differentiate them from one another. The architecture of CNN is similar to that of the connectivity pattern of neurons in the human brain, and it was inspired by the organization of the visual cortex. The three major parameters, including local receptive fields, weight sharing and sub-sampling in the spatial domain, make CNN highly suitable for feature extraction from 2D data such as images. A typical CNN architecture comprises feature extraction layers, including convolution layers, batch-normalization layers, activation layers, and classification layers consisting of fully connected layers, softmax layers and classification output layers.

Fig. 2

An example of a typical CNN architecture

The convolution layer is the first layer of feature extraction from an input image using the convolution operation between an image and a filter (or kernel). A 2D convolution layer applies to slide convolution filters to the input. The layer convolves the input by moving the filters along the input vertically and horizontally and computing the dot product of the weights and the input, and then adding a bias term. The convolution operation is an element-wise product of images and filters, followed by the summation of these values. The output of the convolution operation represents a low-dimensional invariant feature space of high-level feature data. The batch normalization layer normalizes each input channel across a mini-batch. In order to speed up the training of CNN and reduce the network initialization sensitivity, batch normalization layers are used between convolution layers and nonlinearities such as Rectified Linear Unit (ReLU) layers. The layer first normalizes the activations of each channel by subtracting the mini-batch mean and dividing by the standard deviation of the mini-batch. Then, the layer shifts the input by a learnable offset ‘β’ and scales it by a learnable scale factor ‘γ’.

Activation layers are implemented to verify the activation of various elements of the feature space in the presence of specific features at a given spatial location. This is accomplished by introducing nonlinearity to the layers through the use of an activation function. Various activation functions have been used in literature, such as sigmoid or hyperbolic tangent function; however, ReLUs are often used to introduce nonlinearity. A ReLU layer performs a threshold operation to each element of the input, where any value less than zero in this layer is set to zero. Mathematically, it can be expressed as follows:

$$f\left(x\right)=\left\{\begin{array}{c}0\ for\ x <0\\ x\ for\ x \ge 0\end{array}\right.$$

(6)

The objective of a fully connected layer is to take the results of the convolution process and use them to classify the images into labels. The output of a convolution is flattened into a 1D feature vector of values, each representing a probability of a certain feature belonging to a label and is fed into a fully connected layer such as a neural network. The fully connected layer multiplies the input by a weight matrix and then adds a bias vector. In this layer, softmax function is provided as an activation function to determine the probability of feature classification for the data analyzed. The softmax function is a normalized exponential function that transforms a vector of K real values into a vector of K real values that sum to 1. If the input vector of K real numbers is applied to the softmax layer, it normalizes into a probability distribution comprising of K probabilities. The input values can be positive, negative, zero, or greater than one, but the softmax function transforms them into values between 0 and 1 so that they can be interpreted as probabilities. If one of the inputs is small or negative, the softmax function turns it into a small probability, whereas if the input is large, it turns it into a large probability, but it will always remain between 0 and 1. A classification layer computes the cross-entropy loss for multiclass classification problems with mutually exclusive classes. This layer determines the number of classes from the output size of the preceding layer. This layer takes input from the softmax function and allocates each input to one of the mutually exclusive classes using the cross-entropy loss function.

Proposed methodology

The proposed methodology explores the capability of TF signal processing methods (e.g., EMD and CWT) to extract key AE components along with the feature-free classification capability of a deep learning algorithm in a unified manner. First, EMD is undertaken to eliminate the presence of noise in the measured AE data and decompose it into a set of IMFs that represent the undamaged and damaged AE signals. The time response of the AE signal is considered non-stationary, which can be decomposed using EMD due to its capability of analyzing nonlinear and non-stationary signals [15]. More details such as the systematic steps and criteria of EMD and its performance can be found elsewhere [7]. Once the IMFs belong to the crack are identified, their spectrograms can be obtained from the CWT method. Subsequently, the extracted spectrograms are fed into the 2D CNN model to automate the process of detection and localization of the damage.

Assume $x(t)$ is a measured AE signal, which is decomposed using EMD. For a given AE sensor, a number of IMFs are extracted from EMD using Eq. 7.

$$x\left(t\right)=\sum_{i=1}^{n}{H}_{i}(t)+ {v}_{n} (t)$$

(7)

CWT is then applied to each IMF obtained from EMD as expressed in Eq. 8, where ${H}_{i}$(i = 1, 2, 3, ……,$n$) is the IMF of the original signal and ${v}_{n}$ is a residue.

$${c}_{m}^{r}\left({H}_{i}\right)={\int }_{-\infty }^{\infty }{H}_{i}\left(t\right){\psi }^{*}(\frac{t-m}{r} )dt$$

(8)

The resulting CWT images of each IMF are then fed into 2D CNN to localize the damage. In this study, a modified VGG-16  [30], as depicted in Fig. 3, is used to classify the CWT images obtained from AE data. The architecture is comprised of five network blocks that consist of a varying number of convolutional layers that are finalized with a max-pooling layer. The first two network blocks consist of two convolutional layers followed by a max-pooling layer, while the three network blocks consist of three convolutional layers followed by a max-pooling layer. The number of filters of each convolutional layer from the traditional VGG-16 network is reduced by a factor of 16 as the number of classifications and complexity of the features are dramatically reduced as compared to the original network. Additionally, the number of filters for each network block increases by a factor of 2, starting with 4 filters in the first network block and ending with 32 filters in the final network block. Zero padding is introduced in the convolutional layers to ensure that the input and output layer sizes remain constant regardless of the convolutional operation. Each convolutional layer is followed by a Batch Normalization (BN) layer and ReLU to normalize and activate the extracted features. The max-pooling layer has a kernel size of 2 × 2 with a stride value of 2 to reduce the inputted size to half through the max-pooling operation. Moreover, the length of intermediary fully connected layers is reduced from 4096 to 512, with the fully connected layer having a length equivalent to the number of classes used in the study. Each fully connected layer is proceeded by a ReLU layer, and a dropout layer with a probability value set to 0.2 was implemented to reduce overfitting during the training process. The proposed method is illustrated using the flowchart, as shown in Fig. 4.

Fig. 3

The proposed modified VGG-16 architecture used in the study

Fig. 4

Flowchart of the proposed method

Numerical Illustration

The mixed-signal shown in Eq. 5 is added with random noise to develop a suite of training databases required for a 2D CNN model, as shown in Eq. 9:

$$y\left(t\right)=\left({x}_{1}+{x}_{2}+{x}_{3}\right)+ \varepsilon$$

(9)

where ε is a noise component that is a time-series sequence of a normal random variable. Then, EMD is applied to $y$(t) to decompose the signal and extract the mono-component IMF. Finally, a spectrogram of each IMF is obtained, which belongs to each frequency class. Figure 5a shows the spectrograms of the mixed-signal $y$(t), and Fig. 5b-d shows the spectrograms of IMF-1, IMF-2, and IMF-3 obtained from EMD, respectively. It can be seen that EMD is able to decompose the mixed-signal and extract the mono-component signal.

Fig. 5

(a) CWT spectrogram of $y$(t), and (b) IMF-1, (c) IMF-2, and (d) IMF-3 of y(t) obtained from EMD

In this study, 1000 images of each of the three frequency classes (i.e., a total of 3000 images) are used in the training, validation and testing process. 70% of the images are used for training, 20% are used for validation, while the rest, 10%, are used for testing. This resulted in 2100 images, 600 images and 300 images for training, validation and testing, respectively. The training is conducted over 30 epochs using a Stochastic Gradient Descent with Momentum (SGDM) solver with an initial learning rate of 0.0005, minibatch size of 128 and L₂-regularization of 0.0005. The parameters of the CNN were determined by conducting a grid search of the parameters and comparing the training accuracy across all iterations to determine which parameters resulted in the most robust and accurate model. All numerical simulations and CNN training and classifications are conducted using a Lenovo ThinkStation with NVIDIA Quadro GPU and 32 GB RAM. Figure 6 shows the variation of the accuracy and loss throughout the training and validation process. The network is trained for 30 epochs resulting in 63 minibatch iterations. It can be observed that due to the high performance of the network, the accuracy reaches 100% relatively quickly while minimizing the loss function. The trained network is tested using 100 images of each frequency class. The confusion matrix is shown in Fig. 7 (a-b), representing the classification accuracy for the validation and testing datasets, respectively. The CNN model is able to achieve 100% classification accuracy for all frequency categories.

Fig. 6

Training stage: (a) performance and (b) loss evaluation of the proposed CNN architecture using CWT TF images

Fig. 7

Confusion matrices for (a) validation and (b) testing data of the sine signal

The selected AE data acquisition system

Figure 8(a) illustrates a typical setup of the proposed AE monitoring system used in the experimental study. The AE system is used to record AE signals from a wooden beam and plate subjected to minor damage using AE sensors. Two AE sensors with a characteristic frequency band extending from 20–450 kHz are placed on the specimen, as shown in Fig. 8(b). A pre-amplification of the AE signals is added using preamplifiers with 34—40 dB gain and plug-in bandpass filters from 2.5–2400 kHz to magnify the AE signal. A decoupling box is utilized to connect the preamplifier with the data acquisition (DAQ) system to collect the AE data. It is also attached with a direct current supply to power the AE sensor. The DAQ is used to transfer the measured AE signals to the computer. It has a sampling rate of 200 Msamples/s, which is capable of dealing with high-frequency AE data collected by the sensors. In the proposed experimentation, the sampling frequency of the sensors is set to 20 kHz.

Fig. 8

(a) AE monitoring system and the experimental setup: (b) actual and (c) schematic

Damage identification in a one-dimensional structural member

In order to evaluate the performance of the proposed method, a wooden beam is considered in the experimental study. The beam has a dimension of 62 cm length, 6.5 cm width, and 2 cm thickness. Two AE sensors (A₁ and A₂) are used to collect AE data from the beam. The damage location is considered close to A₁ to check the capability of the proposed method for localizing the damage (e.g. location D). Figure 8(b-c) shows the location of sensors and damage on the beam. The damage is created by simulating a hole using a drilling machine, while the data was collected using the AE sensors.

Damage detection

In order to validate the performance of the proposed method as a damage detection tool, the proposed method is applied to the AE data of the undamaged and damaged system. Figure 9 shows the time-history of AE data collected from wooden beam using A₁ and A₂. It can be seen that the damage progression begins near t = 5 s. Therefore, the AE data collected between 0–5 s is considered as an undamaged state and data collected after 5 s is considered as a damaged state. The AE sensors have produced big data due to the higher sampling frequencies that can be used to create enough datasets to feed the CNN model. Therefore, AE data of undamaged and damaged conditions is divided into a finite number of segments. EMD is applied to each segment separately to extract the IMFs. Then, the CWT is used to generate the spectrogram of each IMF obtained from EMD. The spectrograms of IMFs have a size of 936 × 1920, which are used as the training and testing data of the 2D CNN. Figure 10 shows a typical spectrogram of IMFs using undamaged and damaged AE data collected from (a-b) A₁ and (c-d) A₂, respectively.

Fig. 9

Time-history of the measured AE data collected from sensors A₁ and A₂ of damage at D

Fig. 10

Spectrograms of IMFs using (a) undamaged, (b) damaged AE data obtained from A₁, and (c) undamaged, (d) damaged AE data from A₂ for damage at D

In this study, 195 randomly selected images of undamaged and damaged conditions collected using A₁ are used in the training process, as shown in Fig. 11. An additional 56 images and 28 images of each case are used in the validation and testing process.

Fig. 11

Training stage: (a) performance and (b) loss of the proposed method using CWT images of undamaged and damaged cases

To ensure that the training, validation and testing datasets were diverse, each dataset was represented by a 2D cluster and compared with each other to determine the diversity of the datasets. To convert each 3D image matrix to a 2D point for clustering, the RGB images were first converted to grayscale images by converting the pixel intensity using,

$$P=0.2989{P}_{R}+0.5870{P}_{G}+0.1140{P}_{B}$$

(10)

where P is the intensity of any pixel of the grayscale image represented as an integer from 0 to 255 and P_R, P_G and P_B is the intensity of any pixel of the RGB image for the red, green and blue channel, respectively. Once the images were converted, the grayscale image was converted to a 3D point by determining the centroid of the image. The centroid of the image can be considered a point in 2D image space at which all pixel intensities are balanced. To determine the width, height, and pixel intensity which represents the centroid, the following equation can be used:

$${x}_{C}=\frac{\sum {x}_{m}P}{\sum P}$$

(11)

$${y}_{C}=\frac{\sum {y}_{n}P}{\sum P}$$

(12)

$${P}_{C}=\frac{\sum P}{N}$$

(13)

where x_c is the centroid value along the width of the image in pixels, y_c is the centroid value along with the height of the image in pixels, x_m is the centroidal value of the width at pixel m. y_n is the centroidal value of the height at pixel ‘n’, P is the pixel intensity for the pixel located at the 2D location represented by the integers ‘m’ and ‘n’, and N is the number of pixels in the image. For example, at a pixel location of (m,n) = (1,1), the values of y_n and x_m would be considered 0.5 pixels, respectively, as we assumed each pixel is a unit pixel in width and height. Once all the 3D points have been determined, the centroid of each cluster is determined by the average of all values across each of the dependent variables represented in Eqs. 10–12 as represented by Fig. 12.

Fig. 12

Comparison of clusters for training and testing datasets for wooden beam

From the figure above, it can be observed that the centroids of both the training and testing datasets along the x and y-axis of the image are consistent with each other, which suggests that the images belong to the same cluster. Though these images belong to the same cluster, the spread of the images around the centroid point of the datasets suggests that the datasets are sufficiently diverse and that though the testing dataset has similar features to the training dataset, they do not have significantly similar features that would result in overfitting.

The resulting confusion matrix is shown in Fig. 13(a-b), displaying the classification accuracy for the validation and testing datasets extracted from AE data using A₁. The same process is used to detect the damage using AE data collected from A₂. Figure 13(c-d) shows the confusion matrices that display the classification accuracy for the validation and testing datasets obtained from AE data collected using A₂. It can be observed that the CNN model is able to achieve ~ 100% classification accuracy for damaged and undamaged categories. Therefore, the proposed method can be considered a reliable tool for damage detection in structural elements.

Fig. 13

Confusion matrices (a-b) for validation and testing using damaged and undamaged data from A₁ and (c-d) for validation and testing using damaged and undamaged data from A₂

Damage localization

Figure 14 shows the time-history of measured AE data of a typical segment for damage at D. It can be observed that the amplitude of undamaged and damaged signals significantly overlap. Subsequently, the raw amplitude of the time-history of AE data will provide faulty information for the damage location, which motivates the need for the proposed method to automate the process of identifying the potential location of the damage. The spectrogram of IMFs of each AE segment is extracted from CWT. These spectrograms are then used to feed the CNN model to identify the location of damage in the structural element. The proposed method is applied to identify the approximate location of damage in structural elements using a limited number of AE sensors (e.g., two sensors). The damage location is identified by using the term damage-close or far with respect to the sensor location. In order to identify the potential location of damage at D, spectrograms obtained from A₁ are treated as damage-close, whereas spectrograms from A₂ are treated as damage-far.

Fig. 14

Time history of AE data of a typical segment for damage at D

In this study, 473 randomly selected images of each sensor class are used in the training process. An additional 135 images and 67 images of each sensor class are used in the validation and testing process. The training process is shown in Fig. 15. The confusion matrix shown in Fig. 16 (a-b) shows the classification accuracy for the validation and testing datasets. The accuracy, recall, precision and F1 scores of the validation dataset and testing dataset are summarized in Table 1. The performance indicators calculated from the confusion matrices suggest an excellent generalization of the CWT images by the proposed algorithm.

Fig. 15

Training stage: (a) performance and (b) loss of the proposed method using the experimental data

Fig. 16

Confusion matrices for (a) validation and (b) testing data of the wooden beam

Table 1 Performance indicators calculated from the classification of CWT images of close (A₁) and far damage (A₂)

Damage identification in a two-dimensional structural member

In this section, an experimental study using a 2D structural element such as a wooden plate is undertaken to validate the performance of the proposed method. The dimensions of the plate are 1 m length, 1 m width, and 2 cm thickness, respectively, as shown in Fig. 17. Two sensors (A₁ and A₂) are used to measure the AE signal while the plate is subjected to active damage. The plate is subjected to damage at location D. Figure 17 shows the proposed location of sensors and damage on the plate. A drilling machine is used to simulate the damage by creating a hole, while the signal is measured using the selected AE sensors.

Fig. 17

Experimental setup of the plate

Damage detection

In this study, the proposed method is applied to undamaged and damaged AE signals to classify the undamaged and damaged cases of the wooden plate. Figure 18 represents the time-history of AE data collected from the wooden plate using A₁ and A₂. It can be observed that the damage progression begins near t = 1 s and ends at t = 19 s. Therefore, the AE data collected between 0–1 s is considered as undamaged and between 1–19 s is considered as damaged data. Then, the CWT is used to generate the spectrogram of each IMF obtained from EMD. The spectrograms are used as the training and testing data of the 2D CNN. Figure 19 shows a typical CWT spectrogram of IMFs using undamaged and damaged AE data collected from (a-b) A₁ and (c-d) A₂.

Fig. 18

Time history of the measured AE data collected using sensors A₁ and A₂ of damage at D

Fig. 19

Spectrograms of IMFs using (a) undamaged, (b) damaged AE data obtained from sensor A₁, and (c) undamaged, and (d) damaged AE data from sensor A₂ for damage at D

In order to verify the capability of the 2D CNN model for classifying the undamaged and damaged state of the wooden plate, 140 images of undamaged and damaged conditions collected using A₁ are used in the training process. An additional 40 images and 20 images of each case are used in the validation and testing process. As per the process described in Sect. 5.1, the centroid of each image was determined the clusters for the training and testing datasets were compared as shown in Fig. 20. Similarly, though the centroids of both the training and testing images across the entire dataset are similar, the spread of the points around the centroid is relatively high, suggesting there is good robustness within the dataset. Furthermore, though both the beam and plate are made of similar material and are damaged in a similar manner, each damage scenario generates unique clusters, suggesting geometric properties of the element and relative damage location have significant influence with regards to the AE signals generated.

Fig. 20

Comparison of clusters for training and testing datasets of the wooden plate

Figure 21 shows the performance of the proposed method for identifying the undamaged and damaged states of the plate. The resulting confusion matrix is shown in Fig. 22 (a-b), displaying the classification accuracy for the validation and testing datasets extracted from AE data using A₁. The same process is executed to detect the damage using AE data collected from A₂. Figure 22 (c-d) shows the confusion matrices that display the classification accuracy for the validation and testing datasets obtained from AE data collected using A₂. Table 2 summarizes the accuracy, recall, precision and F1 scores of the validation dataset and testing dataset of the undamaged and damaged condition of the plate. It can be observed that the CNN model is able to achieve ~ 100% classification accuracy for damaged and undamaged cases. Therefore, the proposed method shows excellent capabilities to perform damage detection in a structural element such as a plate.

Fig. 21

Training stage: (a) performance and (b) loss of the proposed method using CWT images of undamaged and damaged conditions of the wooden plate

Fig. 22

Confusion matrices (a-b) for validation and testing using damaged and undamaged data from A₁ and (c-d) from A₂

Table 2 Performance indicators calculated from the classification of CWT images of undamaged and damaged conditions

Damage localization

In this section, the proposed method is extended to localize the damage using limited AE sensors. In order to identify the potential location of damage, the spectrograms of IMFs of AE signal obtained from A₁ and A₂ (defined as damage-close and damage-far, respectively) are used to feed into the 2D CNN. In this study, 140 randomly selected images of each sensor class are used in the training process. An additional 40 images and 20 images of each sensor class are used in the validation and testing process. The training process is shown in Fig. 23. The confusion matrix shown in Fig. 24(a-b) displays the classification accuracy for the validation and testing datasets. The accuracy, recall, precision and F1 scores of the validation dataset and testing dataset are summarized in Table 3. The performance indicators calculated from the confusion matrices suggest an excellent generalization of the CWT images by the proposed algorithm.

Fig. 23

Training stage: (a) performance and (b) loss of the proposed method using the experimental data

Fig. 24

Confusion matrices for (a) validation and (b) testing data of the wooden plate for damage at D₁

Table 3 Performance indicators calculated from the classification of CWT images of close and far damage

Crack identification in a concrete beam

In this section, the performance of the proposed method is validated experimentally using a concrete beam. The beam size is 355 mm in length and 78 mm in width and thickness. Ordinary Portland cement, fine aggregate, and coarse aggregate are used in the concrete mixture. MTS testing machine is used to carry out a three-point flexural loading test. Two AE sensors (A₁ and A₂) are used to collect AE data from the beam. During the experiment, the crack opening is visually monitored, where the crack initiates at the center bottom of the beam and then propagates to the top. Figure 25 (a) shows the location of sensors and crack on the concrete beam. AE signal is collected by a Micro-SHM acquisition system that has four input channels. The Micro-SHM acquisition system is used to transfer the measured AE waveforms from sensors to the computer using AE_win software, as shown in Fig. 25 (b). The system has a sampling rate of 10 Msamples/s, which can deal with high-frequency AE signals collected by the sensors due to the micro-cracks in concrete material.

Fig. 25

(a) Experimental setup of the concrete beam and (b) AE monitoring system

Crack localization

In this study, the AE waveforms collected from the beam using two AE sensors are firstly decomposed to n number mono-components signals (IMFs) using the EMD method. Figure 26 represents the time-history response of a typical AE waveform and its first five IMFs obtained from EMD using sensors A₁ (first column) and A₂ (second column). It can be seen that the amplitude of the AE waveform obtained from A1 and A2 are the same. Thus, the amplitude of the time-history of raw AE data might provide inaccurate information for the crack location, which motivates to apply the proposed method to automate the process of identifying the potential location of the crack in the concrete beam. In order to predict the approximate location of the crack, the CWT spectrograms of IMFs extracted from A₁ and A₂ (defined as damage-far and damage-close, respectively) are used to feed into the 2D CNN. Figure 27 shows the CWT spectrograms of the first five IMFs of typical AE waveform obtained from A₁ (considered as crack-far) and A₂ (considered as crack-close).

Fig. 26

Time history of the measured AE waveforms and first five IMFs extracted from sensors A1 and A2

Fig. 27

Spectrograms of IMFs using AE waveform obtained from sensor A₁ and sensor A₂

In this study, 210 randomly selected images of each sensor class are used in the training process. An additional 60 images and 30 images of each sensor class are used in the validation and testing process. The centroidal clustering, as demonstrated in Fig. 28, demonstrates that similar to wooden beam and plate, the concrete beam has a robust dataset that has unique clustering characteristics that are indicative of the uniqueness of the damage caused by the three-point bending test.

Fig. 28

Comparison of clusters for the training and testing datasets of the concrete beam

The training process is shown in Fig. 29. The confusion matrix shown in Fig. 30 (a-b) displays the classification accuracy for the validation and testing datasets. The accuracy, recall, precision and F1 scores of the validation dataset and testing dataset are summarized in Table 4. The performance indicators calculated from the confusion matrices suggest an excellent generalization of the CWT images by the proposed algorithm.

Fig. 29

Training stage: (a) performance and (b) loss of the proposed method using the experimental data

Fig. 30

Confusion matrices for (a) validation and (b) testing data of the concrete beam under three-point flexural loading

Table 4 Performance indicators calculated from the classification of CWT images of close and far damage

Conclusions

In this paper, a 2D CNN framework is proposed to automate the process of detecting and localizing the damage using AE data collected from structures. EMD is used to suppress the presence of noise and extract the key AE components, followed by CWT to generate the spectrograms of the extracted IMFs. Then, the resulting noise-free spectrograms are utilized to feed the CNN model for improved classification accuracy. CNN model is employed to automate the process of identifying the potential location of damage using the AE data. A set of numerical and experimental studies are conducted to validate the performance of the proposed approach as a damage detection tool using a limited number of AE sensors.

The results show the capability of the proposed approach to identify the approximate location of the damage in various structural elements. For example, the extracted results obtained from the wooden beam showed the high performance of the proposed approach in identifying the potential location of damage with 93% accuracy. Also, the proposed technique can identify the approximate location of damage in a 2D wooden plate with an accuracy of 88%. Moreover, the results obtained from the experimental study using concrete beam represented the capability of this approach in identifying the crack location where the accuracy reached 95%. Future studies are reserved for validating the proposed method in a wide range of full-scale structures and identifying the location of multiple damages.