Public defence of doctoral thesis in electronics with Thanh Tran

Main supervisor: Associate Professor Jan Lundgren

Co supervisor: Associate Professor Sebastian Bader

Opponent: Associate Professor Domenico Capriglione, University of Cassino and Southern Lazio, Italy

Grading committee:

Associate Professor Magnus Oskarsson, Lund University

Dr. Sepideh Pashami, Halmstad University

Associate Professor Daniel Jung, Linköping University

Join via Zoom

Abstract

The detection of damage or abnormal behavior in machines is critical in industry, as it allows faulty components to be detected and repaired as early as possible, reducing downtime and minimizing operating and personnel costs. However, manual detection of machine fault sounds is economically inefficient and labor-intensive. While prior research has identified various methods to detect failures in drill machines using vibration or sound signals, there remain significant challenges. Most previous research in this field has used manual feature extraction and selection, which can be tedious and biased. Recent studies have used LSTM, end-to-end 1D CNN, and 2D CNN as classifiers, but these have limited accuracy for machine failure detection. Additionally, machine failure is rare in the data, and sounds in the real-world dataset have complex waveforms that are a combination of noise and sound.

To address these challenges, this thesis proposes modern artificial intelligence methods for the detection of drill failures using image representations of sound signals (Mel spectrograms and log-Mel spectrograms) and 2-D convolutional neural networks (2D-CNN) for feature extraction. The proposed models use conventional machine learning classifiers (KNN, SVM, and linear discriminant) or a recurrent neural network (long short-term memory) to classify three classes in the dataset (anomalous sounds, normal sounds, and irrelevant sounds). For using conventional machine learning methods as classifiers, pre-trained VGG19 is used to extract features, and neighborhood component analysis (NCA) is used for feature selection. For using LSTM, a small 2D-CNN is proposed to extract features, and an attention layer after LSTM focuses on the anomaly of the sound when the drill changes from normal to the broken state. The findings allow for better anomaly detection in drill machines and the development of a more cost-effective system that can be applied to a small dataset.

Additionally, I also present a case study that advocates for the use of deep learning-based machine failure detection systems. We focus on a small drill sound dataset from Valmet AB, a company that supplies equipment and processes for biofuel production. The dataset consists of 134 sounds that have been categorized as "Anomaly" and "Normal" recorded from a drilling machine. However, using deep learning models for detecting failure drills on such a small sound dataset is typically unsuccessful. To address this problem, we propose using a variational autoencoder (VAE) to augment the small dataset. We generated new sounds by synthesizing them from the original sounds in the dataset using the VAE. The augmented dataset was then pre-processed using a high-pass filter with a passband frequency of 1000 Hz and a low-pass filter with a passband frequency of 22,000 Hz, before being transformed into Mel spectrograms. We trained a pre-trained 2D-CNN Alexnet using these Mel spectrograms. We found that using the augmented dataset enhanced the classification results of the CNN model by 6.62% compared to using the original dataset (94.12% when trained on the augmented dataset versus 87.5% when trained on the original dataset). Our study demonstrates the effectiveness of using a VAE to augment a small sound dataset for training deep learning models for machine failure detection.

Background noise and acoustic noise in sounds can affect the accuracy of the classification system. To improve the sound classification application's accuracy, a sound separation method using short-time Fourier transform (STFT) frames with overlapped content is proposed. Unlike traditional STFT conversion, in which every sound is converted into one image, the signal is split into many STFT frames, improving the accuracy of model prediction by increasing the variability of the data. Images of these frames are separated into clean and noisy ones and subsequently fed into a pre-trained CNN for classification, making the classifier robust to noise. The efficiency of the proposed method is demonstrated using the FSDNoisy18k dataset, where 94.14 percent of 21 classes were classified successfully, including 20 classes of sound events and a noisy class.

Link to the thesis in DIVA