Mazhar Hussain presenterade sitt arbete vid ett halvtidsseminarium

Multi-Sensor data fusion for improved estimation and prediction of physical quantities

In recent years, there has been a significant increase in multi-sensor data across diverse fields, spanning from environmental monitoring and industrial automation to smart agriculture, surveillance systems, healthcare analytics, robotics, remote sensing, smart cities, and beyond. The fundamental drive behind leveraging multimodal data is the amalgamation of complementary information extracted from various sensors, facilitating more comprehensive insights and informed decision-making compared to reliance on a single modality.

The analysis of multi-sensor data presents substantial challenges due to its vastness and the presence of structured, semi-structured, and unstructured data, spanning different modalities with distinct sources, types, and distributions. Data fusion, the integration of information from diverse modalities, becomes crucial in addressing inference problems arising from multi-sensor data. Both analytics-based and learning-based data fusion approaches are widely used, with learning-based approaches, leveraging machine learning and deep learning methods, showing notable effectiveness.

However, the question of "where" and "how" to fuse different modalities remains an open challenge. To explore this, our study focused on three applications as case studies to employ data fusion approaches for estimating and predicting physical quantities. These applications include analysing the correlation between the change in the geometrical dimension of a free-falling molten glass gob and its viscosity using Pearson correlation coefficient as analytical method, predicting fuel consumption of city buses through machine learning methods, and classifying and measuring hazardous gases i.e.  H2S and CH3SH using deep learning methods.

Results from these case studies indicate that the choice between analytical, machine learning, or deep learning-based data fusion depends on the specific application, as well as the size and quality of the data. Despite this, advancements in computing power and deep learning technology have made data more accessible and have enhanced its complementarity. Looking ahead, we plan to conduct a comprehensive review to compare a range of deep learning-based data fusion strategies. This review will include an examination of various feature extraction methods, as well as an outline and identification of the research fields that stand to derive the greatest benefits from these evolving approaches.