Application of wearable EEG sensors for monitoring mental fatigue state of construction workers

Introduction

Many studies have shown that workers' unsafe behaviours are the main cause ofconstruction accidents, which directly result in most of the typical accidents on construction sites, such as falls from height, slips, trips, being struck by moving machinery or being trapped by objects. These unsafe behaviours can be attributed to a variety of causes, such as repetitive and boring behaviours, inappropriate safety climate and incorrect estimation of potential risks. Based on the potential risks commonly found in construction sites, CITF has set up the Smart Safe Site System (4S) labelling scheme. Smart Construction Laboratory Ltd. has been involved in several studies aimed at eliminating inherent hazards by monitoring the mental state of workers, which will enrich the functionality of the 4S system and reduce the risk of construction site safety accidents on all fronts.

Mental fatigue is the result of prolonged engagement of the brain in intellectually demanding tasks, which may lead to a decline in behavioural and cognitive performance. However, there are various ways to quantify human mental state. Data from several studies on measuring the level of mental fatigue in workers have been derived from questionnaires. However, such assessments are subjective and therefore lack accuracy. In recent years, wearable sensors have attracted significant attention from researchers. Electroencephalography (EEG) data hasbeen employed by researchers at SCLab to more accurately quantify the level of mental fatigue of construction equipment operators. In this blog, four methods developed by SCLab for quantitative assessment of mental fatigue inconstruction workers based on EEG data will be presented, and their feasibilityon construction sites will be verified.

1. Pre-service fatigue screening for construction workers through wearable EEG-based signal spectral analysis

In this research, SCLab uses a regression model based on gravity frequency and power spectrum entropy to process EEG data. The gravity frequency can reflect the migration of the centre of gravity of the EEG power spectrum, while the power spectrum entropy can characterise the degree of disorder of the temporal signaland the degree of confusion of the multi-frequency components. The questionnaire method was also used to gather basic information about the experimental subjects. Overall, this method provides a mixed subjective (MFL) and objective assessment (MFV) of mental fatigue levels. The MFL allows fordirect screening of disqualified workers, while the MFV is used to demonstrate changes in workers' mental fatigue levels. It was demonstrated that the method was effective in screening out unqualified individuals with high levels ofmental fatigue and screened individuals with significantly longer response times and higher response latencies. The disadvantage of this method is that the procedure is complex and not adapted to the efficiency-seeking and task-complex construction site environment.

2. Detecting and measuring construction workers' vigilance through hybrid kinematic-EEG signals

The model adopted in Method 1 above requires a strictly stationary environment for EEG deployment and treats signals triggered by muscle movements as "artefacts" or "noise". Many research have shown that the signals collected by wearable EEG devices are not "pure" EEG signals, but are mixed not only with extrinsic noise but also with intrinsic artefacts. These noises and artefacts are mainly caused by the large amount of movement ofthe worker and different environmental factors, such as electrode pops, blinking, environmental noise, etc. Therefore, although Method 1 has been validated in the laboratory with high accuracy, it may not be suitable for construction implementation. To fill this gap, SCLab was involved in a research to treat such signals as novel hybrid motor EEG signals using wavelet packettransform (WPT).

Unlike in a laboratory setting, the "artefacts" captured by wearable EEG devices reflect the subject's muscle movements and are used as meaningful information during construction activities. In this research, the hybrid kinematic EEG signal was decomposed into five effective sub bands in the frequency domain. With the help of these sub bands, thirty alertness indices were monitored in a field experiment, and appropriate indices highly relevantto the benchmarking of non-stationary EEG applications in the construction industry were selected. The results of this research establish a quantitative measure of the level of vigilance for construction safety management and provide a new perspective for understanding the risk perception of construction personnel during the execution of construction tasks.

3. Deep learning-based construction equipment operators’ mental fatigue classification using wearable EEG sensor data

With the previous two methods, deep learning was introduced by SCLab to process more complex brainwave data in order to enhance the feasibility of EEG in real construction projects. Similar to Method 1, a questionnaire based on the NASA-TLX scale was used as a ground truth for mental fatigue and wearable EEG sensors was used to record brain activity patterns. Method 2 was used to process complex mixed motor EEG signals.

Three deep learning models were employed and compared in this method, namely long short-term memory, bidirectional long short-term memory and one-dimensional convolutional networks, to analyse the raw EEG data acquired by the wearable sensor. For this purpose, fifteen operators performed an hour-long excavatoroperation at a construction site. The research results show that the Bi-LSTM model out performs other deep learning models with an accuracy of 99.941%, while the F1 scores range from 99.917% to 99.993%. The research results demonstrated the feasibility of the Bi-LSTM model.

4. Multimodal integration for data-driven classification of mental fatigue during construction equipment operations: Incorporating electroencephalography, electrodermal activity, and video signals

Mental fatigue is a multimodal problem, and brainwave data alone may not provide acompletely accurate picture of the level of mental fatigue in equipment operators. In this research, EEG, galvanic skin activity, and video signals were each passed through a machine learning model and were used in conjunction to classify the state of mental fatigue. Researchers at SCLab collected EEG, galvanic skin activity, and video signals from excavator operators, and adecision tree model using multimodal sensor data fusion demonstrated the model's accuracy and feasibility with an accuracy rate of 96.2% and an F1 score of 96.175% and an F1 score of 96.175%. -98.231% F1 score to demonstrate the accuracy and feasibility of the model.