Computer vision is an interdisciplinary field of science that focuses on how computers can generate information to improve decision-making by capturing, processing, and analysing digital images and the data collected from the analysis. By providing digital images and videos about the current environment of a project, computer vision technology can help practitioners better manage the construction process. Computer visionis currently being used to examine specific issues in construction, such as tracking people's movements, progress monitoring, productivity analysis, health and safety monitoring, and postural ergonomics assessment.
While computer vision has made considerable progress in construction applications in areas such as safety monitoring, productivity analysis, and identifying structural defects, several technical challenges have hindered its development. Smart Construction Laboratory Ltd. observed in a study it participated in 2019 that the application and diffusion of computer vision technology in smart construction sites is limited by three main factors. Firstly, there is a lack of databases of sufficient size to ensure the accuracy of computer vision. Secondly, computer vision research is greatly limited by existing data privacy protection regulations. Finally, the poor quality of collected data and analysing datawith only shallow learning methods have emerged as two of the major technical challenges in computer vision research. In this blog, relevant research will belisted to illustrate how SCLab has been involved in studies related to computer vision and given smart safe worksite solutions.
1. Computer Vision and Deep Algorithmic Learning
Deep learning is the study of the intrinsic patterns and levels of representation of sample data, and the information gained in the process can help in the interpretation of the data.The ultimate goal of deep learning is to enable machines to be as analytically capable of learning as humans, which means that machines are able to derive deeper information from the data they collect and help the user make decisions. Whilst a range of techniques have been applied to automatically identify hazards on construction sites in several current studies and practices, a fully automated system based on computer vision has not yet been developed.
SCLab, in a series of studies in which it has been involved, proposed a semantic and a priori knowledge-based localisation method to track the location of construction site entities through the combination of deep learning and computer vision. Firstly, deep learning models and computer vision were used to identify and segment construction-related entities on surveillance video frames. Then, different types of scenes with a priori knowledge were combined to develop projection strategies based on different scenes. The results demonstrated that the methodachieves satisfactory localisation accuracy, is robust under low-scale occlusion, and contributes to security alerts, activity recognition and productivity analysis.
SCLab researchers have also developed aconstruction safety management framework that combines computer vision algorithms and formal ontology modelling, which has been specifically appliedto the prevention of fall-at-height accidents, which are common during construction. Specifically, computer vision was used to detect visual information from site photographs, while deep learning technology based on safetyregulatory knowledge was formally represented through ontologies and SemanticWeb Rule Language (SWRL). By comparing visual information extracted from construction images with predefined SWRL rules, hazards and corresponding mitigation measures can be inferred. Finally, the framework dedicated to fall prevention is applied to a construction project of the Wuhan Transportation Railway System to validate the theoretical and technical feasibility of the developed conceptual framework. The results show that the proposed framework is similar to the safety manager's thinking model and can facilitate on-site hazard identification and prevention by semantically inferring hazards from images and listing the corresponding mitigation measures.
Computer vision can be applied to No.5 Unsafe acts/dangerous situation alert for Mobile Plant Operation Area in 4S Solutions. SCLab developed a 2D target detection, 3D bounding box reconstruction and depth estimation based on monocular 2D vision in a A CVB method based on 2D target detection in monocular 2D vision, 3D bounding box reconstruction, and depth estimation for 3D spatial proximity and crowding estimation was developed to help AI cameras to more accurately identify the spatial relationship between workers and heavy vehicles so as to avoid collision accidents. The CVB method proposed by SCLab addresses two main problems:
(a) distortion of 3D spatial distance information due to 2D projection of 3D objects; and (b) failure of proximity estimation between workers and objects.
Tests of the method show good performance: the precision and recall of the 3D bounding box reconstruction method are 1 and 0.816, respectively; and the mean absolute error of the proximity estimation is less than 0.8 m.
Construction workers are often exposed to ergonomic risks due to improper posture and/or excessive manual handling of materials. Ergonomics-based interventions and aids usually require construction workers to wear a lot of equipment, which conversely adds hindrances to the lifting work. Several studies in which SCLab has been involved have addressed the limitations of the existing technologies by proposing a method for calculating joint-level ergonomic workloads for construction workers based on computer vision and smart insoles. Firstly, smart phone cameras and advanced deep learning algorithms were used to extract skeletal data from construction workers. Secondly, smart insoles were used to quantify the plantar pressures on the worker while performing construction activities. Finally, the collected data was fed into an inverse dynamic modelto calculate joint torques and workloads. The above methodology was tested through experiments including material handling, plastering and rebar simulation. The experimental results showed that the method can:
1) accuratelycollect postural and external load data from construction workers, 2)automatically provide workload assessment without being invasive, and 3) workwell in both indoor and outdoor environments.
Overall, this study contributes to the understanding of occupational safety and health inconstruction management by providing a new method for assessing risk factorsfor work-related musculo skeletal disorders (WMSD).
In addition, SCLab is involved in research on a robot that can perform daily inspections at construction sites and automatically retrieve nails and screws. Construction workers are often at risk of stepping on nails or screws, which can lead to serious infections such as tetanus. Injuries from construction materials may result in workers having to take time off work, pay medical bills, become disabled and break bones. Recycling nails and screws can therefore reduce the risk of injury on construction sites and save money. Much of the removal and recycling of construction waste on site is still carried out manually, which is inefficient and costly. However, recovering nails and screws dropped on construction sites is a challenge because of their small size and the complexity of the site terrain. SCLab has been involved in the design of aprototype nail and screw recovery robot based on computer vision technology andfull-coverage path planning (CCPP) algorithms. The prototype consists of atracked chassis, a four-degree-of-freedom clawed robotic arm, an inspectionsystem with a camera and an omni-directional laser scanner, a multi-cell storage box and a control system. The robot uses computer vision to identify nails and screws and place them into designated storage bins and is expected to search the entire workspace without prior knowledge of obstacles and waste locations, with the work path determined solely by information provided by thelaser scanner. Dedicated computer vision datasets for Faster R-CNN training were created in this study to enable the control system to convert environmental information into neural activity and to guarantee time-efficient coverage.
· Established in 2023
· Derived from the 20-year history of the PolyU Smart Construction Laboratory, HK
· Landlink's First-Generation Prouct
