An approach to combine progressively captured point clouds for BIM update

Building information models (BIMs) provide opportunities to serve as an information repository to store and deliver as-built information. Since a building is not always constructed exactly as the design information specifies, there will be discrepancies between a BIM created in the design phase (called as-designed BIM) and the as-built conditions. Point clouds captured by laser scans can be used as a reference to update an as-designed BIM into an as-built BIM (i.e., the BIM that captures the as-built information). Occlusions and construction progress prevent a laser scan performed at a single point in time to capture a complete view of building components. Progressively scanning a building during the construction phase and combining the progressively captured point cloud data together can provide the geometric information missing in the point cloud data captured previously. However, combining all point cloud data will result in large file sizes and might not always guarantee additional building component information. This paper provides the details of an approach developed to help engineers decide on which progressively captured point cloud data to combine in order to get more geometric information and eliminate large file sizes due to redundant point clouds.     

Formalized knowledge of construction sequencing for visual monitoring of work-in-progress via incomplete point clouds and low-LoD 4D BIMs

Over the last few years, new methods that detect construction progress deviations by comparing laser scanning or image-based point clouds with 4D BIM are developed. To create complete as-built models, these methods require the visual sensors to have proper line-of-sight and field-of-view to building elements. For reporting progress deviations, they also require Building Information Modeling (BIM) and schedule Work-Breakdown-Structure (WBS) with high Level of Development (LoD). While certain logics behind sequences of construction activities can augment 4D BIM with lower LoDs to support making inferences about states of progress under limited visibility, their application in visual monitoring systems has not been explored. To address these limitations, this paper formalizes an ontology that models construction sequencing rationale such as physical relationships among components. It also presents a classification mechanism that integrates this ontology with BIM to infer states of progress for partially and fully occluded components. The ontology and classification mechanism are validated using a Charrette test and by presenting their application together with BIM and as-built data on real-world projects. The results demonstrate the effectiveness and generality of the proposed ontology. It also illustrates how the classification mechanism augments 4D BIM at lower LoDs and WBS to enable visual progress assessment for partially and fully occluded BIM elements and provide detailed operational-level progress information.     

基于 AISI 网络的 BIM 三维重建方法研究

自动从点云数据生成建筑信息模型(BIM)一直是建筑自动化领域的研究热点。基于 传统算法的建筑自动三维重建的缺点包括人工设计特征,识别过程复杂,应用场景有限等。随 着三维机器学习领域的不断成熟,处理点云便有了新的手段。通过引入实例分割中的 ASIS 网 络框架对点云进行处理,即从扫描点云场景中自动分割和分类建筑构建元素并得到实例分割矩 阵。接着,基于包围盒假设从得到的实例分割矩阵中提取建筑构件外轮廓参数,并将外轮廓参 数和分割的语义分类结果作为 BIM 建模的构件参数。最后,将这些提取的构件参数输入到自制 的 IFC 生成器中,自动生成基于工业基础类(IFC)标准的 BIM 模型。实验表明,利用无噪点点 云方法,可实现基于曼哈顿世界假设下的室内单房间的三维重建。     

基于BIM的钢筋骨架语义设计点云自动生成算法

当前预制构配件钢筋骨架质量检查主要依靠人工,存在效率低、容易出错的问题。建筑信息模 型(BIM)、三维重建等技术为改进预制构配件钢筋骨架质量检查方法提供可能。运用这些技术时,有必要由钢 筋骨架 BIM 模型生成可区分每根钢筋的点云。为此,提出了语义设计点云的概念,并构建了基于 BIM 的钢筋 骨架语义设计点云自动生成算法。该算法首先从钢筋骨架 BIM 模型中提取每根钢筋并分别存储于不同的文件, 然后对每根钢筋所在文件进行格式转换,接着生成每根钢筋的语义设计点云,最后基于每根钢筋的语义设计点 云生成钢筋骨架语义设计点云。分别用简单钢筋骨架和复杂钢筋骨架对基于 BIM 的钢筋骨架语义设计点云自 动生成算法进行实验验证,结果表明,该算法能够自动并快速地生成准确的钢筋骨架语义设计点云。      

Towards automatic generation of as-built BIM: 3D building facade modeling and material recognition from images

As-built building information model(BIM) is an urgent need of the architecture, engineering, construction and facilities management(AEC/FM) community. However, its creation procedure is still labor-intensive and far from maturity. Taking advantage of prevalence of digital cameras and the development of advanced computer vision technology, the paper proposes to reconstruct a building facade and recognize its surface materials from images taken from various points of view. These can serve as initial steps towards automatic generation of as-built BIM. Specifically, 3D point clouds are generated from multiple images using structure from motion method and then segmented into planar components, which are further recognized as different structural components through knowledge based reasoning. Windows are detected through a multilayered complementary strategy by combining detection results from every semantic layer. A novel machine learning based 3D material recognition strategy is presented. Binary classifiers are trained through support vector machines. Material type at a given 3D location is predicted by all its corresponding 2D feature points.Experimental results from three existing buildings validate the proposed system.     

Plane-based registration of construction laser scans with 3D/4D building models

With the development of building information modelling (BIM) and terrestrial laser scanning (TLS) in the architecture, engineering, construction and facility management (AEC/FM) industry, the registration of site laser scans and project 3D (BIM) models in a common coordinate system is becoming critical to effective project control. The co-registration of 3D datasets is normally performed in two steps: coarse registration followed by fine registration. Focusing on the coarse registration, model-scan registration has been well investigated in the past, but it is shown in this article that the context of the AEC/FM industry presents specific (1) constraints that make fully-automated registration very complex and often ill-posed, and (2) advantages that can be leveraged to develop simpler yet effective registration methods.This paper thus presents a novel semi-automated plane-based registration system for coarse registration of laser scanned 3D point clouds with project 3D models in the context of the AEC/FM industry. The system is based on the extraction of planes from the laser scanned point cloud and project 3D/4D model. Planes are automatically extracted from the 3D/4D model. For the point cloud data, two methods are investigated. The first one is fully automated, and the second is a semi-automated but effective one-click RANSAC-supported extraction method. In both cases, planes are then manually but intuitively matched by the user. Experiments, which compare the proposed system to software packages commonly used in the AEC/FM industry, demonstrate that at least as good registration quality can be achieved by the proposed system, in a simpler and faster way. It is concluded that, in the AEC/FM context, the proposed plane-based registration system is a compelling alternative to standard point-based registration techniques.     

Textured mesh surface reconstruction of large buildings with multi-view stereo

There are three main approaches for reconstructing 3D models of buildings. Laser scanning is accurate but expensive and limited by the laser’s range. Structure-from-motion (SfM) and multi-view stereo (MVS) recover 3D point clouds from multiple views of a building. MVS methods, especially patch-based MVS, can achieve higher density than do SfM methods. Sophisticated algorithms need to be applied to the point clouds to construct mesh surfaces. The recovered point clouds can be sparse in areas that lack features for accurate reconstruction, making recovery of complete surfaces difficult. Moreover, segmentation of the building’s surfaces from surrounding surfaces almost always requires some form of manual inputs, diminishing the ease of practical application of automatic 3D reconstruction algorithms. This paper presents an alternative approach for reconstructing textured mesh surfaces from point cloud recovered by patch-based MVS method. To a good first approximation, a building’s surfaces can be modeled by planes or curve surfaces which are fitted to the point cloud. 3D points are resampled on the fitted surfaces in an orderly pattern, whose colors are obtained from the input images. This approach is simple, inexpensive, and effective for reconstructing textured mesh surfaces of large buildings. Test results show that the reconstructed 3D models are sufficiently accurate and realistic for 3D visualization in various applications.     

HDRNet: High-Dimensional Regression Network for Point Cloud Registration

Abstract-3D point cloud registration is a crucial topic in the reverse engineering, computer vision and robotics fields. The core of this problem is to estimate a transformation matrix for aligning the source point cloud with a target point cloud. Several learning-based methods have achieved a high performance. However, they are challenged with both partial overlap point clouds and multiscale point clouds, since they use the singular value decomposition (SVD) to find the rotation matrix without fully considering the scale information. Furthermore, previous networks cannot effectively handle the point clouds having large initial rotation angles, which is a common practical case. To address these problems, this paper presents a learning-based point cloud registration network, namely HDRNet, which consists of four stages: local feature extraction, correspondence matrix estimation, feature embedding and fusion and parametric regression. HDRNet is robust to noise and large rotation angles, and can effectively handle the partial overlap and multi-scale point clouds registration. The proposed model is trained on the ModelNet40 dataset, and compared with ICP, SICP, FGR and recent learning-based methods (PCRNet, IDAM, RGMNet and GMCNet) under several settings, including its performance on moving to invisible objects, with higher success rates. To verify the effectiveness and generality of our model, we also further tested our model on the Stanford 3D scanning repository.     

Robust normal estimation and region growing segmentation of infrastructure 3D point cloud models

Modern remote sensing technologies such as three-dimensional (3D) laser scanners and image-based 3D scene reconstruction are in increasing demand for applications in civil infrastructure design, maintenance, operation, and as-built construction verification. The complex nature of the 3D point clouds these technologies generate, as well as the often massive scale of the 3D data, make it inefficient and time consuming to manually analyze and manipulate point clouds, and highlights the need for automated analysis techniques. This paper presents one such technique, a new region growing algorithm for the automated segmentation of both planar and non-planar surfaces in point clouds. A core component of the algorithm is a new point normal estimation method, an essential task for many point cloud processing algorithms. The newly developed estimation method utilizes robust multivariate statistical outlier analysis for reliable normal estimation in complex 3D models, considering that these models often contain regions of varying surface roughness, a mixture of high curvature and low curvature regions, and sharp features. An adaptation of Mahalanobis distance, in which the mean vector and covariance matrix are derived from a high-breakdown multivariate location and scale estimator called Deterministic MM-estimator (DetMM) is used to find and discard outlier points prior to estimating the best local tangent plane around any point in a cloud. This approach is capable of more accurately estimating point normals located in highly curved regions or near sharp features. Thereafter, the estimated point normals serve a region growing segmentation algorithm that only requires a single input parameter, an improvement over existing methods which typically require two control parameters. The reliability and robustness of the normal estimation subroutine was compared against well-known normal estimation methods including the Minimum Volume Ellipsoid (MVE) and Minimum Covariance Determinant (MCD) estimators, along with Maximum Likelihood Sample Consensus (MLESAC). The overall region growing segmentation algorithm was then experimentally validated on several challenging 3D point clouds of real-world infrastructure systems. The results indicate that the developed approach performs more accurately and robustly in comparison with conventional region growing methods, particularly in the presence of sharp features, outliers and noise.     

Real-time collaborative reconstruction of digital building models with mobile devices

Many software-based building processes require digital building models. Since the building stock does not have sufficient data in this regard, the demand for Scan-to-BIM processes is increasing. In this paper we present a system for the reconstruction of ‘as-built’ BIM content of house interiors based on the Google Tango technology. The strength of our approach is the use of low-cost mobile scanning devices and a client-server system that allows for a real-time collaborative scanning and reconstruction of indoor scenes. We developed a server application that continuously aggregates scan data of multiple scanning devices (clients) and applies the data stream to a real-time post-processing pipeline to reconstruct rooms, walls, doors and windows. The reconstruction result is then distributed to all clients, where it is visualized in real time. The collaborative workflow and real-time data processing make our system especially useful in situations that are time-critical and require concurrent collection and processing of data. One of our targeted use cases therefore is the model generation for crime scene documentation. The effectiveness of our approach was demonstrated on three test sites. Our results compare well to other state-of-art methods regarding the reconstruction of walls, but they also revealed potential for improvement regarding the detection of doors and windows in occluded and cluttered environments.     

An evaluation of pose-normalization algorithms for point clouds introducing a novel histogram-based approach

Building Information Modeling is growing more relevant as digital models are not only used during the construction phase but also throughout the building’s life cycle. The digital representation of geometric, physical and functional properties enables new methods for planning, execution and operation. Digital models of existing buildings are commonly derived from surveying data such as laser scanning which needs to be processed either manually or automatically throughout various steps. Aligning point clouds along the coordinate system’s main axes (also commonly known as pose normalization) is a task benefitting any point cloud processing workflow, be it manual or automated. With the goal of automating this task, we compare various existing methods and present our own approach based on point density histograms. We conclude this paper by comparing and discussing all methods in terms of speed and robustness.     

空地融合的城市道路基础设施数字化技术研究

城市道路基础设施三维模型的重构,在城市道路BIM应用与数字化领域具有重大意义;针对城市交通基础设施数字化重构的需求,对车载激光扫描与无人机倾斜摄影采集技术进行综合运用,提出一套从信息采集、空地点云配准、点云分割到三维重构的完整技术方法;首先使用车载激光扫描技术和无人机倾斜摄影技术对交通基础设施信息进行采集,并使用运动恢复结构算法(SfM,structure from motion)生成基础设施空地点云;其次使用迭代最近点法(ICP,iterative closest point)对空地点云进行精配准,然后利用基于PointNet网络的方法对融合点云进行语义分割;最后对分割出的交通基础设施对象进行三维重构;提出的空地融合的城市交通基础设施数字化技术能够高效地实现交通基础设施重构,为城市交通基础设施数字化提供基础、为后续交通专业领域的应用研究提供便利.     

A cloud approach to unified lifecycle data management in architecture,engineering, construction and facilities management: Integrating BIMs and SNS

The problem of data integration throughout the lifecycle of a construction project among multiple collaborative enterprises remains unsolved due to the dynamics and fragmented nature of the construction industry. This study presents a novel cloud approach that, focusing on China’s special construction requirements, proposes a series of as-built BIM (building information modeling) tools and a self-organised application model that correlates project engineering data and project management data through a seamless BIM and BSNS (business social networking services) federation. To achieve a logically centralised single-source data structure, a unified data model is constructed that integrates two categories of heterogeneous databases through the adoption of handlers. Based on these models, key technical mechanisms that are critical to the successful management of large amounts of data are proposed and implemented, including permission, data manipulation and file version control. Specifically, a dynamic Generalised List series is proposed to address the sophisticated construction file versioning issue. The proposed cloud has been successfully used in real applications in China. This research work can enable data sharing not only by individuals and project teams but also by enterprises in a consistent and sustainable way throughout the life of a construction project. This system will reduce costs for construction firms by providing effective and efficient means and guides to complex project management, and by facilitating the conversion of project data into enterprise-owned properties.     

Live semantic data from building digital twins for robot navigation: Overview of data transfer methods

Increasing reliance on automation and robotization presents great opportunities to improve the management of construction sites as well as existing buildings. Crucial in the use of robots in a built environment is their capacity to locate themselves and navigate as autonomously as possible. Robots often rely on planar and 3D laser scanners for that purpose, and building information models (BIM) are seldom used, for a number of reasons, namely their unreliability, unavailability, and mismatch with localization algorithms used in robots. However, while BIM models are becoming increasingly reliable and more commonly available in more standard data formats (JSON, XML, RDF), they become more promising and reliable resources for localization and indoor navigation, in particular in the more static types of existing infrastructure (existing buildings). In this article, we specifically investigate to what extent and how such building data can be used for such robot navigation. Data flows are built from BIM model to local repository and further to the robot, making use of graph data models (RDF) and JSON data formats. The local repository can hereby be considered to be a digital twin of the real-world building. Navigation on the basis of a BIM model is tested in a real world environment (university building) using a standard robot navigation technology stack. We conclude that it is possible to rely on BIM data and we outline different data flows from BIM model to digital twin and to robot. Future work can focus on (1) making building data models more reliable and standard (modelling guidelines and robot world model), (2) improving the ways in which building features in the digital building model can be recognized in 3D point clouds observed by the robots, and (3) investigating possibilities to update the BIM model based on robot feedback.     

多特征融合与残差优化的点云语义分割方法

目的 当前的大场景3维点云语义分割方法一般是将大规模点云切成点云块再进行处理。然而在实际计算过程中,切割边界的几何特征容易被破坏,使得分割结果呈现明显的边界现象。因此,迫切需要以原始点云作为输入的高效深度学习网络模型,用于点云的语义分割。方法 为了解决该问题,提出基于多特征融合与残差优化的点云语义分割方法。网络通过一个多特征提取模块来提取每个点的几何结构特征以及语义特征,通过对特征的加权获取特征集合。在此基础上,引入注意力机制优化特征集合,构建特征聚合模块,聚合点云中最具辨别力的特征。最后在特征聚合模块中添加残差块,优化网络训练。最终网络的输出是每个点在数据集中各个类别的置信度。结果 本文提出的残差网络模型在S3DIS （Stanford Large-scale 3D Indoor Spaces Dataset）与户外场景点云分割数据集Semantic3D等2个数据集上与当前的主流算法进行了分割精度的对比。在S3DIS数据集中,本文算法在全局准确率以及平均准确率上均取得了较高精度,分别为87.2%,81.7%。在Semantic3D数据集上,本文算法在全局准确率和平均交并比上均取得了较高精度,分别为93.5%,74.0%,比GACNet （graph attention convolution network）分别高1.6%,3.2%。结论 实验结果验证了本文提出的残差优化网络在大规模点云语义分割的应用中,可以缓解深层次特征提取过程中梯度消失和网络过拟合现象并保持良好的分割性能。     

Automatic detection of building roofs from point clouds produced by the dense image matching technique

Three-dimensional (3D) spatial information of object points is a vital requirement for many disciplines. Laser scanning technology and techniques based on image matching have been used extensively to produce 3D dense point clouds. These data are used frequently in various applications, such as the generation of digital surface model (DSM)/digital terrain model (DTM), extracting objects (e.g., buildings, trees, and roads), 3D modelling, and detecting changes. The aim of this study was to extract the building roof points automatically from the 3D point cloud data created via the image matching techniques with optical aerial images (with red, green, and blue band (RGB) and infrared (IR)). In the first stage of the study, as an alternative to laser scanning technology, which is more expensive than optical imaging systems, the 3D point clouds were produced by matching high-resolution images using a Semi Global Matching algorithm. The normalized difference vegetation index (NDVI) values for each point were calculated using the spectral information (RGB + IR) in the 3D point cloud data, and the points that represented the vegetation cover were determined using these values. In the second stage, existing ground and non-ground points that were free of vegetation cover were determined within the point cloud. Subsequently, only the points on the roof of the building were detected automatically using the proposed algorithm. Thus, points of the roofs of buildings located in areas with different topographic characteristics were detected automatically detected using only images. It was determined that the average values of correctness (Corr), completeness (Comp), and quality (Q) of the pixel-based accuracy analysis metrics were 95%, 98%, and 93%, respectively, in the selected test areas. According to the results of the accuracy analysis, it is clear that the proposed algorithm is very successful in automatic extraction of building roof points.     

Scan-to-BIM for ‘secondary’ building components

Works dealing with Scan-to-BIM have, to date, principally focused on 'structural' components such as floors, ceilings and walls (with doors and windows). But the control of new facilities and the production of their corresponding as-is BIM models requires the identification and inspection of numerous other building components and objects, e.g. MEP components, such as plugs, switches, ducts, and signs. In this paper, we present a new 6D-based (XYZ + RGB) approach that processes dense coloured 3D points provided by terrestrial laser scanners in order to recognize the aforementioned smaller objects that are commonly located on walls. This paper focuses on the recognition of objects such as sockets, switches, signs, extinguishers and others. After segmenting the point clouds corresponding to the walls of a building, a set of candidate objects are detected independently in the colour and geometric spaces, and an original consensus procedure integrates both results in order to infer recognition. Finally, the recognized object is positioned and inserted in the as-is semantically-rich 3D model, or BIM model. The assessment of the method has been carried out in simulated scenarios under virtual scanning providing high recognition rates and precise positioning results. Experimental tests in real indoors using our MoPAD (Mobile Platform for Autonomous Digitization) platform have also yielded promising results.     

基于透视式增强现实的BIM建筑三维重建仿真

为了提高三维建筑模型的精准度,需要深入研究BIM建筑三维重建方法。当前方法耗时较长,得到的三维建筑模型与实际建筑之间的误差较大,存在效率低和精准度低的问题。将透视式增强现实技术应用到BIM建筑三维重建中,提出基于透视式增强现实的BIM建筑三维重建方法,通过BIM构建初始三维建筑模型,采用直接线性变换算法计算摄像机的内部参数和外部参数,完成摄像机标定。在摄像机标定结果的基础上采用LK光流计算方法得到像素在图像中的光流,根据光流的方向阈值和光流的大小筛选图像中的光流,提取到图像的匹配点,基于初始三维建筑模型针对建筑图像匹配点构成空间三维点云,采用Delaunay方法对空间三维点云进行三角化处理,针对处理后的建筑图像通过贴纹理完成BIM建筑三维重建。仿真结果表明,所提方法的效率高、精准度高。     

Extracting structural components of concrete buildings from laser scanning point clouds from construction sites

In construction projects, inspection of structural components mostly relies on classical measurements obtained by measuring tapes, levelling, or total stations. With those methods, only a few points on the structure can be measured, and the resulting inspection may not fully reflect the actual, detailed condition of the complete object. Laser scanning is an emerging remote sensing technology to accurately and quickly capture surfaces of structures in high details. However, because of the complex, massive point cloud data acquired at a construction project, in practice, data processing is still manual work with computer aided programs. To improve upon current workflows, this paper proposes a method to automatically extract point clouds of individual surfaces of structural components of a concrete building, which subsequently can be used to inspect construction quality based on geometric information of the surfaces. The proposed method explores both spatial point cloud information and contextual knowledge of structures (e.g., orientation or shape) derived from building design specifications and practice. For extracting point clouds of surfaces of each structural component, the proposed method consists of 4 consecutive steps for extracting: (1) floors, ceiling slabs, and walls, (2) columns, and (3) primary and (4) secondary beams. Each step consists of two ingredients: (i) rough extracting the candidate points of the component and (ii) fine filtering of the surface points of the components via cell-based and voxel-based region growing segmentation (CRG and VRG) incorporating contextual knowledge of the structural members. Experimental tests on two different types of concrete buildings showed that the proposed method successfully extracts the structural elements, in which the completeness, correctness, and quality from the point-based evaluation are larger than 96.0%, 96.9%, and 92.0%, respectively. Moreover, the evaluation based on a shape similarity showed that the extracted floor, ceiling slab and wall overlap to the ground truth more than 92.5%.     

Developing final as-built BIM model management system for owners during project closeout: A case study

To apply final as-built BIM models to facility management (FM) during the operation phase, it is important for owners to obtain an accurate, final as-built model from the general contractors (GCs) following project closeout. Confirming the accuracy of the final as-built BIM model is one of the most important works executed by owners to meet the accuracy requirement of final as-built models for FM. However, many practical problems arise relating to the management of final as-built models such as final as-built model mismatch, the lack of available final as-built models, and the entry of incorrect non-geometric information into the final as-built models. To solve these practical problems, this study develops a Final As-built BIM Model Management (FABMM) system for owners to handle final as-built BIM model inspection, modification, and confirmation (BMIMC) work beyond project closeout. The proposed approach and system can be used to manage the status and results of BMIMC management work for the final as-built BIM model to be performed. The proposed approach and system were applied in a case study in a selected building in Taiwan to verify and demonstrate its practical effectiveness. This study identifies the benefits, limitations, and conclusions of the FABMM system, and presents suggestions for its further application.