Groundbreakers in Digital Twining: Computer science abilities by Panagiotis Giannakeris, MsC Research Fellow

Meeting with our Groundbreaking Researchers

Imagine a world where machines see and learn, paving the way for revolutionary applications. These two abilities, Computer Vision and Machine Learning, play a crucial role in creating intelligent and dynamic digital twins for the construction industry. Their fusion enhances data processing, enables real-time monitoring, facilitates predictive analytics, ensures quality control, improves safety, optimizes resource allocation and contributes to the entire lifecycle management of built assets. At the forefront of this research, in the ASHVIN team, we count on our partner The Centre for Research & Technology, Hellas (CERTH), with the contributions of research associate at the Multimodal Data Fusion and Analytics Group of the Information Technologies Institute Panagiotis Giannakeris.

We are pleased to present to you this #5 interview from our series  of discussions with the new generation of researchers that participate in the ASHVIN project as a part of their MCs, PhD or Postdoc studies. They will share their research approach in ASHVIN, and plans to continue exploiting this work beyond the project.

Today, we explore Panagiotis’ expertise in computer vision and machine learning. Boasting six years of professional experience, he holds an engineer’s diploma and a Master of Science (MSc) in Digital Media and Computational Intelligence. Throughout his career, he has actively participated in diverse research projects spanning crisis management, decision support systems, digital healthcare services, and the construction industry since 2023 with ASHVIN.

We invite you to watch Panagiotis’ video presentation to gain a comprehensive understanding. This is the fifth episode of our #ResearchStories series, and it sets the stage for further discussions.

Hello Panagiotis, we are very happy to share this space with you.

Hello, I am happy to walk you through my work and research.


Could you describe the scope of your research, and how it is related to Digital Building Twins?

Well, the concept of Digital Building Twins is rooted in the creation of highly detailed, real-time digital replicas of physical structures and construction projects.

These digital twins provide a holistic and up-to-date view of a building’s status and are used for various purposes, including design optimization, construction monitoring, and facility management.

My research revolves around developing and implementing advanced computer vision and machine learning algorithms on the task of 3D point cloud semantic segmentation. This involves the intricate process of categorising individual points within a 3D point cloud into meaningful classes, such as walls, floors, structural components, and equipment.

Semantic segmentation is essential for creating a comprehensive and semantically meaningful digital representation of a building or construction site. By enabling the creation of more accurate, detailed, and real-time digital representations, we can enhance the efficiency, safety, and overall effectiveness of construction projects.


Which aspects of ASHVIN are relevant to your MsC research ? 

Key aspects of ASHVIN that are directly relevant to my work are the digitisation of the construction industry and the fact that this can be accelerated through digital twin technologies.

My current work in 3D point cloud semantic segmentation can be a crucial component of advanced applications such as scan-to-BIM, where one must first have to capture a space, and then turn it into a digital model embedded with information that can be used for planning, monitoring or managing the built environment.

Normally, this process requires the identification and labelling of individual elements within the point cloud, such as walls, floors and other structural components and it can be a cumbersome task if done manually.

Our work is aiming to achieve the automatic labelling of such semantic regions so that the BIM models will be available as quickly as possible and at multiple stages with enhanced efficiency and precision so that the manual work of humans can be allocated elsewhere in the actual important tasks.


Which challenges have you faced while doing your research?

One of the top challenges is the limited availability of real-world construction data, especially in the form of 3D point cloud scans. Obtaining comprehensive and diverse datasets that accurately represent the complexity of construction sites can be challenging.

Even when such data is available, it often exhibits high variability and noise due to factors like weather conditions, lighting, and the presence of moving equipment. Dealing with such variability and noise is a substantial challenge in ensuring the accuracy and robustness of semantic segmentation models.

Another important thing to mention is that semantic segmentation of 3D point clouds is inherently complex, as it involves the classification of each individual point into various object classes, such as walls, floors, and equipment and developing algorithms that can handle this complexity and maintain high accuracy is a continual challenge. Training and evaluating the algorithms also demand substantial computational resources, thus it is always desirable that the solutions are not very computationally heavy but this is another aspect that’s always explored further after we have developed methods that ensure a high level of accuracy first. Achieving models that generalise well across different construction sites and scenarios is a non-trivial challenge.

Algorithms must be trained on one dataset but should perform effectively when applied to new, unseen data from various construction sites. For all those reasons I am currently further investigating methodologies that require very few additional data to work well in unseen buildings or sites.


ASHVIN is ending in March 2024, what will you take with you from the project for the next steps of your research and professional development ? 

First and foremost the ASHVIN project has provided me with the opportunity to work on genuine challenges faced by the construction industry. This practical experience is invaluable as it has broadened my perspective and has given me a deeper understanding of the nuances of applying my practice in a real-world context in the construction domain and it is knowledge which will be directly transferable to future research work, and to mention also opening doors for potential future collaborations with industry professionals.

In addition the access that was given through ASHVIN to real constriction data enabled me to carry out experiments in datasets that are otherwise very difficult to obtain. These datasets serve as the basis for training and fine-tuning the algorithms related to 3D point cloud semantic segmentation and as such allow the expansion of our knowledge in this domain.

Thanks for the talks! 


💡 Did you know that behind the scenes in the ASHVIN project, there are over 30 university students, from different degrees, contributing to the project’s research related to the technical development of the digital twin system and to its demonstration in real-life sites. Our academic partners ; Technical University of Berlin (TUB), Erasmus Univeristy of Rotterdam (EUR) and Polytechnic University of Catalonia (UPC) oversee their involvement in this project.