Digital twins are virtual 3D copies of real spaces such as factories, warehouses and industrial facilities. Companies use them to monitor equipment, plan repairs and simulate changes without stopping production. But for such a copy to be truly useful, the system must understand what exactly it is seeing: where a pipe is, where a container is, and where a machine is. The task of recognizing and labeling objects is called segmentation.
This problem is the subject of a thesis by Ofia Rahman Alif, carried out jointly with the University of Helsinki and the Finnish company Twinverse, which creates digital twins of complex industrial facilities. Twinverse Chief Technology Officer Arttu Ylin noted that cooperation with the university produced valuable results, and urged companies to take a more active part in such projects: for business this is mainly a cost of time, not money.
The work systematically analyzed where the approach works and where it fails under real factory conditions. The study touched on several factors: how the quality of 3D reconstruction affects segmentation accuracy, how camera settings change the result, and how the consistency and accuracy of object masks (that is, the highlighted regions in an image) affect the final quality.
Conditions that complicate the work of artificial intelligence in manufacturing were examined separately: dark metal surfaces, complex geometry, partially occluded objects and objects that resemble one another. Each of these factors affects different stages of processing.
To make the comparison clear, methods for improving AI performance were tested both on real industrial data from Twinverse and on standard test scenes usually used with the 3D Gaussian splatting (3DGS) method. This method reconstructs a real space as millions of tiny three-dimensional elements fitted to photographs taken from different angles. This approach made it possible to directly compare idealized conditions with the more complex reality of an operating enterprise.
The work is based on combining three existing tools. The first, 3DGS, creates a photorealistic 3D reconstruction from ordinary images and camera position data. The second, FlashSplat, assigns these three-dimensional elements to specific objects through voting across many viewpoints. The third, SAM2 (Segment Anything Model 2 from Meta), recognizes and outlines objects in images. The thesis proposes a scheme linking these tools: SAM2 creates object masks, its video tracking function maintains mask consistency across viewpoints, and the results are passed to the FlashSplat voting system. In addition, an adaptive component was added that automatically selects the best SAM2 settings for each scene, taking into account lighting, surface reflectivity and scene complexity.
One of the most unexpected results: increasing the volume of data for the AI made performance worse, not better. A small, carefully verified input dataset produced a noticeably cleaner result than a much larger but noisy one. The system cannot undo bad evidence once it has been taken into account, so a few errors can outweigh a lot of additional data. According to Alif, this is a reminder that in AI, quality is often more important than quantity.
The transition from controlled research datasets to a real industrial environment proved to be a much more serious step. Tools that worked well on clean scenes unexpectedly broke down in an operating production setting. The main conclusion: failure is not a dead end if it can be explained, and understanding the causes of a failure is often as valuable as successful operation.