How to turn your Digital Twin Project into a VR Experience

Hello Manifold and the rest of the community!

Let me introduce myself! I am Keith, a Computer Science student in Hong Kong! I have wrote some forum post previously regarding the VR Experience with your own Digital Twin project.

I'm trying to turn my own Digital Twin project into a VR experience since I'm doing a final year project for my final year of university in Hong Kong. It's related to a Immersive Digital Experience on how to travel from pre-booking to post-flight. Therefore, I would like to see if I could use a Point Cloud scanning and import it into a VR Experience.

In this post, I'll be summarising the procedure since I'm having trouble writing the steps in this Forum Page, and also I'll be uploading the full detail of the procedure in this post. So, I hope everything works well when following my procedure! If you have any questions, please let me know or even Manifold Tech as they assisted me with their software and some setup! Let's begin!

As I go along with this post, the detailed procedure (.pdf file) is down below:

You may also refer to my forum post for the progress:

Part 1: https://www.manifoldtech.cn/forum/general-discussion/part-1-digital-twin-vr-experience-setting-up-the-project

Part 2: https://www.manifoldtech.cn/forum/general-discussion/part-2-digital-twin-vr-experience-general-procedure-updated

Part 3 + VR video: https://www.manifoldtech.cn/forum/general-discussion/part-3-digital-twin-vr-experience-vr-clip-of-the-digital-twin-project

Part 1: Setup Requirement

Hardware:

A Windows Computer (Windows 10 or above)
VR Headset
LiDAR Scanner

I used a Windows Computer since MindCloud Software only supports Windows and the files that I have scanned with the LiDAR Scanner will be used in the MindCloud Software. And then, the LiDAR Scanner is used for 3D scanning to get point cloud data of our desired location.

File Structure:

(Please refer to the file .pdf file)

Software:

Potree viewer and Potree
converter
Point Cloud software
VR Client Software (Latest Firmware)
Mobile Software for the LiDAR Scanner (depends if your LiDAR scanner requires)
Python (Version 3.10 is recommended)

I used the Potree viewer for previewing our Point Cloud project on the browser since it has a VR function implemented in Potree, so I need to use the Point Cloud PC and mobile software to preview my scanned files, convert it into .las format, and then Potree format with the converter. Python is needed to support the Potree format. And then, the VR Client to show our Point Cloud data in VR Experience.

Part 2: 3D Scanning Point Cloud

Part 3: Data Conversion

Part 4: View in VR

Extra: Troubleshoot

Conclusion:

It was a interesting experience being able to use my own digital twin projects into Virtual Reality. Although the model is only showing in point cloud form, there is a potential of improving the Potree project and we may see a more "photorealistic" digital twin in the future. Additionally, I did the 3D scanning after the rain ended, therefore, my scanning could have been better, yet it was interesting results.

My next step for enhancing this VR experience is to import my digital twin project into a game development engine such as NVIDIA Omniverse, Unreal Engine, or Unity, which I believe there will be a lot of trial-and-error and consumes lots of time.

I hope this forum post gives you an insight of my perspective when doing the VR experience of your own Digital Twin project. Please look at the detailed procedure as there are clearer instructions and screenshots of my demonstration.

VR Clips:

VR Park

VR Football Pitch

VR Beach BBQ area