Rehoboth
LiDAR hologram VR experience — a walkable memory of a place and its people
A LiDAR hologram-based virtual memory of the elderly that family members can step back into after they die.
In XR narratives, spaces become characters. Photogrammetry imbues those spaces with static detail, but LiDAR holograms bring them to life. The first time I combined 2.5-dimensional depth video with a photogrammetry model I was struck by how much it felt like I had stepped into a memory. Which made me ask, what do we most want to remember? One tragic, beautiful answer is: our loved ones after they die.
These holograms are videos that have been rendered into 3D meshes – they are glitchy and pixelated, imperfect like our own memories. In this liminal space, the photogrammetric walls melt into the floors and yet the sense of presence, of standing in a real space with real people, is undeniable.
In XR narratives, spaces become characters. Photogrammetry imbues those spaces with static detail, but LiDAR holograms bring them to life. The first time I combined 2.5-dimensional depth video with a photogrammetry model I was struck by how much it felt like I had stepped into a memory. Which made me ask, what do we most want to remember? One tragic, beautiful answer is: our loved ones after they die.
This project took Painting Life’s explorations into XR narrative conventions a step further: when you move through the house, you can pick up a board game and the family appears playing it together at a table. If you walk outside through the garden, you hear the sounds of the geese in the lake. You may end up in the woodshed where the grandfather is turning wooden bowls with his grandchildren.
These holograms are videos that have been rendered into 3D meshes – they are glitchy and pixelated, imperfect like our own memories. In this liminal space, the photogrammetric walls melt into the floors and yet the sense of presence, of standing in a real space with real people, is undeniable.
One playtester also reported learning how to carve a wooden spoon after playing through this experience, hinting at the educational potential of such virtual memories.
The tech stack and development process for this was similar to Painting Life with the added complexity of hologram 2.5D depth video. Azure Kinect RGBD cameras streamed data to a gaming laptop, and an iPhone 12 running Record3D app used the front-facing LiDAR sensor to record "depth videos". Depthkit Studio cleaning and cropping the depth data and exporting RGBD video. Here you can see the Depth LiDAR on left, and the RGB Color video on right – both exported as a single MP4.
I used Depthkit's custom plugin in Unity, which interprets the MP4 as a hologram, which you can position in 3D space and see in VR.
| Wyatt Roy | Creative Director, Photogrammetry, Hologram Capture, Sound Design |
| Luis Zanforlin | Unity Scripting |
