Lab

Virtual production

We really like to organize PHYGITAL NIGHTS educational parties at our shtab, but the pandemic has imposed some restrictions on them. Longing for the guests and the atmosphere of the holiday season, we decided to hold a party in an online format – and of course with technological effects.
We built our own Virtual Production system, and the main goal for us was to show various mechanics that can be applied in real time.

Shooting

For shooting, we used:
  • iPad Pro 2020 with LiDAR camera for calculating the depth and applying effects in real time,
  • two RGB Kinect cameras for shooting from different angles

Visual effects

We used the visual effects of Keijiro RCam2, which we modified for our tasks.
Among the effects there were:
  • The effect of occlusion – segmentation of the human body
  • 360 video
  • 3D models


3D model of Rita Ora made with the PiFU neural network

Result

Recording of the livestream

Read more about virtual production

2021-08-17 16:47 Experiments