How it works
- A Unity scene consisting of objects is rendered using a regular pipeline.
- At the output, each object has a special mask in the form of an image, which is processed by a neural network that converts it into the final image.
- The generated image is the result of the final render displayed on the screen.
Technology stack
- The neural network runs on the server
- Unity acts as a client application
- For the VR experience, the HTC Vive and SteamVR helmet were used
We see the development potential in creating a tool for drawing applications in virtual reality and obtaining a photorealistic result.
Examples
- Potential for art – creating visuals for music or installations
- GauGAN in virtual reality