Striving for immersive and interactive experiences, grounded in reality.

The goal of this project is to work towards an immersive art tool that is interactive, does not require the use of headsets, trackers, or other cumbersome accessories, and that is open and accessible to artists with lower budgets and access.

The vision for the initial product is a markerless tracker that would map a user’s spatial positioning in front of a screen with a virtual environment. Their position affects the view of the scene, causing it to parallax. The result for the user is very similar to what the camera would see in a virtual production environment. Additionally, this tool works with nDisplay, in order to allow live changes to the environment by the artist or audience.

Current Stage of Development

Under my Individual Artists Digital Evolutions award, I developed a prototype tool. This version of the tool utilizes computer vision technology in a web-based app, using only a webcam. The positional data is then sent to Unreal Engine through the Web Control plugin, where the data is mapped to control an nDisplay object.

Looking Forward