Virtual Production

VIEW SYSTEMS

Virtual Production is a term that is quickly becoming more and more popular and there is a lot of discussion around its current use and potential, but what exactly is Virtual Production? This page will explain it in detail, look at the different types of virtual production and run-through what you need to incorporate it in a workflow.

What is Virtual Production?

Virtual Production is a method of production that combines the physical world with virtual elements using real-time software. This can be achieved through different methods such as visualisation, performance capture or using an LED wall. One of the key benefits to virtual production is that creative decisions, such as moving elements within a scene, can be seen in real-time and this makes collaboration a much more stream-lined process.

The Problem with Traditional Production

The majority of productions today will feature visual effects in one form or another and there are entire genres that will rely on it heavily. The issue with a traditional style of production is that utilizing visual effects often requires techniques such as filming on a green screen or compositing an element in later, without knowing exactly what the final image will look like when these elements are replaced. This causes several problems with framing, lighting, actors staging and so on. This linear style of production also separates the filming and visual effects stages for the most part and that separation can lead to iteration, which can be a lengthy process.

Virtual production however, takes this process and makes it a more non-linear and collaborative workflow, all in real-time. This can manifest in pre-production using visualisation to bring all team members a closer representation of what the final image will look like before production has begun. Alternatively, it can be implemented in production with the visual effects department engaged to provide real-time imagery on what the final image is intended to look like.

Who Can Use Virtual Production?

We are currently seeing big-budget productions pioneer the use of virtual production techniques and benefiting the likes of cinematographers who can view camera visual effects in real-time, visual effects departments who can work more collaboratively across all stages of production and producers who can get the best results out of production and demonstrate their intentions with a project at earlier stages. As the required technology becomes more readily available, we are seeing this being used across all levels of production to gain benefits.

Virtual Production Types

Virtual production can encompass so many different techniques and terminology, when it’s broken down though, virtual production simply consists of four types of production that share the common feature of a real-time engine. Knowing what these are can help give an understanding of the benefits and advantages to a virtual production workflow.

Visualization

Visualisation is often part of the pre-production stage and is the creation of prototype imagery to plan and represent certain shots or sequences. This can take several forms such as:

Pitchvis - Used in the development process to pitch key sequences or even trailers to a studio or potential investors
Previs - You might recognise this from behind-the-scenes or before/after footage showing a concept for a visual effect element. This is the next step to storyboarding, allowing for more creative detail.
Virtual Scouting - Location scouting occurs in traditional production to choose a location or plan the shoot. Virtual Scouting is the process of interacting with a completely digital version of a location, down to using virtual cameras and lenses to plan out specific shots.
Techvis - The process of bringing virtual elements to the real world and assessing the physics of camera moves, placement and lens choices to plan certain shots, sequences or entire scenes.
Stuntvis - Planning how stunt moves will integrate with the camera placement, set and props down to specific beats which can be used to improve safety or creativity of a stunt.
Postvis - In a similar way to previs, this is demonstrating the combination of live-action elements with temporary visual effects to help guide earlier stages of post-production.

Performance Capture

Motion capture is the process of creating animated models by recording the movements of objects or actors, this is usually accomplished by an actor wearing a suit covered in markers and using depth-sensor cameras for facial capture. Performance capture is used to have virtual characters interact with real-world environments in real-time, which is designed to help aid in framing and timing on location.

Hybrid Virtual Production

In traditional production, when filming in a green screen studio the footage is sent to post-production who composite the image after the shoot and this can often be the first time the combination of the elements are seen. Hybrid virtual production is used either to provide a crew a live preview of composite shots which are then to be finalised later in post-production or this has already been utilised to key in real-time for live broadcast, especially in sports.

Full Live LED Wall

This is becoming the most popular and discussed form of virtual production at the moment, and is the process of combining camera-tracking with real-time images displayed on a live LED wall. When compared to traditional green screen filming some of the benefits of this include it allowing all departments to see the final-pixel imagery, the actors are lit by the wall while also being able to react to the environment and it removes any uncertainty.

Virtual Production Project Types

Fully Animated Virtual Productions

Utilising virtual production in the form of performance capture has already had a massive impact on the creation of animated films and we have seen this being utilised for a while now. As technology develops we are only seeing the accuracy and quality of this grow, coupled with the inclusion of virtual cameras, fully animated virtual productions are only going to benefit as they become more realistic and lifelike.

Live-Action Virtual Productions

It can be quite easy to imagine how live-action productions can use virtual production in the form of an LED-wall to bring a new dimension to visual effects-driven productions that would have previously been based in a green-screen. However we are also seeing how visualization techniques can enhance productions that are set entirely on location with minimal effects.

Key Features and Benefits of Virtual Production

 

Real-time Workflow

One of the greatest benefits of virtual production is that everything happens in real-time, down to the compositing. This means everything can be assessed as it happens and instead of requiring the formerly linear process of post-production to determine any issues that arrive or creative decisions that need to be made, they can now be spotted or made at the time of production.

 

Remote and Multi-User Collaboration

As technology and computing get faster, communication and collaboration in production is only getting stronger and in a virtual production workflow this has benefits across all stages of production such as remote virtual scouting sessions to collaborative performance capture sessions.

 

Malleable Tools

Using a real-time engine as part of a production gives various different departments presets and tools they will use as part of their virtual production workflow. These have the advantage that they can be revisited and re-used across different scenes, locations and even projects.

 

Real-time Physics

The physics in a real-time engine are a key part to certain areas of virtual production and are what makes tools such as simulating camera movement possible. This brings benefits to areas such as visualisation down to the visual effects seen in the final image.

The importance of a powerful Virtual Production Systems

Whether it’s compositing in real-time or displaying a high-resolution image across several LED screens this can require a lot of computing power and one of the most important tools you need to power a virtual production setup is going to be the workstation. Making sure you have the right workstation for your needs is crucial for achieving the best results and Scan Pro Video offer solutions for all scales of production. Below are some examples of specifications for some of the different virtual production uses.

Virtual Production - LED Wall Virtual Production - Green Screen / Content Creation
CPU AMD Threadripper 3990X 64-Core AMD Ryzen 9 5950X 16-Core
RAM 128GB DDR4 64GB DDR4
GPU NVIDIA RTX A6000 48GB NVIDIA GeForce RTX 3090 24GB
Drive - Primary 500GB Samsung 980 Pro NVMe 500GB Samsung 980 Pro NVMe
Drive - Projects 2TB Samsung 980 Pro NVMe 2TB Samsung 980 Pro NVMe
Add-In Card Quadro Sync II Capture Card

3XS Virtual Production Workstations

Scan Pro Video have a range of virtual production workstations that are designed and optimised for driving LED walls in a virtual production. These systems leverage the power of NVIDIA's A5000 and A6000 GPUs alongside a SyncII card, allowing you to sync your output across multiple high-resolution LED Walls. Check out our workstations below for more information.

VIEW SYSTEMS

Collaborative Workflows

The industry had a massive shift recently as our way of working was made to be more remote. The response from the latest technology made it that working collaboratively and remotely is the new normal. Real-time technologies make all this possible including tools such as NVIDIA Omniverse.

NVIDIA Omniverse is the platform that provides content creators the ultimate real-time collaboration experience.

To find out more visit: here

Tracking Systems

Depending on the type of virtual production workflow, there are some hardware requirements that will be involved. Part of utilising real-world data for virtual productions requires translating camera operation to create more realistic movements. Companies such as MoSys provide camera tracking solutions such as StarTracker, an optical solution built to address in-studio camera tracking.

LED Walls

LED Walls are created by combining multiple LED panels to create a virtual screen using powerful processing hardware and real-time software. Unlike using a green screen backdrop, the image displayed on the LED Wall is recorded to the camera and delivered in the final project. To keep up with the quality of modern-day cinema cameras, it’s important the LED panels are high-resolution, have a wide colour space and are of the highest quality.

As part of offering solutions for all stages of the virtual production workflow, Scan Pro Video have solutions for LED walls, including all the necessary connectivity.

Contact the Scan Pro Video for more information at:
[email protected] or 01204 47 47 49