Wonder Dynamics now lets you go straight from multi-camera video to fully animated 3D scene

Wonder Dynamics made a strong opening play in AI-enhanced visual effects, providing tools animators and filmmakers actually find useful — and earning the startup a prompt acquisition by Autodesk. Their latest tool further automates the animation process, letting you put in practically any video and get a fully editable 3D scene, characters and all.

The feature combines the company’s existing drop-in actor replacement, in which you could easily replace a person with an animated 3D character keyed to their movements, with a full scale 3D background of your choosing.

To be clear, as with all the company’s previous features, this is not intended to provide a finished product — just a base layer to build on. Animation is labor intensive, and location shooting is difficult and expensive — if you can film two friends walking down the hall and convert it to robots walking down a futuristic street, that’s super useful for previsualization, story flow, and other early stages of production.

“The idea is that you can shoot and edit your video in your living room, with desired performances, cut, and camera framings as you would want to see in your animation,” said Wonder Dynamics co-founder Nikola Todorovic. “This process would provide you with a quick previs and, most importantly, 3D scenes to edit each individual element in either Blender, Maya or Unreal.”

Impressively, you can use multiple cameras to shoot the same scene, as you might normally with human actors, and the video-to-3D flow will place them in the same scene as virtual cameras following the original motion by estimating foot travel and other metrics. Again, helpful for early in the process when camera work is being roughed out — not something you want to decide on shoot day!

You can see it in action here:

Unlike some generative and text to video (or video to video) type models, the output of the Wonder Animation tool is all industry-standard 3D assets. Once you shoot it, it’s not locked in — you can drag and drop the cameras and tweak the character’s movements, or totally recompose and relight the scene. Some of this was possible in the original Wonder Studio, but the focus here is on fully animated scenes rather than, say, replacing one actor with a CG character.

Anyone using Wonder Studio should be able to access the animation feature today, though it’s technically in beta and will probably see improvements as feedback is submitted.

Subscribe for the industry’s biggest tech news