Video to Motion Capture — How Markerless Mocap Works and How to Get Good Results

Motion capture used to mean a studio: a suit covered in reflective markers, a ring of synchronized infrared cameras, and a technician cleaning up the data afterwards. That setup still produces the best results, but for a huge share of animation work you no longer need it. Modern pose estimation can recover a full-body skeleton from a single ordinary video, which means a phone clip of someone walking across a room can become an animation file in your 3D software a few minutes later.

This guide explains how single-camera markerless capture works, where its limits are, how to film so the tracker succeeds, and how to get the resulting BVH file onto your own character in Blender, Unity, or Unreal Engine.

How Markerless Motion Capture Works

The pipeline has four stages. First, a neural network trained on millions of annotated images finds a person in each video frame and predicts the position of 33 body landmarks: head, shoulders, elbows, wrists, hips, knees, ankles, plus reference points on the hands, feet, and face. Second, the network estimates depth for each landmark, producing 3D positions in meters relative to the hips. Third, a low-pass filter removes the frame-to-frame jitter that per-frame prediction always produces. Fourth, the filtered positions are converted into a skeleton: bone lengths are measured from the footage itself, and joint rotations are solved so that a standard armature reproduces the observed motion.

The critical difference from studio mocap is that everything is inferred from pixels rather than measured by triangulating markers. Inference is remarkably good when the body is clearly visible, and it degrades in predictable ways when it isn't. Understanding those failure modes is most of what separates a clean capture from a messy one.

Capture Methods Compared

Single-camera capture sits at the bottom of that table on fidelity and at the top on convenience. For blocking, previz, game prototypes, background characters, and reference for hand-keyed animation, it is usually all you need. For hero character animation in a shipped film, it is a starting point that an animator polishes.

How to Film for Good Tracking

The tracker can only recover what the camera can see, so the filming choices matter more than any processing setting. These rules come directly from how pose estimation fails:

One person in frame. The tracker follows the most prominent person; a second person walking through the shot can steal the track. Keep the whole body visible, head to feet, for the entire clip. When the feet leave the frame the leg estimates become guesses. Use a static camera, on a tripod or propped on a shelf, because camera motion adds apparent body motion the tracker cannot separate. Light the subject evenly; noise from dim footage becomes jitter in the skeleton. Wear fitted clothing, since the network estimates joints from the body silhouette, and a skirt or hoodie hides it. Film at 30 fps or higher with a fast shutter to avoid motion blur on quick moves, and keep clips short: ten to sixty seconds of one action tracks better and is easier to edit than five minutes of everything.

Camera angle deserves a special mention. A three-quarter or side view gives the depth estimator the most information. Movement directly toward or away from the lens is the hardest case, because depth from a single camera is the weakest measurement. A punch thrown straight at the camera will read with less energy than one seen from the side.

What a BVH File Actually Contains

BVH (Biovision Hierarchy) is a plain-text format from the 1990s that remains the lingua franca of mocap interchange. It has two parts. The HIERARCHY section defines the skeleton: a tree of joints, each with a rest-pose offset from its parent, which encodes the bone lengths and the T-pose. The MOTION section is a table with one row per frame, holding the root position plus rotation angles for every joint. Because rotations are stored relative to each parent joint, the same file plays back correctly on any software that rebuilds the hierarchy.

Some tools export CSV alongside BVH. A CSV export contains the raw landmark trajectories, one row per frame with the 3D position and confidence score of each tracked point. You lose the rig but keep the measurements, which is what you want for biomechanics analysis, sports coaching data, ML datasets, or a custom retargeting script in Python.

Getting the Animation Into Blender

Blender imports BVH natively: File → Import → Motion Capture (.bvh). Two settings matter. Set the scene frame rate to match the capture (the import dialog shows the file's frame time), otherwise the motion plays too fast or too slow. And check the rotation order if you edit curves later; imported BVH curves use the channel order from the file. Once imported, you get an armature with keyframes on every joint. Play it back and scrub through before doing anything else, so you know what you are working with.

To move the animation onto your own character, retarget it: constrain your rig's bones to the imported armature (Copy Rotation constraints joint by joint), or use a dedicated tool such as Auto-Rig Pro's remap module or Rokoko Studio Live's retargeting, both of which map bone names between skeletons and bake the result.

Unity and Unreal Engine

Neither engine imports BVH directly, so Blender acts as the bridge: import the BVH, export an FBX with the armature selected. In Unity, set the imported rig to Humanoid; the Avatar system then retargets the clip onto any humanoid character. In Unreal Engine 5, import the FBX as a skeletal animation and use the IK Retargeter to map it onto your character skeleton. In both cases the retargeting layer also gives you a chance to fix proportion differences between the captured person and your character.

Known Limits, Stated Plainly

Single-camera capture has limits that no setting can remove. Root motion through space is unreliable, because absolute position from one camera is a weak estimate; exported animations stay roughly in place and you add world travel in your software. Fingers and faces are not captured; the hand is tracked as a single point. Depth ambiguity means limbs pointed directly at the camera can flip or lose energy. And occlusion is unforgiving: when a limb is hidden behind the body, its position is an educated guess, which shows up as sliding during those frames. Plan shots around these limits and the output is consistently usable; fight them and no tool will save the take.