Measured vs inferred ground on real slopes.

01The question

Human motion data for robots usually comes without the ground it was performed on. Studio motion capture happens on a flat floor, and video taken outside rarely includes a survey of the site. A robot trained on that motion learns to step onto a floor that was never there.

One way around it is to infer the ground from the motion: a foot that stays still for a moment is probably standing on something, so the heights of the planted feet trace out a surface. TERRA, published by the Mathis lab at EPFL, does this with a model of a flat floor plus boxes, stairs and ramps. It needs nothing but the motion. Our capture measures the ground directly from the same drone footage that gives us the body. We wanted to know how far apart the two answers are on real outdoor ground.

02Setup

• Footage. One afternoon at a public park (October 17, 2026), three consumer drones following one person, with no markers, survey or calibration board. The full capture is written up here.
• Windows. Four slope windows chosen before scoring: a levee crossing with about 2 m of relief over 50 m, a grass hill, a hill climb and a hill descent. A flat-grass window is the control. Three picnic-table step-up windows were also run; they wait on a tape measurement of the tables and are left out here.
• Reference. The USGS 3DEP 1 m bare-earth lidar elevation model of the park, which is public domain and independent of everything we capture. Its vertical accuracy is about 10 cm. The capture is placed on the map from the drones’ own GPS, with no ground control points, so the horizontal placement is good to a metre or two.
• Foot contacts. Detected on our measured 3D body, on frames where at least two drones saw the person: 11 to 57 per window. Every method is scored on the same contacts, the ones where our measured ground exists: 10 to 39 per window.
• Score. For each foot contact, the method’s ground height minus the lidar height there. Each method’s median of that difference over the window is subtracted, and we report the median and the 90th percentile of what is left, as absolute values in centimetres. The motion-only method has no absolute height to anchor to, so this compares the shape of the ground, which is what a robot’s feet meet.
• Baselines. Motion-only inference, run from its public code on our 3D body motion at its default smoothing, and a flat plane at the median foot height. We ran it as a benchmark; none of its code or output is part of what we deliver.

03Results

Each cell is median / 90th percentile of the per-contact error. The slope summary is the median of the four window values. In every window all three methods are scored on the same contacts: the ones our measured ground covers, which is all but one or two per window except on the grass hill, where only the first half was solved (28 of 57). Scored on all contacts, the two baselines on the grass hill are 19 / 144 cm (flat) and 18 / 144 cm (motion-only), and the slope summary becomes 87 cm (flat) and 67 cm (motion-only) at the 90th percentile.

On flat grass the three agree to within a few centimetres, which is what a sanity check should show. Once the ground tilts, they separate. The measured ground follows the lidar to 3 cm at the median and about 10 cm at the 90th percentile, which is at the level of the lidar’s own accuracy. The motion-only surface is right where the person stands still on a level patch and wrong between those patches.

04What the errors look like

The descent shows the pattern clearly. Motion-only inference turns the slope into a staircase. Its fitted boxes, ramps and stairs covered between none and 84% of the walked strip, depending on the window, and the rest is its floor. On the descent each planted foot makes a step at its own height, so it gets the overall drop right (10 cm at the 90th percentile here), but a robot’s foot landing between two steps meets a vertical edge that is not there. On the levee, a rolling 50 m crossing with a crest, the staircase lags behind the rise and leaves spikes where the fitted geometry ends, and the error at the 90th percentile reaches 87 cm.

This is not a failure of the method on the job it was designed for. Its terrain model is built from flat pieces: a floor with boxes, stairs and ramps, which suits structured indoor scenes. A continuous grass slope or a rolling levee is outside that model, whatever the input. Our input also differed from what it was built for (laboratory optical motion capture with foot markers, against video-derived body motion here), which may cost it a little more. The comparison answers a narrower question that matters to us: if all you keep from a field capture is the motion, how well can you get the ground back? On these slopes, not well enough for a robot’s feet.

05The two weak windows

• Grass hill: a coverage failure. The drones saw the ground under only the first half of this walk, so our measured ground covers 49% of the contacts and a third of the walked strip. Where it exists it scores 2 / 10 cm, against 25 / 174 cm for motion-only and 6 / 45 cm for flat ground on the same 28 contacts. The rest is not measured, and we do not fill it in.
• Hill climb: registration, not shape. Here the whole capture sits about 1.5 to 1.8 m sideways from where the survey puts that slope, so even the person’s own foot contacts land up to 60 cm below the lidar for the first few seconds, and every method reads high. As a diagnostic, one horizontal shift fitted on the feet alone (not on any surface) brings the 90th percentile to 5.5 cm for the measured ground, 19.5 cm for motion-only and 73.5 cm for flat. The table keeps the unshifted 45 cm.

06Why this matters for robot training

When motion is retargeted to a humanoid and replayed in physics simulation, the ground model decides where the feet are supposed to land. A ground model that is 40 to 80 cm off on a slope leaves stance feet tens of centimetres above or below the real surface, so the reference no longer describes motion a robot could perform on that ground. How much this costs a trained policy depends on the training setup; we are measuring it with matched training runs and will add the result here.

On an earlier capture we trained a G1 tracking policy on ground rebuilt from the footage and it stayed up in 196 of 200 randomised runs on that path, against 128 for a flat-trained policy (details). The benchmark here is the other half of that story: the ground has to be measured to be trusted.

07Limits

• One afternoon, one park and one person. Four slope windows and one flat control, with 11 to 57 contacts each.
• The lidar is a 1 m grid with about 10 cm vertical accuracy, so differences below about 10 cm at the 90th percentile are within the reference’s own noise.
• The score removes one constant per window, so it says nothing about absolute heights. The capture is registered to the survey map without ground control points, and absolute placement is not part of this test.
• The motion-only method ran at its default settings on video-derived motion rather than the laboratory motion capture it was built for. On its own terrain types (floors, boxes, stairs) it would likely do better.
• Table and bench heights are pending a tape measurement and are not claimed here.
• All robot numbers on this page are from physics simulation.