Where the work stands

This is a plain account of what exists, what we learned building it, and what we are working on now. It is deliberately short on mechanism.

What exists

A pipeline that takes ordinary video, from any camera, at any frame rate, and turns it into a normalized record of how the body moved. A forecaster that watches a few seconds of that record and streams a continuation, one frame at a time, for as long as you ask. It runs on a single consumer GPU, fully offline, and the phone capture that prompts it never has to leave the device.

The forecaster has been evaluated on held-out subjects from a public laboratory corpus and a public dance corpus, and on phone recordings of people the model had never seen, where the real future plays out alongside the forecast and serves as its own ground truth.

What we learned

Forecasting motion is mostly a fight against the model’s own output. A model that predicts one frame from the last predicts the next frame well and then, fed its own predictions, coasts to a standstill within about a second. It isn’t that the motion is gone. The motion is there, shrunk toward the average, because hedging is what squared error rewards. Generating a whole block at once keeps the motion alive, but every place one block hands off to the next leaves a visible seam. A streaming formulation, where the model is trained on corrupted history so that its own mistakes look like noise it already knows how to remove, removed both problems at once.

The learned dynamics travel. Models trained on one kind of capture predict the direction and speed of joint movement about as well on a kind they never saw, whether the source is a studio camera, a depth sensor, or optical motion capture. What does not travel is the prior. The model trained on dance streams an elderly person’s walk flawlessly and confidently wrong, because it has never seen anyone move slowly. That is the cleanest statement of where things are: the architecture holds, and what it forecasts well is now a question of what it has seen.

What we are working on

A written form for movement. Language became something machines could work with once it had a script. Movement has none, so today it is either raw video, which is heavy and private, or a step count, which has thrown the movement away. We are building the layer in between: a way to write down what a body did that is compact enough to search years of footage, faithful enough to reconstruct the motion, and free of imagery by construction. The forecaster reads and writes it. We will say more once there is more to say.

Limits

Everything so far is from a single camera, so depth is compressed. The training corpora are dance and laboratory exercise, not clinical populations. Agreement with expert clinical judgment has not yet been measured. None of these is an architecture problem. All of them are data.

An earlier, more detailed technical overview from July 2026 remains available as a dated snapshot. It predates the current work.