Where the work stands

We build native data models, trained on a domain’s own data and prompted with it, and human movement is the first domain. This is a plain account of what exists, what we learned building it, and what we are working on now. It leaves out most of the mechanism.

Language became something machines could work with once it had a script. Movement had none, so it was either raw video, which is heavy and private, or a step count, which has thrown the movement away. The layer in between now exists: a way to write down what a body did that is compact enough to keep years of it, faithful enough to read the motion back, and free of imagery by construction.

What exists

A written form for movement. In September it was something we were building; a clip of movement can now be written down and read back. At full fidelity the read-back is indistinguishable from the original by eye, and the written record is about a tenth the size of the keypoints it came from and a small fraction of a video. It has been checked clip by clip on five public corpora and on phone captures, and the same writing serves every camera, frame rate and source we have.

A language model that reads that writing. It is a plain next-word model, and given eight seconds of a clip it writes what comes next, one word at a time, for as long as asked.

A pipeline that takes ordinary video, from any camera, at any frame rate, and turns it into a normalized record of how the body moved. A forecaster that watches a few seconds of that record and streams a continuation, one frame at a time, for as long as you ask. It runs on a single consumer GPU, fully offline, and the phone capture that prompts it never has to leave the device.

The forecaster has been evaluated on held-out subjects from a public laboratory corpus and a public dance corpus, and on phone recordings of people the model had never seen, where the real future plays out alongside the forecast and serves as its own ground truth.

What we learned

Forecasting motion is mostly a fight against the model’s own output. A model that predicts one frame from the last predicts the next frame well and then, fed its own predictions, coasts to a standstill within about a second. It isn’t that the motion is gone. The motion is there, shrunk toward the average, because hedging is what squared error rewards. Generating a whole block at once keeps the motion alive, but every place one block hands off to the next leaves a visible seam. A streaming formulation, where the model is trained on corrupted history so that its own mistakes look like noise it already knows how to remove, removed both problems at once.

The learned dynamics travel. Models trained on one kind of capture predict the direction and speed of joint movement about as well on a kind they never saw, whether the source is a studio camera, a depth sensor, or optical motion capture. What does not travel is the prior. The model trained on dance streams an elderly person’s walk flawlessly and confidently wrong, because it has never seen anyone move slowly. That is the cleanest statement of where things are: the architecture holds, and what it forecasts well is now a question of what it has seen.

The written form changed one of those conclusions. In July we wrote that frame-by-frame forecasting does not extend, that it coasts to a standstill and rolling it forward only deepens the collapse. On the written form, a plain next-word model, the oldest kind of sequence model there is, continues held-out clips for twelve seconds at the recorded level of motion, on nine clips in ten within half to twice the real energy, with no seam at the handoff. The same model, asked for its single most likely next word every time instead of drawing one, freezes within a word or two, exactly as the old forecasters did. So the freeze was never a property of predicting one step at a time. It was a property of hedging, and a model that chooses rather than averages does not hedge.

The scoreboard stays honest. The best of ten written futures lands a third to a half closer to what really happened than a freeze-frame does, at one and two seconds; the average of the ten does not beat the freeze-frame, because averaging rewards blur and a written future is one definite guess. Where the model was never shown a kind of movement it still over-animates it, as before. On a clinical motion-capture corpus it had never seen it reads better than counting, but keeps little of the edge it has at home. The prior is still data.

What we are working on

The work now is on what reads and writes the written form. The first model is small and was trained on a few corpora; the next ones train on more, including the slow and deliberate movement the current ones have never seen.

Removing the imagery is not the same as removing the person. Two people taking the same step do not take it the same way, and that difference alone can identify someone. We will say more once there is more to say.

Limits

Everything so far is from a single camera, so depth is compressed, and the same movement seen from another camera is written differently. The training corpora are dance, fitness and laboratory exercise, not clinical populations. Agreement with expert clinical judgment has not yet been measured. A written future is one guess among many, so a single forecast should be read as a possibility, not a prediction. None of these is an architecture problem. All of them are data, or ways of reading what the model says.

An earlier, more detailed technical overview from July 2026 remains available as a dated snapshot. It predates the current work.