StreamPI Adds Efficient Temporal Memory to Vision-Language-Action Robots

StreamPI gives a single-frame VLA a streaming memory

Researchers from the University of Hong Kong and ACE Robotics released the StreamPI paper on arXiv on August 26, 2026. StreamPI is a temporal modeling framework for vision-language-action (VLA) robot policies designed to add memory and multi-frame spatial evidence to a pretrained single-frame model without adding new parameters.

The work targets a practical weakness of VLA systems such as pi0.5: when a policy reasons from only the current observation, it can lose information about what happened moments earlier. A straightforward multi-frame window can preserve history, but repeatedly re-encoding the full window adds compute and latency.

StreamPI instead turns each visual-observation and language-instruction pair into a temporal unit. It uses bidirectional attention inside each unit for visual-language grounding, causal attention across units for history, and a KV cache so previously encoded observations do not need to be processed again at every step.

What the researchers report

In the project repository, the authors report gains over their pi0.5 single-frame baseline on four real-robot manipulation tasks covering precise perception and memory dependence. The reported improvements range from 26.7 to 36.6 percentage points across those tasks.

On LIBERO, the authors report a 98.3% average success rate for the five-frame StreamPI setting, compared with 96.9% for their pi0.5 baseline. On CALVIN, StreamPI reports an average of 4.547 consecutively completed tasks versus 4.313 for pi0.5.

The efficiency result is also notable. In the authors' RTX 4090 latency test, increasing the streaming context from one to five frames raised mean inference latency from 94.4 ms to 103.6 ms—an additional 9.2 ms.

These are author-reported research results, not independent benchmark replications. Real-world performance will depend on hardware, task design, training data and deployment conditions.

Why the architecture matters

StreamPI's key idea is to preserve the pretrained VLA representation rather than bolt on a separate memory module. The temporal behavior comes from extending the token sequence and changing the attention pattern, so the model can inherit the pretrained pi0.5 weights.

The method also uses randomized frame intervals and temporal masking during training to expose the policy to variable observation timing. That is aimed at a common robotics problem: real sensors and action loops are asynchronous rather than perfectly synchronized.

During streaming inference, the cache keeps prior temporal units while only the newly arrived unit is encoded. This makes temporal reasoning more practical for robot control loops where repeatedly processing the full video history would be expensive.

An important reproducibility caveat

The public project repository contains implementation code and multi-node JAX training support, but its own README notes a reproducibility mismatch: the checked-in LIBERO and CALVIN configurations currently set enable_jitter=False, while the paper's reported random-interval experiments sample intervals from a range. The authors state that exact reproduction requires aligning both the configuration and the data-sampling path with the paper protocol.

The repository's News section also says code and model weights are scheduled for August 30, 2026, even though implementation files are already visible publicly. For that reason, the trained model-weight release should still be treated as upcoming rather than assumed complete.

What is released versus upcoming

Released now: the paper, project documentation and a public implementation repository are available.

Upcoming: according to the authors' project page, the formal code and model-weight release is scheduled for August 30. Until those artifacts and checkpoints are confirmed, users should not assume the announced release package is complete.

Why this matters for physical AI

Many robot tasks require remembering where an object moved, combining evidence from multiple viewpoints or maintaining a goal while observations change. StreamPI suggests that part of this temporal capability can be obtained by reorganizing attention and caching rather than increasing parameter count.

If the results reproduce across broader robots and environments, the approach could be useful for lower-latency physical-AI systems that need short-term memory without continuously reprocessing a full video history.