Cut robotic arm oscillation with Kyutai's REKI 2.4B model using register tokens for efficient visual memory
A new 2.4 billion parameter model uses register tokens to track recent visual observations, reducing oscillation in robotic arms during complex tasks.
Key points
- Kyutai released the REKI vision-language-action model with 2.4 billion parameters for robot manipulation tasks.
- The system uses 32 register tokens to maintain a cheap memory of recent visual frames without increasing computational load.
- REKI achieved top scores on LIBERO-Plus and RoboTwin Clean2Rand benchmarks among models pretrained on public data.
Kyutai has released a new artificial intelligence model called REKI that helps robots remember their recent visual observations to perform tasks more smoothly and accurately. According to MarkTechPost, the 2.4 billion parameter model is designed specifically for robot manipulation and is now available with all its weights live on Hugging Face. The system allows robots to process visual instructions and physical actions while maintaining an efficient record of what was seen just moments before.
In plain English
Most robot AI models look only at the current camera image to decide what to do next. This short-term blindness can cause robots to repeat mistakes or shake between positions because they forget their immediate past. REKI solves this by adding a small, fast memory system that tracks the last few seconds of visual activity. This memory does not slow down the robot or require expensive hardware upgrades. Instead, it lets the robot understand context by remembering what it saw a second ago, which mitigates confusing loops in movement.
The background
Kyutai developed REKI in collaboration with Inria and École des Ponts, according to MarkTechPost. The model combines a Qwen3.5-2B vision-language backbone with a smaller action expert that generates movement commands. Unlike previous approaches that added heavy memory modules or retrained entire networks, REKI places memory inside the action expert using 32 learned register tokens. These tokens process visual data from the current frame while also looking back at cached registers from previous frames. This design keeps the memory cost extremely low, adding only a fraction of a millisecond to each action calculation.
What changes now
The release makes advanced robot manipulation accessible through publicly pretrained weights, as noted by MarkTechPost. REKI was pretrained exclusively on public data, making it a transparent option for researchers and developers. It demonstrated strong performance on out-of-distribution benchmarks, outperforming larger models in several categories. On the LIBERO-Plus benchmark, it achieved a 90.4 percent success rate, surpassing Qwen-RobotManip and π0.5. However, it scored lower on RoboCasa365, indicating that performance varies depending on the specific task environment and complexity.
Key terms explained
Teams usually evaluate new robot models by checking how they handle unfamiliar tasks without retraining. When testing such systems, administrators should verify that the model retains context over short timeframes to prevent repetitive errors. It is important to ensure the memory mechanism does not introduce significant latency during real-time operation. Developers should also confirm that the model relies on standard hardware components rather than specialized accelerators, which ensures broader compatibility across different robotic platforms and reduces deployment costs for experimental setups.
What to do and how to stay safe: Kyutai
- Review the model card and project page to understand the specific benchmarks and limitations before deploying the model in production environments.
- Verify that your hardware supports BF16 precision and has sufficient GPU memory to handle the 2.4 billion parameter model efficiently.
- Test the model on your specific robot hardware to ensure the action chunks align with your mechanical constraints and safety protocols.
- Monitor performance on out-of-distribution tasks to identify scenarios where the model may struggle, such as complex household environments.
General security guidance from the Vector Update newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
How does REKI differ from other vision-language-action models?
REKI uses 32 register tokens within its action expert to maintain a cheap memory of recent frames, whereas most models act on the current frame alone or use costly memory methods.
What benchmarks did REKI perform well on?
According to MarkTechPost, REKI achieved the best scores among public-data pretrained models on LIBERO-Plus and RoboTwin Clean2Rand, with a 90.4 percent score on the former.
Is REKI available for public use?
Yes, all weights for the REKI model are live on Hugging Face, and it was pretrained exclusively on public data, according to the source material.
