Loading Experience...
T-Rex

Tactile-Reactive
Dexterous Hand

High-Frequency Physical Interaction

Scroll to explore

T-Rex Tactile-Reactive
Dexterous Manipulation

1UC Berkeley    2NVIDIA    3Stanford    4Panasonic    5La Sapienza University    6ItalAI
*Equal Contribution
T-Rex teaser figure

Abstract

The ability to react dynamically to tactile signals has long been considered crucial to agile human-level dexterity. Yet contemporary learning-based VLAs for robotic manipulation generally either overlook the tactile modality or are limited to encoders with static cues, in part due to the scarcity of diverse training data and standardized evaluation, architectural constraints in current Vision-Language-Action (VLA) models, and limitations of static tactile encoders. In this paper, we push the frontier of tactile-reactive manipulation addressing all of these limitations. We propose a large-scale, 100-hour tactile-rich dataset collected via a novel, data-efficient recipe that prioritizes elementary motor primitives. To effectively exploit naturally high-frequency touch signals without sacrificing the existing capabilities of existing VLAs, we introduce a variable-rate Mix-of-Transformer (MoT) architecture equipped with a novel temporal tactile VQ-VAE encoder. We demonstrate the effectiveness of tactile-reactive policies on 12 manipulation tasks requiring delicate force control, deformable object manipulation, achieving over 30% higher average success rate than the strongest baseline.

T-Rex model architecture: latent, action, and tactile experts in a Mixture-of-Transformer backbone

Demonstration

Real-world autonomous policy rollouts on our bimanual dexterous platform

Results

T-Rex outperforms the strongest baseline by +30 absolute success-rate points on average across 12 real-world tactile-reactive manipulation tasks — spanning force-sensitive contact, deformation-aware manipulation, and bimanual coordination.

Average Success Rate (%) — 12 tactile-reactive tasks 0 25 50 75 100 Success rate (%) ViTacFormer 3 RDP 6 π0.5 + tactile 6 Tactile-VLA 15 π0.5 17 EgoScale 35 T-Rex (Ours) 65

Per-Task Success Rate (%)

Method Flip
Page
Transfer
Egg
Wipe
Plate
Apply
Paste
Split
Cup
Sort
Mahjong
Open
Lock
Refill
Tablet
Acid-Base
Neut.
Extract
Card
Deal
Poker
Screw
Bulb
Avg
ViTacFormer9041470002213
RDP128182692001276
Tactile-VLA381424021278094111815
EgoScale68443438333619124341281835
π0.53617281318325124891117
π0.5 + tactile892724142073006
T-Rex (Ours)96756966786547417670573565

Each cell is the success rate (%) averaged over 16 evaluation rollouts; the right-most column is the macro-average across all 12 tasks.

Citation to be released

The design of this project page was adapted from the Doorman Humanoid project page template — many thanks to the original authors. If you would like to reuse this T-Rex page as a starting point for your own project, please kindly link back to tactile-rex.github.io.