TacClaw-InfinityA Large-Scale Vision-Tactile-Language-Action Dataset
for Dexterous Manipulation

Fucai Zhu^1,3,* Xidong Zhang^1,3,* Yichi Zhang^1,* Yipan Wang⁵ Fubo Wang³ Zibin Wu⁶ Kaiqi Li⁷ Jianhao Jiao⁸ Shihan Lu⁹ Di Hu¹⁰ Mengdi Xu¹¹ Siyu Zhu¹² Xiaoxiao Long² Yao Yao² Xun Cao² Zhuo Xu¹³ Koichi Hashimoto⁴ Qifeng Chen¹⁴ Zhe Li⁷ Jianfei Yang⁷ Haofei Lu¹⁵ Danica Kragic¹⁵ Shuicheng Yan¹⁶ Jianghua Duan¹ Michael Yu Wang^1,3 Weihao Yuan^1,2,✉

¹Daimon Robotics
²Nanjing University
³Great Bay University
⁴Tohoku University
⁵Southern University of Science and Technology
⁶The Hong Kong University of Science and Technology (Guangzhou)
⁷Nanyang Technological University
⁸University College London
⁹Northwest University
¹⁰Renmin University of China
¹¹Tsinghua University
¹²Fudan University
¹³DeepMind
¹⁴The Hong Kong University of Science and Technology
¹⁵Royal Institute of Technology
¹⁶National University of Singapore

^* Equal Contributions Corresponding Author

Paper PDF GitHub Coming soon ModelScope Coming soon Hugging Face Coming soon

TacClaw-Infinity: a worldwide collaboration. 2 min 15 sec

LARGE SCALE
200+ Devices
1,000+ Tasks
10,000+ Hours
1 Million+ Episodes
1 Billion+ Frames

HIGH DIVERSITY
20+ Organizations
50+ Locations Worldwide
Homes · Labs · Hotels
Industrial · Retail
Outdoor Environments

OMNI MODALITY
High-Resolution Tactile
Multi-View Vision
Structured Language
Accurate Action

Abstract

Large-scale robot learning increasingly relies on diverse human demonstrations, yet tactile manipulation data remain difficult to scale, and existing multimodal datasets often lack consistent quality control and structured supervision. We present TacClaw-Infinity, a large-scale structured Vision-Tactile-Language-Action (VTLA) dataset for contact-rich manipulation, collected with TacClaw, a tactile-enabled UMI-style handheld interface that records synchronized visual observations, fingertip tactile signals, and end-effector trajectories without requiring a target robot during data collection. TacClaw-Infinity is built through distributed collection across homes, laboratories, hotels, industrial workplaces, retail stores, and outdoor environments around the world, including tasks contributed by academic and industrial collaborators. Together, these efforts yield 10,000+ hours, 1 million+ episodes, and 1 billion+ frames spanning 1,000+ task types. We develop a scalable pipeline for trajectory reconstruction, multimodal quality control, embodiment-aware filtering, and structured annotation, covering contact-centric segmentation, subtask description, and spatial grounding. We validate TacClaw-Infinity at both the data and model levels: At the data level, we assess the collection interface, the action-space compatibility of the curated demonstrations, and the structured annotations, while for the model level, we demonstrate cross-embodiment transfer across three heterogeneous robots without robot-specific demonstrations. The code and dataset will be made public.

A dataset built across places, people, and tasks

Explore the tasks, objects, vocabulary, and tactile signals captured through worldwide collection with TacClaw.

Dataset statistics showing the hierarchy of manipulation actions and objects, coverage from distributed collection, action and object vocabularies, and tactile feature diversity. — Dataset statistics: manipulation action–object distributions, multi-site versus single-site coverage, vocabulary frequencies, and tactile feature diversity. Shaded bands in the coverage comparison are 95% resampling intervals.

Beyond a single environment

Shared task protocols provide repeated demonstrations of common skills, while collector-defined tasks contribute a long tail of manipulation behaviors. Both single-hand and bimanual interactions are represented.

Four synchronized modalities

Episodes combine visual observations, high-resolution fingertip tactile signals, end-effector trajectories, and language descriptions. Contact events connect the temporal, semantic, and spatial annotations.

From human demonstrations to structured supervision

TacClaw enables demonstrations without a target robot during collection. A shared processing pipeline then reconstructs trajectories, checks robot executability, and uses touch to anchor annotations.

01
Collect with TacClaw
Wrist and egocentric views, gripper states, fingertip tactile signals, and end-effector motion capture how people interact with objects.
02
Curate trajectories
Reconstructed poses are evaluated for workspace reachability and manipulability on each target embodiment before policy training.
03
Anchor annotations in contact
Contact and release delimit interaction phases. The inter-finger contact point localizes the manipulated object across camera views.

Contact-anchored annotation: tactile contact and release define temporal segments, and a shared 3D contact point anchors object masks across camera views. — Contact-anchored structured annotation. Tactile events define segment boundaries and supply a physical prior for identifying the manipulated object across synchronized views.

TacClaw interface

Preserving the operator’s sense of contact

A short, rigid path from the fingers to the jaws lets operators feel and regulate contact during a demonstration.

In the paper’s blindfolded study of nine participants, the mean first-perceived contact force was 11.6 ± 3.3 gf with TacClaw, compared with 49.8 ± 12.0 gf for the UMI interface and 204.0 ± 58.2 gf for the finger-loop interface.

Blindfolded contact perception experiment with three handheld interfaces and a plot of the measured perception thresholds. — Lower thresholds indicate that lighter contact can be perceived. All nine participants had their lowest threshold with TacClaw.

Zero-robot cross-embodiment transfer

Policies trained on TacClaw demonstrations transfer to Tianji, RealMan, and DM-Sparky without collecting robot-specific demonstrations. We evaluate cup stacking, bottle sorting, and vase wiping, using 50 trials per task.

With tactile input, mean success reaches 74.0% for π_0.5 and 72.7% for StarVLA across the three tasks and robot embodiments.

Representative execution sequences for cup stacking, bottle sorting, and vase wiping across the evaluated robot embodiments. — Cup stacking tests long-horizon coordination; bottle sorting tests tactile-dependent state discrimination; vase wiping tests sustained contact regulation.

Success rates by task and embodiment

Success rates (%) reported in the manuscript
Policy	Embodiment	Cup stacking	Bottle sorting	Vase wiping	Average
π_0.5	Tianji	90	44	28	54.0
π_0.5	RealMan	86	42	16	48.0
π_0.5	DM-Sparky	82	44	18	48.0
π_0.5 + Tactile	Tianji	94	96	54	81.3
π_0.5 + Tactile	RealMan	80	76	38	64.7
π_0.5 + Tactile	DM-Sparky	100	92	36	76.0
StarVLA	Tianji	78	46	12	45.3
StarVLA	RealMan	78	38	8	41.3
StarVLA	DM-Sparky	80	54	20	51.3
StarVLA + Tactile	Tianji	70	94	60	74.7
StarVLA + Tactile	RealMan	76	84	58	72.7
StarVLA + Tactile	DM-Sparky	84	90	38	70.7

Citation

arXiv:2610.20320 · 2026

@misc{zhu2026tacclawinfinity,
  title  = {TacClaw-Infinity: A Large-Scale Vision-Tactile-Language-Action Dataset for Dexterous Manipulation},
  author = {Zhu, Fucai and Zhang, Xidong and Zhang, Yichi and Wang, Yipan and Wang, Fubo and
            Wu, Zibin and Li, Kaiqi and Jiao, Jianhao and Lu, Shihan and Hu, Di and Xu, Mengdi and
            Zhu, Siyu and Long, Xiaoxiao and Yao, Yao and Cao, Xun and Xu, Zhuo and
            Hashimoto, Koichi and Chen, Qifeng and Li, Zhe and Yang, Jianfei and Lu, Haofei and
            Kragic, Danica and Yan, Shuicheng and Duan, Jianghua and Wang, Michael Yu and
            Yuan, Weihao},
  year   = {2026},
  eprint = {2610.20320},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url    = {https://arxiv.org/abs/2610.20320}
}

TacClaw-InfinityA Large-Scale Vision-Tactile-Language-Action Dataset for Dexterous Manipulation

LARGE SCALE

HIGH DIVERSITY

OMNI MODALITY