Rethinking Tactile Representation Learning with Material Priors

Weihao Xia    Chenliang Zhou    Cengiz Öztireli

University of Cambridge

[Paper]      [Code]

Method

The Overall Framework

RETRO is a tactile representation learning framework that integrates material-specific knowledge as priors. It comprises two components: (a) learning a material prior, and (b) tactile representation learning with the material prior.

Our method achieves superior or competitive performance across multiple datasets and applications with various touch sensors, including tasks like material classification and cross-modal retrieval.

Touch-aided Material Editing

We introduce a touch-aided material editing task, where tactile inputs are used to retrieve the most relevant references from a high-resolution and diverse material library to guide the editing process. Our method outperforms existing alternative modality-driven material editing approaches—such as text-guided or image-referenced methods—in terms of photorealism and faithfulness, highlighting the potential of tactile signals in real-world, material-centric creative design.

Results

Touch-based Material Retrieval
Touch-aided Material Editing

Acknowledgements

This work was supported by the Engineering and Physical Sciences Research Council [grant EP/W523835/1] and a UKRI Future Leaders Fellowship [grant number G104084].

BibTex

@inproceedings{xia2025retro,
  title={RETRO: REthinking Tactile Representation Learning with Material PriOrs},
  author={Xia, Weihao and Zhou, Chenliang and Öztireli, Cengiz},
  journal={British Machine Vision Conference (BMVC)},
  year={2025}
}