alexzhang598 commited on
Commit
029dfc7
·
verified ·
1 Parent(s): ac4a823

Add model card

Browse files
Files changed (1) hide show
  1. README.md +47 -0
README.md ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ tags:
4
+ - text-to-motion
5
+ - bimanual-hands
6
+ - diffusion
7
+ library_name: pytorch
8
+ ---
9
+
10
+ # HandX — Diffusion Text-to-Motion Checkpoints
11
+
12
+ Diffusion checkpoints for **HandX: Scaling Bimanual Motion and Interaction Generation** (CVPR 2026).
13
+ They generate two-hand motion from text (separate text branches for the left hand, right hand,
14
+ and their interaction), using an MDM-style diffusion model with a frozen T5-base text encoder.
15
+
16
+ - 📄 Paper: https://arxiv.org/abs/2603.28766
17
+ - 📦 Dataset: https://huggingface.co/datasets/alexzhang598/HandX
18
+
19
+ ## Checkpoints
20
+
21
+ | Folder | Decoder layers | latent_dim |
22
+ |--------|----------------|------------|
23
+ | `layers4` | 4 | 256 |
24
+ | `layers8` | 8 | 512 |
25
+ | `layers12` | 12 | 512 (best model in the paper) |
26
+
27
+ Each folder has `model.pt` (weights) and `config.yaml`.
28
+
29
+ ## Loading
30
+
31
+ ```python
32
+ import torch
33
+ from huggingface_hub import hf_hub_download
34
+ from omegaconf import OmegaConf
35
+ # run from the `diffusion/` directory of the HandX repo
36
+ from src.diffusion.utils.model_utils import create_model_and_diffusion
37
+
38
+ variant = "layers12"
39
+ cfg = OmegaConf.load(hf_hub_download("alexzhang598/HandX-diffusion", f"{variant}/config.yaml"))
40
+ model, diffusion = create_model_and_diffusion(cfg.model)
41
+ sd = torch.load(hf_hub_download("alexzhang598/HandX-diffusion", f"{variant}/model.pt"),
42
+ map_location="cpu")["state_dict"]
43
+ model.load_state_dict(sd, strict=False) # missing keys are the frozen T5 encoder (loaded from t5-base)
44
+ ```
45
+
46
+ The checkpoints load with a standard `load_state_dict(..., strict=False)`; the only missing keys are
47
+ the frozen T5 weights, restored from `t5-base` at construction.