PuzzleAvatar: Assembling 3D Avatars from Personal Albums
SIGGRAPH Asia 2024 · ACM TOG
本报告涉及的工作
Publications
UP2You: Fast Reconstruction of Yourself from Unconstrained Photo Collections
Zeyu Cai, Ziyang Li, Xiaoben Li, Boqian Li, Zeyu Wang, Zhenyu Zhang, Yuliang Xiu
ICLR 2026
ETCH: Generalizing Body Fitting to Clothed Humans via Equivariant Tightness
Boqian Li, Haiwen Feng, Zeyu Cai, Michael J. Black, Yuliang Xiu
ICCV 2025 · Highlight
ETCH-X: Robustify Expressive Body Fitting to Clothed Humans with Composable Datasets
Xiaoben Li, Jingyi Wu, Zeyu Cai, Siyuan Yu, Boqian Li, Yuliang Xiu
ECCV 2026
OmniFit: Multi-modal 3D Body Fitting via Scale-agnostic Dense Landmark Prediction
Zeyu Cai, Yuliang Xiu, Renke Wang, Zhijing Shao, Xiaoben Li, Siyuan Yu, Chao Xu, Yang Liu, Baigui Sun, Jian Yang, Zhenyu Zhang
ECCV 2026
GaussiAnimate: Rig Animatable Categories with Level of Dynamics
Jiaxin Wang, Dongxin Lyu, Zeyu Cai, Zhiyang Dou, Cheng Lin, Anpei Chen, Yuliang Xiu
SIGGRAPH Asia 2026
Human3R: Everyone Everywhere All at Once
Yue Chen, Xingyu Chen, Yuxuan Xue, Anpei Chen, Yuliang Xiu, Gerard Pons-Moll
ICLR 2026
DirtyMoCap: Robust Motion Capture from Unconstrained Markers
Long Wang, Shuting Zhao, Shen Yan, Siyuan Yu, Xiaoben Li, Zeyu Cai, Yumeng Hou, Yuliang Xiu
SIGGRAPH Asia 2026
Pose Flow: Efficient Online Pose Tracking
Yuliang Xiu, Jiefeng Li, Haoyu Wang, Yinghong Fang, Cewu Lu
BMVC 2018
SSDR: Smooth Skinning Decomposition with Rigid Bones
Binh Huy Le, Zhigang Deng
SIGGRAPH Asia 2012 · ACM TOG
论文与项目页面可点击标题访问 · SSDR 为方法引用
GaussiAnimate
Rig Animatable Categories with Level of Dynamics
Jiaxin Wang¹ Dongxin Lyu¹ Zeyu Cai¹˒² Zhiyang Dou³ Cheng Lin⁴ Anpei Chen¹ Yuliang Xiu¹
¹ Westlake University ² Nanjing University ³ The University of Hong Kong ⁴ Macau University of Science and Technology
From Captured Motion to Editable Motion
Motivation
INPUT
A dynamic 3D asset
Temporally consistent Gaussian or mesh sequences.
GOAL
New skeletal motions
Intuitive control with non-rigid surface deformation.
A captured sequence records motion. An animation rig lets us change it.
The Control–Deformation Gap
Motivation
Reconstructed motionSMPL KNN LBS
Articulation is only
part of the motion.
A skeleton provides clear controls, but simple skinning misses clothing deformation.
The surface must respond to motion beyond rigid joint transforms.
How can we retain skeletal control and deformation fidelity in one rig?
Skelebones
Representation
Inner skeleton
Intuitive articulation
Outer free-form bones
Local surface deformation
Animated surface
Geometry and appearance
An inner skeleton controls articulation. Outer bones approximate non-rigid surface motion.
Outer Bones from Surface Motion
Rig Construction
01
Explain the motion
Fit local rigid motion to a region.
02
Split where error is high
Initialize a new cluster from poorly explained motion.
03
Refine skinning
SSDR jointly refines weights and bone transforms.
Dense deformation becomes a compact set of free-form bone transformations.
An Inner Skeleton from Skinning Evidence
Rig Construction
Across Categories and Representations
Results
One representation,
different assets.
Clothed humans
Reconstructed 4D Gaussians
Garments and animals
Temporally consistent meshes
Each asset has its own rig and motion database.
Applicable to articulated assets with a deformable exterior.
Smooth Skinning Decomposition
SSDR
把示例形变分解为刚性骨骼变换 B 与稀疏蒙皮权重 W
B 骨骼变换 {R, T}
W 稀疏、非负、归一化权重
固定 B 更新 W ↔ 固定 W 更新 B
Results and Applications
SSDR
DirtyMoCap
Robust Motion Capture from Unconstrained Markers
and Construction of a Kung Fu Motion Dataset
and Construction of a Kung Fu Motion Dataset
01/16
Commercial MoCap System
Background
Vicon
Magesnik
A marker configuration specifies where markers are placed on the body.
Unknown or incompatible marker configurations prevent direct use of standard automatic labeling and skeletal-solving pipelines.
03/16
From Controlled Capture to Unconstrained Markers
Background
Placement errors and occlusion introduce marker noise
Jitter · Dropout · Ghost
04/16
From Controlled Capture to Unconstrained Markers
Background
Markers that reappear after occlusion may also receive new labels, breaking temporal consistency.
Motion reconstruction must also accommodate different marker configurations across captures.
Unconstrained Markers: dirty and unordered markers under different unknown marker configurations.
05/16
Raw Kung Fu Motion Captures
Motivation
134
motion sequences
motion sequences
22
martial-arts styles
martial-arts styles
~180
minutes of motion
minutes of motion
We obtained raw marker recordings of traditional Chinese martial arts, captured between 2013 and 2022.
Marker configuration metadata, marker identities, and marker-to-body correspondences are unavailable.
Our goal: Recover usable human motion from these raw recordings
and construct a standardized Kung Fu motion dataset.
and construct a standardized Kung Fu motion dataset.
07/16
DirtyMoCap
Contributions
Unconstrained Markers
Proxy Anchors
Human Motion
Auto-rotating · Drag to take control
52 joint anchors61 surface anchors
Canonical Proxy Anchors
Fixed anchor-to-body correspondences08/16
Anchor Initialization
Method
A coarse-to-fine strategy for the first frame
6 anatomical anchors
113 proxy anchors
● Markers ● Joint anchors ● Surface anchors
1
Coarse localization
Center markers by their median.
Predict pelvis, neck, hips, and shoulders.
Predict pelvis, neck, hips, and shoulders.
2
Body-centric canonicalization
Use these six anchors to remove
global rotation and translation.
global rotation and translation.
3
Fine anchor prediction
Predict 52 joint + 61 surface anchors,
then transform back to original coordinates.
then transform back to original coordinates.
Each prediction stage uses a Point Transformer
and a decoder with learnable anchor queries.
and a decoder with learnable anchor queries.
Why surface anchors? Resolve bone-axis twist ambiguity and constrain body shape.
10/16
Differentiable and Learnable Solver
Method
Tracked anchorsA1:TKnown body correspondences
Weight predictorΨInput-dependent weighting policy
ObservationEobs wcAnchor confidence
SmoothnessEsmooth wsTemporal coherence
Pose priorEprior wpDPoser-X
Differentiable
Gauss–Newton SolverJointly optimizeθ1:T pose · τ1:T translation · β shape
Gauss–Newton SolverJointly optimizeθ1:T pose · τ1:T translation · β shape
Recovered SMPL-H motion
Colored anchors constrain the motion.
Colored anchors constrain the motion.
End-to-end parameter supervision. Final pose, translation, and shape errors backpropagate through unrolled Gauss–Newton iterations to train Ψ.
12/16
Results on Synthetic Data
Experiments
Representative reconstructions across marker configurations and corruption types.
Each clip shows input markers, the DirtyMoCap reconstruction, and ground truth.
14/16
HKMALA-Motion Dataset
Dataset
134段动作sequences
22种拳术styles
≈180分钟minutes
8.4年采集跨度years · 2013–2022
≈8,500帧 / 每段frames / sequence
代表拳种 Selected styles
洪拳Hung Kuen
白眉拳Pak Mei
咏春拳Wing Chun
蔡李佛拳Choy Lee Fut
福建白鹤拳Fujian White Crane
螳螂拳Praying Mantis
鹰爪拳Eagle Claw
太极拳Taijiquan
八卦掌Baguazhang
通背拳Tongbeiquan
Historical marker captures → reusable SMPL-H motion
鸿胜蔡李佛 · Hung Sing Choy Lee Fut
杨式太极 · Yang-style Taijiquan
鹰爪翻子门 · Eagle Claw Fanzi
咏春拳 · Wing Chun
15/16
16/16