Uncertainty-Aware Testing-Time Optimization for 3D Human Pose Estimation
In this paper, we propose an Uncertainty-Aware testing-time Optimization (UAO) framework for 3D human pose estimation. During the training process, we propose the GUMLP to estimate 3D results and uncertainty values for each joint. For test-time optimization, our UAO framework freezes the pre-trained network parameters and optimizes a latent state initialized by the input 2D pose. To constrain the optimization direction in both 2D and 3D spaces, projection and uncertainty constraints are applied. Extensive experiments show that our approach achieves state-of-the-art performance on two popular datasets
Feb 15, 2024
Skeleton-in-context: Unified skeleton sequence modeling with in-context learning
In this work, we propose the Skeleton-in-Context, designed to process multiple skeleton-base tasks simultaneously after just one training time. Specifically, we build a novel skeleton-based in-context benchmark covering various tasks. In particular, we propose skeleton prompts composed of TGP and TUP, which solve the overfitting problem of skeleton sequence data trained under the training framework commonly applied in previous 2D and 3D in-context models. Besides, we demonstrate that our model can generalize to different datasets and new tasks, such as motion completion. We hope our research builds the first step in the exploration of in-context learning for skeleton-based sequences, which paves the way for further research in this area.
Dec 15, 2023
Co-Evolution of Pose and Mesh for 3D Human Body Estimation from Video
This paper proposes the Pose and Mesh Co-Evolution network (PMCE), a new two-stage pose-to-mesh framework for recovering 3D human mesh from a monocular video. PMCE frst estimates 3D human pose motion in terms of spatial and temporal domains, then performs image-guided pose and mesh interactions by our proposed AdaLN that injects body shape information while preserving their spatial structure. Extensive experiments on popular datasets show that PMCE outperforms state-of-the-art methods in both perframe accuracy and temporal consistency. We hope that our approach will spark further research in 3D human motion estimation considering both pose and shape consistency.
Oct 2, 2023
PCLoss: Fashion landmark estimation with position constraint loss
In this paper, we design a Position Constraint Loss (PCLoss) for fashion landmark estimation, which incorporates the position correlation into landmark estimation models. Specifically, the PCLoss adds a regular term for each landmark to regularize their relative positions. Compared with other alternatives, our PCLoss effectively mitigates the outliers and duplicate detection problems without modifying existing CNN architectures. In addition, our skeleton-like optimization method further strengthens the position constraints between landmarks. The proposed method can be applied to both regression and heatmap based methods and it provides a novel perspective towards position relation learning in key point estimation tasks. Extensive experimental results on three challenging datasets, DeepFashion, FLD and FashionAI, demonstrate that our method outperforms other state-of-the-art methods. The experiment on COCO 2017 shows the potential applications of PCLoss for other key point estimation tasks, which can be explored more in future work.
Oct 1, 2021