Skip to content

Robotics-STAR-Lab/DynamicPose

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

DynamicPose

DynamicPose: Real-time and Robust 6D Object Pose Tracking for Fast-Moving Cameras and Objects

Accepted to IROS 2025


Tingbang Liang1,2,*, Yixin Zeng1,*, JiaTong Xie1, Boyu Zhou3,†

1 School of Artificial Intelligence, Sun Yat-Sen University, Zhuhai, China.   
2 School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an, China.   
3 Department of Mechanical and Energy Engineering, Southern University of Science and Technology, Shenzhen, China.   
*Equal contribution. Listing order is random
Corresponding Authors

Paper Video Link
TOP

News

  • [30/09/2025]: Release part of the code for DynamicPose.
  • [16/06/2025]: DynamicPose is accepted to IROS 2025

Introduction

[IROS 2025] This repository maintains the implementation of "DynamicPose: Real-time and Robust 6D Object Pose Tracking for Fast-Moving Cameras and Objects".

Please kindly star ⭐ this project if it helps you. Thanks for your support! 💖

TOP

Demo

Real-world:

real-world-exp.mp4

Pose Tracking Comparison:

real_compare.mp4

Install

conda create -n dynamicpose python=3.9.19
conda activate dynamicpose
conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=11.8 -c pytorch -c nvidia
pip3 install pycryptodomex 
pip3 install python-gnupg
  • FoundationPose In this step, you should ensure that CUDA_11.8 has been installed.
// install eigen 3.4.0 [optional]
cd $HOME && wget -q https://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.gz && \
tar -xzf eigen-3.4.0.tar.gz && \
cd eigen-3.4.0 && mkdir build && cd build
cmake .. -Wno-dev -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-std=c++14 ..
sudo make install
cd $HOME && rm -rf eigen-3.4.0 eigen-3.4.0.tar.gz

// use conda to install the Python environment
cd ${YOUR_WORKSPACE_PATH}/src/Dynamic_6D/object_pose_estimator/include/FoundationPose
conda activate dynamicpose
python -m pip install -r requirements.txt
python -m pip install --quiet --no-cache-dir git+https://github.com/NVlabs/nvdiffrast.git
python -m pip install --quiet --no-cache-dir kaolin==0.15.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.1.0_cu118.html
python -m pip install "git+https://github.com/facebookresearch/pytorch3d.git"
CMAKE_PREFIX_PATH=$CONDA_PREFIX/lib/python3.9/site-packages/pybind11/share/cmake/pybind11 bash build_all_conda.sh

[1]. Download all network weights from here and put them under the folder weights/. For the refiner, you will need 2023-10-28-18-33-37. For scorer, you will need 2024-01-11-20-02-45.

[2]. Download demo data and extract them under the folder demo_data/

right now, you can check if FoundationPose is ready by:

conda activate dynamicpose
cd ${YOUR_WORKSPACE_PATH}/src/Dynamic_6D/object_pose_estimator/include/FoundationPose
python run_demo.py
  • MixFormerV2
cd ${YOUR_WORKSPACE_PATH}/src/Dynamic_6D/object_pose_estimator/include/MixFormerV2
conda activate dynamicpose
pip install tensorboardX jpeg4py lmdb easydict transformers empy
python tracking/create_default_local_file.py --workspace_dir . --data_dir ./data --save_dir .
  • Grounding DINO
cd ${YOUR_WORKSPACE_PATH}/src/Dynamic_6D/object_pose_estimator/config/gdino_checkpoints/
conda activate dynamicpose
huggingface-cli download IDEA-Research/grounding-dino-tiny --local-dir grounding-dino-tiny
  • Potential bug fixes [Optional] Using rospy in python=3.9, there are some potential bug:

    1. version of libffi:
    libp11-kit.so.0: undefined symbol: ffi_type_pointer, version LIBFFI_BASE_7.0
    

    Solution:

    BASE_ENV_PATH=$(conda info | grep "base environment" | awk '{print $4}')
    cd $BASE_ENV_PATH/envs/dynamicpose/lib
    mv ./libffi.so.7 ./libffi.so.7.bak
    sudo ln -s /lib/x86_64-linux-gnu/libffi.so.7.1.0 libffi.so.7
    

Run Demo

  • Download demo bag and extract them under the folder 'demo_bag/'

  • Run Pose Estimator

    cd ${YOUR_WORKSPACE_PATH}/
    catkin_make
    roslaunch object_pose_estimator rviz.launch
    roslaunch object_pose_estimator run_demo.launch
    

    Execution Example:

    2025-09-30.22-53-38.mp4

Acknowledge


Maintenance

For any technical issues, please contact:


Citation

@article{liang2025dynamicpose,
  title={DynamicPose: Real-time and Robust 6D Object Pose Tracking for Fast-Moving Cameras and Objects},
  author={Liang, Tingbang and Zeng, Yixin and Xie, Jiatong and Zhou, Boyu},
  journal={arXiv preprint arXiv:2508.11950},
  year={2025}
}

About

[IROS 2025] DynamicPose: Real-time and Robust 6D Object Pose Tracking for Fast-Moving Cameras and Objects

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published