hughie

Introduction#

This article provides a brief introduction to HQTrack.

HQTrack is a framework for high-performance video object tracking and segmentation.

Main Content#

1. Introduction to HQTrack#

HQTrack is a framework for high-performance video object tracking and segmentation. It can track multiple target objects simultaneously and output accurate object masks.

2. Using HQTrack#

1. Download the project: git clone https://github.com/jiawen-zhu/HQTrack.git

2. Install the required dependencies:

Make modifications to the versions of certain libraries based on the official requirements, otherwise the installation will not be successful:

• pytorch needs to be 1.10.0 or above to install DCNv3, and use the precompiled wheel file provided by the DCNv3 project for installation.

• gcc needs to be 7.3.0 or above for compiling the vot-toolkit framework.

# Create a virtual environment
conda create -n hqtrack python=3.8
conda activate hqtrack

# Install torch 1.12.0
conda install pytorch==1.12 torchvision cudatoolkit=11.3 -c pytorch

# Install HQ-SAM
cd segment_anything_hq
pip install -e .
pip install opencv-python pycocotools matplotlib onnxruntime onnx

# Install Pytorch-Correlation-extension
cd packages/Pytorch-Correlation-extension/
python setup.py install

# Install DCNv3
pip install DCNv3-1.0+cu113torch1.12.0-cp38-cp38-linux_x86_64.whl

# Install vot-toolkit framework
pip install vot-toolkit

# Install other dependencies
pip install easydict
pip install lmdb
pip install einops
pip install jpeg4py
pip install 'protobuf~=3.19.0'
conda install setuptools==58.0.4
pip install timm
pip install tb-nightly
pip install tensorboardx
pip install scikit-image
pip install rsa
pip install six
pip install pillow

3. Download the pre-trained models

• Download the VMOS model, extract it and place it in the result/default_InternT_MSDeAOTL_V2/YTB_DAV_VIP/ckpt/ directory of the project.
• Download the HQ-SAM model and place it in the segment_anything_hq/pretrained_model/ directory of the project.

4. Run the demo

Add the image sequence you want to process to demo/your_video, modify the demo_video parameter in the demo/demo.py script to your custom folder name, and run the following command to manually label the object to be tracked in the first frame, then press r to start processing the image sequence.

cd demo
python demo.py

3. Conclusion#

The project achieves high-precision object tracking mainly relying on HQ-SAM, which enables SAM to accurately segment any object while maintaining SAM's original design, efficiency, and zero-shot generalization ability. Only a small number of additional parameters and computations are introduced based on the pre-trained model weights of SAM.

Finally#

References:

Official Project

HQ-SAM

Disclaimer#

This article is for personal learning purposes only.
This article is synchronized with HBlog.