Social 3D Pose Reconstruction
Refer to “PIPELINE PLAYGROUND > Install & Run”， and test the codes。
Raw multi-view videos, after segmentation, keypoint reconstruction and smoothing, obtain stable 3D pose coordinates.
Video: Multi-animal 3D pose tracking using appearance-based segmentation
Raw Video
The video comes from OBS Studio recorded 9-view camera video. The multi-camera images have been time-synchronized, with an error of 1 frame or less, which can be ignored.
Segmentation
Using the Mask-RCNN model, the identity of black and white rats is recognized, and the pixel area is segmented.
Keypoint Reconstruction
Execute DANNCE keypoint prediction + neural network smoothing to obtain 3D pose.
