everybody dance Openpose GAN Pose2vid tensorflow实现 人体姿态迁移 动作传递 人物姿态生成

 

实现效果

 

人物freestyle 动作

 

teacher 动作传递给student， 

按照teacher动作生成student，

 

 

 

 

 

 

 

其他的实例效果

 

 

 

只训练了一小时，有时间多训练GAN的参数，生成的人物会效果更好。

 

有时间整理好代码，分享。

 

 

 

 

 

Dance 实例

We train and evaluate on Ubuntu 16.04, so if you don't have linux environment, you can set nThreads=0 in EverybodyDanceNow_reproduce_pytorch/src/config/train_opt.py.

 

 

 

 

 

 

 

 

 

 

 

 

1 运行 make_source

 

 

 

 

Images  存储成图片

 

人脸GAN用

Test_head_ori  头部位置

 

make label images for pix2pix'''

Test_img  

Test_label_ori  姿态骨架位置

 

 

 

2 运行 make_target.py

 

Put target video mv.mp4 in ./data/target/ and run make_target.py, pose.npy will save in ./data/target/, which contain the coordinate of faces (will use in step6).

 

Images  存储图片

'''make label images for pix2pix'''

head_img

脸部特征

train_img

特征点和整体图

 

train_label

姿态骨架图

 

Pose.npy

  包含脸部定位

 

3  运行 train_pose2vid.py

 

 

• Run Run train_pose2vid.py and check loss and full training process in ./checkpoints/
• If you break the traning and want to continue last training, set load_pretrain = './checkpoints/target/ in ./src/config/train_opt.py

• and check loss and full training process in ./checkpoints/
• If you break the traning and want to continue last training, set load_pretrain = './checkpoints/target/ in ./src/config/train_opt.py

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

4 运行normalization.py

 

Run normalization.py rescale the label images, you can use two sample images from ./data/target/train/train_label/ and ./data/source/test_label_ori/ to complete normalization between two skeleton size

 

 

 

5运行 transfer.py

 

 

Run transfer.py and get results in ./result

 

 

 

 

 

def create_label(shape, joint_list, person_to_joint_assoc):

    label = np.zeros(shape, dtype=np.uint8)

    cord_list = []

    for limb_type in range(17):

        for person_joint_info in person_to_joint_assoc:

            joint_indices = person_joint_info[joint_to_limb_heatmap_relationship[limb_type]].astype(int)

            if -1 in joint_indices:

                continue

            joint_coords = joint_list[joint_indices, :2]

            coords_center = tuple(np.round(np.mean(joint_coords, 0)).astype(int))

            cord_list.append(joint_coords[0])

            limb_dir = joint_coords[0, :] - joint_coords[1, :]

            limb_length = np.linalg.norm(limb_dir)

            angle = math.degrees(math.atan2(limb_dir[1], limb_dir[0]))

            polygon = cv2.ellipse2Poly(coords_center, (int(limb_length / 2), 4), int(angle), 0, 360, 1)

            cv2.fillConvexPoly(label, polygon, limb_type+1)

    return label,cord_list

 

 

 

 

 

 

Pytorch pose

https://github.com/tensorboy/pytorch_Realtime_Multi-Person_Pose_Estimation/tree/681d16fa6eac64d8828affa477af78dd358381d2

 

 

 

pix2pix

https://github.com/NVIDIA/pix2pixHD/tree/20687df85d30e6fff5aafb29b7981923da9fd02f

 

 

 

 

 

 

 

 

6 运行 ./face_enhancer/prepare.py

 

Run ./face_enhancer/prepare.py and check the results in ./data/face/test_sync and ./data/face/test_real.

 

Test_real

test_sync

 

 

 

7 运行./face_enhancer/main.py

 

 

优化脸部

 

 

Run ./face_enhancer/main.py train face enhancer and run./face_enhancer/enhance.py to gain results

训练

 

 

 原图---------------------------------------------------------》训练优化前-----------------------------------------------------》训练优化后脸部

 

Logs

This is comparision in original (left), generated image before face enhancement (median) and after enhancement (right). FaceGAN can learn the residual error between the real picture and the generated picture faces.

 

 

 原图---------------------------------------------------------》训练优化前-----------------------------------------------------》训练优化后脸部 

 

 

 原图---------------------------------------------------------》训练优化前-----------------------------------------------------》训练优化后脸部 

 

8 运行 ./face_enhancer/enhance.py

存储结果

 

 

 

 

9 运行 make_gif.py

Run make_gif.py and make result pictures to gif picture

pip install -U scikit-image

 

安装没有

需要copy文件到python目录下