Use mediapipe to find multiple person 3d keypoints

mediapipe_multiple_person.py

import cv2
import numpy as np
import mediapipe as mp
from mediapipe.tasks import python
from mediapipe.tasks.python import vision
from mediapipe.framework.formats import landmark_pb2

# Model available to download here: https://developers.google.com/mediapipe/solutions/vision/pose_landmarker#models
model_path = "pose_landmarker_full.task"

video_source = 0

num_poses = 4
min_pose_detection_confidence = 0.5
min_pose_presence_confidence = 0.5
min_tracking_confidence = 0.5


def draw_landmarks_on_image(rgb_image, detection_result):
    pose_landmarks_list = detection_result.pose_landmarks
    annotated_image = np.copy(rgb_image)

    # Loop through the detected poses to visualize.
    for idx in range(len(pose_landmarks_list)):
        pose_landmarks = pose_landmarks_list[idx]

        pose_landmarks_proto = landmark_pb2.NormalizedLandmarkList()
        pose_landmarks_proto.landmark.extend([
            landmark_pb2.NormalizedLandmark(
                x=landmark.x,
                y=landmark.y,
                z=landmark.z) for landmark in pose_landmarks
        ])
        mp.solutions.drawing_utils.draw_landmarks(
            annotated_image,
            pose_landmarks_proto,
            mp.solutions.pose.POSE_CONNECTIONS,
            mp.solutions.drawing_styles.get_default_pose_landmarks_style())
    return annotated_image


to_window = None
last_timestamp_ms = 0


def print_result(detection_result: vision.PoseLandmarkerResult, output_image: mp.Image,
                 timestamp_ms: int):
    global to_window
    global last_timestamp_ms
    if timestamp_ms < last_timestamp_ms:
        return
    last_timestamp_ms = timestamp_ms
    # print("pose landmarker result: {}".format(detection_result))
    to_window = cv2.cvtColor(
        draw_landmarks_on_image(output_image.numpy_view(), detection_result), cv2.COLOR_RGB2BGR)


base_options = python.BaseOptions(model_asset_path=model_path)
options = vision.PoseLandmarkerOptions(
    base_options=base_options,
    running_mode=vision.RunningMode.LIVE_STREAM,
    num_poses=num_poses,
    min_pose_detection_confidence=min_pose_detection_confidence,
    min_pose_presence_confidence=min_pose_presence_confidence,
    min_tracking_confidence=min_tracking_confidence,
    output_segmentation_masks=False,
    result_callback=print_result
)

with vision.PoseLandmarker.create_from_options(options) as landmarker:
    # Use OpenCV’s VideoCapture to start capturing from the webcam.
    cap = cv2.VideoCapture(video_source)

    # Create a loop to read the latest frame from the camera using VideoCapture#read()
    while cap.isOpened():
        success, image = cap.read()
        if not success:
            print("Image capture failed.")
            break

        # Convert the frame received from OpenCV to a MediaPipe’s Image object.
        mp_image = mp.Image(
            image_format=mp.ImageFormat.SRGB,
            data=cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
        timestamp_ms = int(cv2.getTickCount() / cv2.getTickFrequency() * 1000)
        landmarker.detect_async(mp_image, timestamp_ms)

        if to_window is not None:
            cv2.imshow("MediaPipe Pose Landmark", to_window)

        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

    cap.release()
    cv2.destroyAllWindows()

I find how to using this model with multiple person, Blaze Pose model from mediapipe pose model is operating only Pose Detect (two types of Modules from this model). then If we set all component of multiple pose(option num pose >= 2), 1st module(detector) were detect multi pose landmark at first video frame and 2nd module(Tracker) were using from frist frame pose landmark. so we can use multiple pose detection model

+) if you want to know detail of model. Enter my post link of multiple pose detetion model architecture.

Link : https://dablro12.tistory.com/12

In order to widely inform Koreans about this met hod, I wrote it in the Korean version. It's a hassle, but please understand.

did it work? I'm gonna convert it into mediapipe flugin unity

Maybe, I couldn't convert it into mediaipie plugin unity, so if you tried this methodm, Can you reply for me? [image: Mailtrack] <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality11&> Sender notified by Mailtrack <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality11&> 24. 04. 15. 오후 12:22:57 2024년 4월 15일 (월) 오후 12:15, Phạm Trần Triêm ***@***.***>님이 작성:

…

***@***.**** commented on this gist. ------------------------------ I find how to using this model with multiple person, Blaze Pose model from mediapipe pose model is operating only Pose Detect (two types of Modules from this model). then If we set all component of multiple pose(option num pose >= 2), 1st module(detector) were detect multi pose landmark at first video frame and 2nd module(Tracker) were using from frist frame pose landmark. so we can use multiple pose detection model [image: 다운로드] <https://private-user-images.githubusercontent.com/54443308/317915326-836b8684-a945-4615-8f99-0ec73c7498a7.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTMxNTAwMDUsIm5iZiI6MTcxMzE0OTcwNSwicGF0aCI6Ii81NDQ0MzMwOC8zMTc5MTUzMjYtODM2Yjg2ODQtYTk0NS00NjE1LThmOTktMGVjNzNjNzQ5OGE3LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA0MTUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNDE1VDAyNTUwNVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTRiYjlmZTg3MDdiMzNmMWU4MGRkNjk0ZmVmMGM4MTU3NDYwMWZkNWQ1NDhlYTgyNTg1ZTdhZTYzOGJkYWQ4MWQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.Ya2P1FyYToSWYrX2UgccqdMbJVd799iBiNXlIB058QQ> +) if you want to know detail of model. Enter my post link of multiple pose detetion model architecture. Link : https://dablro12.tistory.com/12 In order to widely inform Koreans about this met hod, I wrote it in the Korean version. It's a hassle, but please understand. did it work? I'm gonna convert it into mediapipe flugin unity — Reply to this email directly, view it on GitHub <https://gist.github.com/lanzani/f85175d8fbdafcabb7d480dd1bb769d9#gistcomment-5023346> or unsubscribe <https://github.com/notifications/unsubscribe-auth/AM7L2LFKFKR2JDPH6SPTAP3Y5NA4JBFKMF2HI4TJMJ2XIZLTSKBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDHNFZXJJDOMFWWLK3UNBZGKYLEL52HS4DFVRZXKYTKMVRXIX3UPFYGLK2HNFZXIQ3PNVWWK3TUUZ2G64DJMNZZDAVEOR4XAZNEM5UXG5FFOZQWY5LFVEYTENRUGM2DCNBUU52HE2LHM5SXFJTDOJSWC5DF> . You are receiving this email because you commented on the thread. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub> .

-- [image: image.jpeg] 최대현 바이오메디컬공학 및 AI&Laguage m. 010-5661-6371 e. ***@***.*** p. choedaehyeon Portfolio <https://valiant-barnacle-6bf.notion.site/>