openpose java sample

openpose.java

import org.opencv.core.*;
import org.opencv.dnn.*;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.imgproc.Imgproc;
import java.util.*;

public class SimpleSample {

    public static void main(String[] args) {
        // Load the native library.
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME);

        Mat img = Imgcodecs.imread("c:/data/img/persons/single2.png");
        // read the network model
        //Net net = Dnn.readNetFromTensorflow("c:/data/mdl/body/tf_small.pb");
        Net net = Dnn.readNetFromCaffe("c:/data/mdl/body/openpose_pose_coco.prototxt", "c:/data/mdl/body/pose_iter_440000.caffemodel");

        // send it through the network
        //Mat inputBlob = Dnn.blobFromImage(img, 1.0, new Size(368,368), new Scalar(0, 0, 0), false, false);
        Mat inputBlob = Dnn.blobFromImage(img, 1.0 / 255, new Size(368,368), new Scalar(0, 0, 0), false, false);
        net.setInput(inputBlob);
        Mat result = net.forward().reshape(1,19); // 19 body parts
        //Mat result = net.forward().reshape(1,57); // 19 body parts + 2 * 19 PAF maps

        System.out.println(result);

        // get the heatmap locations
        ArrayList<Point> points = new ArrayList();
        for (int i=0; i<18; i++) { // skip background
            Mat heatmap = result.row(i).reshape(1,46); // 46x46
            Core.MinMaxLocResult mm = Core.minMaxLoc(heatmap);
            Point p = new Point();
            if (mm.maxVal>0.1f) {
                p = mm.maxLoc;
            }
            points.add(p);
            System.out.println(i + " " + p + " " + heatmap);
        }

        // 17 possible limb connections
        int pairs[][] = {
            {1,2}, {1,5}, {2,3},
            {3,4}, {5,6}, {6,7},
            {1,8}, {8,9}, {9,10},
            {1,11}, {11,12}, {12,13},
            {1,0}, {0,14},
            {14,16}, {0,15}, {15,17}
        };

        // connect body parts and draw it !
        float SX = (float)(img.cols()) / 46;
        float SY = (float)(img.rows()) / 46;
        for (int n=0; n<17; n++)
        {
            // lookup 2 connected body/hand parts
            Point a = points.get(pairs[n][0]).clone();
            Point b = points.get(pairs[n][1]).clone();

            // we did not find enough confidence before
            if (a.x<=0 || a.y<=0 || b.x<=0 || b.y<=0)
                continue;

            // scale to image size
            a.x*=SX; a.y*=SY;
            b.x*=SX; b.y*=SY;

            Imgproc.line(img, a, b, new Scalar(0,200,0), 2);
            Imgproc.circle(img, a, 3, new Scalar(0,0,200), -1);
            Imgproc.circle(img, b, 3, new Scalar(0,0,200), -1);
        }
        Imgcodecs.imwrite("pose.png", img);
    }
}

hello,
Im getting this error on line 22 "Mat result = net.forward().reshape(1,19);"
error: (-209:Sizes of input arguments do not match) Requested and source matrices have different count of elements in function 'reshape'

hi @abedshouman , 19 is hardcoded to the coco body model (using opencv's tailored prototxt)

what data do you try to use ?

Hello my Friend, Thanks for you help, when I used the same prototxt and caffemodel like yours I got another error error: (-215:Assertion failed) inputs[0].size[1] % blobs[0].size[1] == 0 in function 'forward' Then I changed the Color of the mat image to BGR and it worked perfectly Thank you very much Regards

…

________________________________ From: berak <notifications@github.com> Sent: Tuesday, August 20, 2019 11:51:54 AM To: berak <berak@noreply.github.com> Cc: abedshouman <abedshouman@hotmail.com>; Mention <mention@noreply.github.com> Subject: Re: berak/openpose.java hi @abedshouman<https://github.com/abedshouman> , 19 is hardcoded to the coco body model (using opencv's tailored prototxt) what data do you try to use ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://gist.github.com/eef67a1afb4b6445d117ea1a0135c8db?email_source=notifications&email_token=AJS66I37UP4IZV6U4CLDDSTQFOWCVA5CNFSM4INRNEA2YY3PNVWWK3TUL52HS4DFVNDWS43UINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAFXJF6#gistcomment-3002975>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AJS66I2ZCR5OXXZEA5RNOZTQFOWCVANCNFSM4INRNEAQ>.

hehe, perfect, pleasure ;)

Hi, berak. Your sample is very useful for a beginner like me.
Can you please explain furthermore on how should I change these value if I want to work with handPoseImage

Mat inputBlob = Dnn.blobFromImage(img, 1.0 / 255, new Size(368,368), new Scalar(0, 0, 0), false, false);
Mat heatmap = result.row(i).reshape(1,46);

How do you get 368 and 46?

Thank you so much.

sorry, but i never tried with the hand model. you'll have to adapt the pairs:

https://github.com/opencv/opencv/blob/a4d16acd0052a43e8b312d119b16004874d56a81/samples/dnn/openpose.cpp#L49-L55

and number of limbs/pairs:

https://github.com/opencv/opencv/blob/a4d16acd0052a43e8b312d119b16004874d56a81/samples/dnn/openpose.cpp#L90

How do you get 368 and 46?

from the original code / paper. 368 = 46 * 8. the heatmaps are 1/8 of the input size.

good luck ;)

Does this support multiple person?

@docete, no only single person

learnopencv.com has some code for multiple

@berak I rewrite the learnopencv.com's multi-person-openpose with Java API, but the key points of 'group.jpg' was all jumbled up.
I post my code at https://gist.github.com/docete/38bb39390da8e8c823c3211419b9a5ad
Is there any way you can have a look at it?

@berak I rewrite the learnopencv.com's multi-person-openpose with Java API, but the key points of 'group.jpg' was all jumbled up.
I post my code at https://gist.github.com/docete/38bb39390da8e8c823c3211419b9a5ad
Is there any way you can have a look at it?

Figure it out that I misuse the 'reshape' in splitNetOutputBlobToParts.