How to implement Touch-to-Focus in Android using Camera2 APIs

AndroidCamera2TouchToFocus.java

    //Override in your touch-enabled view (this can be differen than the view you use for displaying the cam preview)
    @Override
    public boolean onTouch(View view, MotionEvent motionEvent) {
        final int actionMasked = motionEvent.getActionMasked();
        if (actionMasked != MotionEvent.ACTION_DOWN) {
            return false;
        }
        if (mManualFocusEngaged) {
            Log.d(TAG, "Manual focus already engaged");
            return true;
        }

        final Rect sensorArraySize = mCameraInfo.get(CameraCharacteristics.SENSOR_INFO_ACTIVE_ARRAY_SIZE);

        //TODO: here I just flip x,y, but this needs to correspond with the sensor orientation (via SENSOR_ORIENTATION)
        final int y = (int)((motionEvent.getX() / (float)view.getWidth())  * (float)sensorArraySize.height());
        final int x = (int)((motionEvent.getY() / (float)view.getHeight()) * (float)sensorArraySize.width());
        final int halfTouchWidth  = 150; //(int)motionEvent.getTouchMajor(); //TODO: this doesn't represent actual touch size in pixel. Values range in [3, 10]...
        final int halfTouchHeight = 150; //(int)motionEvent.getTouchMinor();
        MeteringRectangle focusAreaTouch = new MeteringRectangle(Math.max(x - halfTouchWidth,  0),
                                                                 Math.max(y - halfTouchHeight, 0),
                                                                 halfTouchWidth  * 2,
                                                                 halfTouchHeight * 2,
                                                                 MeteringRectangle.METERING_WEIGHT_MAX - 1);

        CameraCaptureSession.CaptureCallback captureCallbackHandler = new CameraCaptureSession.CaptureCallback() {
            @Override
            public void onCaptureCompleted(CameraCaptureSession session, CaptureRequest request, TotalCaptureResult result) {
                super.onCaptureCompleted(session, request, result);
                mManualFocusEngaged = false;

                if (request.getTag() == "FOCUS_TAG") {
                    //the focus trigger is complete -
                    //resume repeating (preview surface will get frames), clear AF trigger
                    mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_TRIGGER, null);
                    mCameraOps.setRepeatingRequest(mPreviewRequestBuilder.build(), null, null);
                }
            }

            @Override
            public void onCaptureFailed(CameraCaptureSession session, CaptureRequest request, CaptureFailure failure) {
                super.onCaptureFailed(session, request, failure);
                Log.e(TAG, "Manual AF failure: " + failure);
                mManualFocusEngaged = false;
            }
        };

        //first stop the existing repeating request
        mCameraOps.stopRepeating();

        //cancel any existing AF trigger (repeated touches, etc.)
        mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_TRIGGER, CameraMetadata.CONTROL_AF_TRIGGER_CANCEL);
        mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_OFF);
        mCameraOps.capture(mPreviewRequestBuilder.build(), captureCallbackHandler, mBackgroundHandler);

        //Now add a new AF trigger with focus region
        if (isMeteringAreaAFSupported()) {
            mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_REGIONS, new MeteringRectangle[]{focusAreaTouch});
        }
        mPreviewRequestBuilder.set(CaptureRequest.CONTROL_MODE, CameraMetadata.CONTROL_MODE_AUTO);
        mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_AUTO);
        mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_TRIGGER, CameraMetadata.CONTROL_AF_TRIGGER_START);
        mPreviewRequestBuilder.setTag("FOCUS_TAG"); //we'll capture this later for resuming the preview

        //then we ask for a single request (not repeating!)
        mCameraOps.capture(mPreviewRequestBuilder.build(), captureCallbackHandler, mBackgroundHandler);
        mManualFocusEngaged = true;

        return true;
    }

    private boolean isMeteringAreaAFSupported() {
        return mCameraInfo.get(CameraCharacteristics.CONTROL_MAX_REGIONS_AF) >= 1;
    }

Hey guys @Nan0fm @siralam (possibly @JeetSingh4 as well), simple fix, don't know if it's the correct fix but it works (and without freezing anything), remove captureCallbackHandler from the captures and where it says "//then we ask for a single request (not repeating!)" change it from .capture to .setRepeatingRequest. Finally you will run into the boolean lock caused by "mManualFocusEngaged" so set line 79: "mManualFocusEngaged = true;" to "mManualFocusEngaged = false;" and add "mManualFocusEngaged=true" in line 23

In some devices(Sony xperia G8142 etc..),CaptureRequest.CONTROL_AF_TRIGGER can not be set to null,or the camera throws an error. but how to solve this, I have no idea..

I found that setting CONTROL_AF_TRIGGER to CONTROL_AF_TRIGGER_IDLE instead of null works on Xperia devices.

This code does not take in to account rotation or cropping. If you are displaying a video feed in 1920x1080 tapping in 0,0 should probably not focus in 0, 0 in sensor coordinates.

can you please provide me full code at my email id : kingervinny@gmail.com . Thanks in advance.

can you please provide me full code at my email id : kjcsp321@gmail.com . Thanks in advance.

Can you please provide full code access link to victorvishal23@gmail.com . It would be very helpful

Sad... I copied everything, but when I tap on the view, the Texture view freezes. I added breakpoint, and can reach onCaptureCompleted with the correct tag ("FOCUS_TAG"). The line mCameraOps.setRepeatingRequest(mPreviewRequestBuilder.build(), null, null); should be executed without problem, but the preview still freezes.

Hey, if you have the executing code can you please share the link to jyotirmay.ghosh2@gmail.com

Does any one have the full code? And has any one else implemented it base on this? How do you handle the default initialization of thoes variables?

@mortenholmgaard
I took used the google sample project that is using camera api 2 from here
https://github.com/android/camera-samples/tree/master/Camera2BasicJava/Application/src/main/java/com/example/android/camera2basic
After you manage to work with this example just copy and paste the onTouch code and it will work.

There is still an issue which I'm trying currently to figure out, after you do an auto focus you are not able to capture a photo.
I will update if I will manage to fix it, feel free to ask questions, if you got to some problems.

can you please provide me full code at my email id :logapriyasa@gmail.com . Thanks in advance.

can you please provide me full code at my email - saurabhathavale02@gmail.com.Thanks in advance

How to touch on focus using CameraX library and how to disable autofocus? Only focus on touch?

can you please provide me full code at my email id : dlehdgus0413@gmail.com . Thanks in advance.

In some devices(Sony xperia G8142 etc..),CaptureRequest.CONTROL_AF_TRIGGER can not be set to null,or the camera throws an error. but how to solve this, I have no idea..

I found that setting CONTROL_AF_TRIGGER to CONTROL_AF_TRIGGER_IDLE instead of null works on Xperia devices.

Works much better this way on OnePlus devices as well
Thanks

If anyone looking for translation of Sensor Orientation to Screen Orientation then instead of

        final Rect sensorArraySize = mCameraInfo.get(CameraCharacteristics.SENSOR_INFO_ACTIVE_ARRAY_SIZE);

        //TODO: here I just flip x,y, but this needs to correspond with the sensor orientation (via SENSOR_ORIENTATION)
        final int y = (int)((motionEvent.getX() / (float)view.getWidth())  * (float)sensorArraySize.height());
        final int x = (int)((motionEvent.getY() / (float)view.getHeight()) * (float)sensorArraySize.width());
        final int halfTouchWidth  = 150; //(int)motionEvent.getTouchMajor(); //TODO: this doesn't represent actual touch size [...]
        final int halfTouchHeight = 150; //(int)motionEvent.getTouchMinor();
        MeteringRectangle focusAreaTouch = new MeteringRectangle(Math.max(x - halfTouchWidth,  0),
                                                                 Math.max(y - halfTouchHeight, 0),
                                                                 halfTouchWidth  * 2,
                                                                 halfTouchHeight * 2,
                                                                 MeteringRectangle.METERING_WEIGHT_MAX - 1);

Use

// [...]

    MeteringRectangle focusAreaTouch = convertTouchToSensorCoordinates(view, motionEvent);

// [...]

protected MeteringRectangle convertTouchToSensorCoordinates(View view, MotionEvent motionEvent) {
    final Rect sensorArraySize = Objects.requireNonNull(mCameraInfo.get(CameraCharacteristics.SENSOR_INFO_ACTIVE_ARRAY_SIZE));
    // Set default rotation to no rotation
    int sensorOrientation = 0;
    // Prevent from failing, check before conversion
    if(mCameraInfo.get(CameraCharacteristics.SENSOR_ORIENTATION) != null) {
        sensorOrientation = Objects.requireNonNull(mCameraInfo.get(CameraCharacteristics.SENSOR_ORIENTATION));
    }

    // Set the half-touch sizes (AF rectangle half-size)
    final int halfTouchWidth  = (int)motionEvent.getTouchMajor();
    final int halfTouchHeight = (int)motionEvent.getTouchMinor();

    // Normalize touch event coordinates
    final float xNormalized = motionEvent.getX() / (float)view.getWidth();
    final float yNormalized = motionEvent.getY() / (float)view.getHeight();

    // Set default coordinates translation for sensorOrientation==0, so with no relative rotation
    int xSensor = (int)(xNormalized * (float)sensorArraySize.width());
    int ySensor = (int)(yNormalized * (float)sensorArraySize.height());

    // Translation for rotation 90 [ A(x,y)->A'(-y,x) ]
    if (sensorOrientation == 90) {
        xSensor = (int)(yNormalized * (float)sensorArraySize.width());
        ySensor = (int)((1.0f-xNormalized) * (float)sensorArraySize.height());
    // Translation for rotation 180 [ A(x,y)->A'(-x,-y) ]
    } else if (sensorOrientation == 180) {
        xSensor = (int)((1.0f-xNormalized) * (float)sensorArraySize.width());
        ySensor = (int)((1.0f-yNormalized) * (float)sensorArraySize.height());
    // Translation for rotation 270 [ A(x,y)->A'(y,-x) ]
    } else if (sensorOrientation == 270) {
        xSensor = (int)((1.0f-yNormalized) * (float)sensorArraySize.width());
        ySensor = (int)(xNormalized * (float)sensorArraySize.height());
    }

    // Return new object with computed values
    return new MeteringRectangle(Math.max(xSensor - halfTouchWidth,  0),
            Math.max(ySensor - halfTouchHeight, 0),
            halfTouchWidth  * 2,
            halfTouchHeight * 2,
            MeteringRectangle.METERING_WEIGHT_MAX - 1);
}

@Fiooodooor it's still not correct, first we need to convert Touch coordinateness of View Size to Camera Preview Size coordinates and only after that we can convert it to Camera Sensor Size - https://stackoverflow.com/a/33181620/7767664 (the guy which worked in Google on camera2 API):

The units for them are in the sensor active array coordinate system, so you'll have to translate from your UI touch coordinates to coordinates relative to your preview view, and from there, to the active array coordinates.

@anonym24 great point, that is correct. I've made couple of subconscious assumptions, including working with full-screen, no icons, no menus preview of camera sensor. Thanks for correction!