gistfile1.txt

Hello,

This mail is about a new trend in smartphones, which is under-display fingerprint sensors, which lacks Treble HAL, and I believe it would be very useful to have one.
I believe it is possible for AOSP team to both define the HAL (well imo that's the minimum), and have a minimal implementation (though ugly from an UX point of view), without too much work. This would require cooperation from people handling biometrics, and people handling surfaceflinger.

There are currently two kinds of under-display fingerprint.
- Ultrasonic which basically just work, except that they require visual clues for the user
- Optical fingerprint sensor which requires cooperation from the display

First issue for both technologies is to show to the user the position of the fingerprint sensor.
I believe that the HAL for that can be a simple property. That's what Xiaomi, and Goodix does for instance: they define persist.vendor.sys.fp.fod.location.X_Y and persist.vendor.sys.fp.fod.size.width_height to be the location of the sensor in pixels of the screen at native rotation. [1]

Second issue, again for both technologies, is the fingerprint driver needs to know when to scan for the finger.
Technically, the touchscreen driver could talk directly to fingerprint driver to propagate this, but this requires specific hack to the kernel, and the kernel needs to know the position of the fingerprint sensor on the touchscreen.
Most OEMs didn't do that, and have their own changes in Android framework to report touch/untouched status to the fingerprint driver.
This could be fixed with an addition to biometrics.fingerprint HAL which would be `oneway onFodPressed(boolean pressed);` [2]

Third issue is the most complicated one. Optical under-display fingerprint sensor requires the screen to backlight the finger, with maximum brightness (usually even above it, using High Brightness Mode), usually in green, but sometimes in cyan.
The issue here, is that you obviously can't have the screen go to brightness and be done with it, because changing brightness against user's will is very bad, especially in low light.
The way to fix this is by adding a dim layer between the green circle used to light the finger, and the rest of the Android system. The issue then is to guarantee that switching to High Brightness Mode and the dim layer is perfectly synchronous, otherwise there would be one frame with too high brightness or too low, which would be even worse for the user experience.
The way most OEMs do it, is by having the framework create two layers, one which contains the green circle, and one beneath it (usually to display the fingerprint position clue), and then hwcomposer does its magic: when it sees the top-most layer, it sets the display to HBM, and it replaces the second layer with a dim layer, whose value has been computed based on current brightness, and backlight gamma. [3]
My proposal here is to have the framework create a layer when enroll/auth is started, and another one when the user press the fingerprint senor, and simply extend gralloc usage bits with two new usages:
FOD_DIM_LAYER (the layer created when enroll/auth starts)
FOD_FINGERPRINT_BACKLIGHT (the layer created when the user presses the fingerprint sensor)
Compared to what I've seen OEMs do, the limitation here is that many OEMs compute the alpha of the dim layer framework-side, but I believe that given that HAL, OEMs should be able to implement it hwcomposer-side.


I hope this explanation can be the start of a discussion towards the inclusion of such support in AOSP.



[1] For a list of ways this is handled:
- Xiaomi has properties giving values in px
- Samsung has a /sys file with values in mm
- Oppo has properties that gives offset from the bottom, and width
- other OEMs hardcode position in the framework

[2]
Samsung handle this by switching touchscreen driver when starting auth/enroll in a mode where it sends a specific bounding box's event to the fingerprint driver. The bounding box is computed by the framework.
Other OEMs just have their own HAL which have the equivalent of that "onFodPressed"

[3]
- Asus doesn't create any surface framework-side, everything is done in-kernel, but framework cooperation should be better
- Samsung uses USAGE_BITS, but only have one usage bit to trigger HBM. The dim is completely handled by the framework
- Oppo uses magical Z order in HWC Layers instead of usage bits, but except for that they have the behaviour described: dim layer is computed automatically by hwcomposer
- Xiaomi uses magical Z order to enable HBM and create dim layer, but the value of the dim alpha is computed by the framework