This is how I usually approach inpainting in HTML. I use three canvas objects: canvas_orig
, canvas_mask
, canvas_screen
- Start with an image uploaded by the user or a URL
- Load original image into an offscreen canvas, let's call it
canvas_orig
- Draw the the same image onto a canvas displayed to the user, that's
canvas_screen
. This is the only canvas visible to the user - Capture click and drag events for
canvas_screen
and draw them to another offscreen canvas, that'scanvas_mask
- Whenever
canvas_mask
changes, redrawcanvas_orig
ontocanvas_screen
and drawcanvas_mask
on top of it with some alpha (useCanvasRenderingContext2D.globalAlpha
, see example at https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/globalAlpha) - When the user is done with drawing the mask, you send
canvas_orig
andcanvas_mask
off to your GPU provider (I use replicate https://replicate.com/) - Receive the result, store it in
canvas_orig
, clearcanvas_mask
and redraw `canv