Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Optimal sharpening strength (according to objective metrics) - 0.5. Can be applied only to luma channel (change OUTPUT to LUMA). To use it on-demand add the following line to input.conf: n change-list glsl-shaders toggle "~~/adaptive-sharpen.glsl"
// Copyright (c) 2015-2021, bacondither
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions
// are met:
// 1. Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer
// in this position and unchanged.
// 2. Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
//
// THIS SOFTWARE IS PROVIDED BY THE AUTHORS ``AS IS'' AND ANY EXPRESS OR
// IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
// OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
// IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
// INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
// NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
// THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
// Adaptive sharpen - version 2021-10-17
// Tuned for use post-resize
//!HOOK OUTPUT
//!BIND HOOKED
//!DESC adaptive-sharpen
//--------------------------------------- Settings ------------------------------------------------
#define curve_height 1.0 // Main control of sharpening strength [>0]
// 0.3 <-> 2.0 is a reasonable range of values
#define overshoot_ctrl false // Allow for higher overshoot if the current edge pixel
// is surrounded by similar edge pixels
// Defined values under this row are "optimal" DO NOT CHANGE IF YOU DO NOT KNOW WHAT YOU ARE DOING!
#define curveslope 0.5 // Sharpening curve slope, high edge values
#define L_compr_low 0.167 // Light compression, default (0.167=~6x)
#define L_compr_high 0.334 // Light compression, surrounded by edges (0.334=~3x)
#define D_compr_low 0.250 // Dark compression, default (0.250=4x)
#define D_compr_high 0.500 // Dark compression, surrounded by edges (0.500=2x)
#define scale_lim 0.1 // Abs max change before compression [>0.01]
#define scale_cs 0.056 // Compression slope above scale_lim
#define pm_p 1.0 // Power mean p-value [>0-1.0]
//-------------------------------------------------------------------------------------------------
#define max4(a,b,c,d) ( max(max(a, b), max(c, d)) )
// Soft if, fast linear approx
#define soft_if(a,b,c) ( sat((a + b + c + 0.056/2.5)/(maxedge + 0.03/2.5) - 0.85) )
// Soft limit, modified tanh approx
#define soft_lim(v,s) ( sat(abs(v/s)*(27.0 + pow(v/s, 2.0))/(27.0 + 9.0*pow(v/s, 2.0)))*s )
// Weighted power mean
#define wpmean(a,b,w) ( pow(w*pow(abs(a), pm_p) + abs(1.0-w)*pow(abs(b), pm_p), (1.0/pm_p)) )
// Get destination pixel values
#define get(x,y) ( HOOKED_texOff(vec2(x, y)).rgb )
#define sat(x) ( clamp(x, 0.0, 1.0) )
#define dxdy(val) ( length(fwidth(val)) ) // =~1/2.5 hq edge without c_comp
#ifdef LUMA_tex
#define CtL(RGB) RGB.x
#else
#define CtL(RGB) ( sqrt(dot(sat(RGB)*sat(RGB), vec3(0.2126, 0.7152, 0.0722))) )
#endif
#define b_diff(pix) ( (blur-luma[pix])*(blur-luma[pix]) )
vec4 hook() {
// [ c22 ]
// [ c24, c9, c23 ]
// [ c21, c1, c2, c3, c18 ]
// [ c19, c10, c4, c0, c5, c11, c16 ]
// [ c20, c6, c7, c8, c17 ]
// [ c15, c12, c14 ]
// [ c13 ]
vec3 c[25] = vec3[](get( 0, 0), get(-1,-1), get( 0,-1), get( 1,-1), get(-1, 0),
get( 1, 0), get(-1, 1), get( 0, 1), get( 1, 1), get( 0,-2),
get(-2, 0), get( 2, 0), get( 0, 2), get( 0, 3), get( 1, 2),
get(-1, 2), get( 3, 0), get( 2, 1), get( 2,-1), get(-3, 0),
get(-2, 1), get(-2,-1), get( 0,-3), get( 1,-2), get(-1,-2));
float e[13] = float[](dxdy(c[0]), dxdy(c[1]), dxdy(c[2]), dxdy(c[3]), dxdy(c[4]),
dxdy(c[5]), dxdy(c[6]), dxdy(c[7]), dxdy(c[8]), dxdy(c[9]),
dxdy(c[10]), dxdy(c[11]), dxdy(c[12]));
// RGB to luma
float luma[25] = float[](CtL(c[0]), CtL(c[1]), CtL(c[2]), CtL(c[3]), CtL(c[4]), CtL(c[5]), CtL(c[6]),
CtL(c[7]), CtL(c[8]), CtL(c[9]), CtL(c[10]), CtL(c[11]), CtL(c[12]),
CtL(c[13]), CtL(c[14]), CtL(c[15]), CtL(c[16]), CtL(c[17]), CtL(c[18]),
CtL(c[19]), CtL(c[20]), CtL(c[21]), CtL(c[22]), CtL(c[23]), CtL(c[24]));
float c0_Y = luma[0];
// Blur, gauss 3x3
float blur = (2.0 * (luma[2]+luma[4]+luma[5]+luma[7]) + (luma[1]+luma[3]+luma[6]+luma[8]) + 4.0 * luma[0]) / 16.0;
// Contrast compression, center = 0.5
float c_comp = sat(0.266666681f + 0.9*exp2(blur * blur * -7.4));
// Edge detection
// Relative matrix weights
// [ 1 ]
// [ 4, 5, 4 ]
// [ 1, 5, 6, 5, 1 ]
// [ 4, 5, 4 ]
// [ 1 ]
float edge = ( 1.38*b_diff(0)
+ 1.15*(b_diff(2) + b_diff(4) + b_diff(5) + b_diff(7))
+ 0.92*(b_diff(1) + b_diff(3) + b_diff(6) + b_diff(8))
+ 0.23*(b_diff(9) + b_diff(10) + b_diff(11) + b_diff(12)) ) * c_comp;
vec2 cs = vec2(L_compr_low, D_compr_low);
if (overshoot_ctrl) {
float maxedge = max4( max4(e[1],e[2],e[3],e[4]), max4(e[5],e[6],e[7],e[8]),
max4(e[9],e[10],e[11],e[12]), e[0] );
// [ x ]
// [ z, x, w ]
// [ z, z, x, w, w ]
// [ y, y, y, 0, y, y, y ]
// [ w, w, x, z, z ]
// [ w, x, z ]
// [ x ]
float sbe = soft_if(e[2],e[9], dxdy(c[22]))*soft_if(e[7],e[12],dxdy(c[13])) // x dir
+ soft_if(e[4],e[10],dxdy(c[19]))*soft_if(e[5],e[11],dxdy(c[16])) // y dir
+ soft_if(e[1],dxdy(c[24]),dxdy(c[21]))*soft_if(e[8],dxdy(c[14]),dxdy(c[17])) // z dir
+ soft_if(e[3],dxdy(c[23]),dxdy(c[18]))*soft_if(e[6],dxdy(c[20]),dxdy(c[15])); // w dir
cs = mix(cs, vec2(L_compr_high, D_compr_high), sat(2.4002*sbe - 2.282));
}
// Precalculated default squared kernel weights
const vec3 w1 = vec3(0.5, 1.0, 1.41421356237); // 0.25, 1.0, 2.0
const vec3 w2 = vec3(0.86602540378, 1.0, 0.54772255751); // 0.75, 1.0, 0.3
// Transition to a concave kernel if the center edge val is above thr
vec3 dW = pow(mix( w1, w2, sat(2.4*edge - 0.82)), vec3(2.0));
// Use lower weights for pixels in a more active area relative to center pixel area
// This results in narrower and less visible overshoots around sharp edges
float modif_e0 = 3.0 * e[0] + 0.02/2.5;
float weights[12] = float[](( min(modif_e0/e[1], dW.y) ),
( dW.x ),
( min(modif_e0/e[3], dW.y) ),
( dW.x ),
( dW.x ),
( min(modif_e0/e[6], dW.y) ),
( dW.x ),
( min(modif_e0/e[8], dW.y) ),
( min(modif_e0/e[9], dW.z) ),
( min(modif_e0/e[10], dW.z) ),
( min(modif_e0/e[11], dW.z) ),
( min(modif_e0/e[12], dW.z) ));
weights[0] = (max(max((weights[8] + weights[9])/4.0, weights[0]), 0.25) + weights[0])/2.0;
weights[2] = (max(max((weights[8] + weights[10])/4.0, weights[2]), 0.25) + weights[2])/2.0;
weights[5] = (max(max((weights[9] + weights[11])/4.0, weights[5]), 0.25) + weights[5])/2.0;
weights[7] = (max(max((weights[10] + weights[11])/4.0, weights[7]), 0.25) + weights[7])/2.0;
// Calculate the negative part of the laplace kernel and the low threshold weight
float lowthrsum = 0.0;
float weightsum = 0.0;
float neg_laplace = 0.0;
for (int pix = 0; pix < 12; ++pix)
{
float lowthr = sat((20.*4.5*c_comp*e[pix + 1] - 0.221));
neg_laplace += luma[pix+1] * luma[pix+1] * weights[pix] * lowthr;
weightsum += weights[pix] * lowthr;
lowthrsum += lowthr / 12.0;
}
neg_laplace = sqrt(neg_laplace / weightsum);
// Compute sharpening magnitude function
float sharpen_val = curve_height/(curve_height*curveslope*edge + 0.625);
// Calculate sharpening diff and scale
float sharpdiff = (c0_Y - neg_laplace)*(lowthrsum*sharpen_val + 0.01);
// Calculate local near min & max, partial sort
float temp;
for (int i1 = 0; i1 < 24; i1 += 2)
{
temp = luma[i1];
luma[i1] = min(luma[i1], luma[i1+1]);
luma[i1+1] = max(temp, luma[i1+1]);
}
for (int i2 = 24; i2 > 0; i2 -= 2)
{
temp = luma[0];
luma[0] = min(luma[0], luma[i2]);
luma[i2] = max(temp, luma[i2]);
temp = luma[24];
luma[24] = max(luma[24], luma[i2-1]);
luma[i2-1] = min(temp, luma[i2-1]);
}
float min_dist = min(abs(luma[24] - c0_Y), abs(c0_Y - luma[0]));
min_dist = min(min_dist, scale_lim*(1.0 - scale_cs) + min_dist*scale_cs);
// Soft limited anti-ringing with tanh, wpmean to control compression slope
sharpdiff = wpmean(max(sharpdiff, 0.0), soft_lim( max(sharpdiff, 0.0), min_dist ), cs.x )
- wpmean(min(sharpdiff, 0.0), soft_lim( min(sharpdiff, 0.0), min_dist ), cs.y );
float sharpdiff_lim = sat(c0_Y + sharpdiff) - c0_Y;
/*float satmul = (c0_Y + max(sharpdiff_lim*0.9, sharpdiff_lim)*0.3 + 0.03)/(c0_Y + 0.03);
vec3 res = c0_Y + sharpdiff_lim + (c[0] - c0_Y)*satmul;
*/
return vec4(sharpdiff_lim + c[0], HOOKED_texOff(0).a);
}
@CrHasher
Copy link

CrHasher commented Apr 8, 2022

One more question, does adaptive sharpen need to be rgb?

No, try luma version

I updated the adaptive sharpen luma version to latest from you @igv and we have a new winner, it took the crown 👑 from the RGB version. Will be back with results soon. Given that it works on luma before downscaling (aka bigger image) I will have to check performance.

@CrHasher
Copy link

CrHasher commented Apr 9, 2022

Results:
image

@CrHasher
Copy link

CrHasher commented Apr 9, 2022

Source (curve_height 0.1, linear_laplace true)

@igv
Copy link
Author

igv commented Apr 9, 2022

Cool, but I hardly believe that luma version is sharper (because it's applied at higher res). Is it really sharper according to TENV?

@CrHasher
Copy link

CrHasher commented Apr 9, 2022

Cool, but I hardly believe that luma version is sharper (because it's applied at higher res). Is it really sharper according to TENV?

It's a bit softer but in most cases that's a good thing less over-sharpening vs reference image. And bonus colors don't get darker, color noise is not enhanced.

RGB version:
FSRCNNX+AS-0_1

LUMA version:
FSRCNNX+AS-LUMA-0_1

@igv
Copy link
Author

igv commented May 7, 2022

Should be even better with the latest changes. With the same curve_height, it looks like, it's a bit sharper now.

@slashbeast
Copy link

slashbeast commented May 7, 2022

Would that be possible for you to publish new revision rather than edit current one?

I am not sure how Github Gist works all that much, but whenever I check for changes, they are there, but the revision claim its still change from 2022.03.28. Hard to keep up when you publish new version. :)

@igv
Copy link
Author

igv commented May 7, 2022

Would that be possible for you to publish new revision rather than edit current one?

Sure, I'll try.

the revision claim its still change from 2022.03.28

I've fixed the date.

@igv
Copy link
Author

igv commented May 15, 2022

@CrHasher SSimDownscaler doesn't oversharp ringing anymore.

@CrHasher
Copy link

CrHasher commented May 16, 2022

@CrHasher SSimDownscaler doesn't oversharp ringing anymore.

Thanks for the info, I tested the new modifications, here are the results:
image

FSRCNNX+dscale-lanczos+AS-0_1 scored lower because the new one RGB or LUMA is apparently better and took all the points from it. The LUMA version scored more points again.

@igv
Copy link
Author

igv commented May 16, 2022

Nice, thanks for your tests.

@yeezylife
Copy link

yeezylife commented Jun 12, 2022

I unsubscribed this gist because I thought I don't need adaptive-sharpen anymore...I was using FSRCNNX+SSimDownscaler (dscale=spline16 and correct-downscaling).
Anyway after I read about CrHasher's tests and igv's coments now,I was thinking maybe there's way to make Adaptivesharpen only works when downscaling.

@yeezylife
Copy link

yeezylife commented Jun 12, 2022

Adaptivesharpen 0.1+dscale=lanczos looks good,but Adaptivesharpen kicks in with upscale too.
I'm currently settled with SSimDownscaler oversharp 0.1 + dscale=lanczos.

@yeezylife
Copy link

yeezylife commented Jun 13, 2022

I try and changed/added these to adaptive-sharpen(Don't really know what I'm doing...)
//!HOOK LUMA
//!WHEN OUTPUT.w LUMA.w / 1.000 < OUTPUT.h LUMA.h / 1.000 < *
Now it seems only works when downscaling!!!

And in your tests did you changed #define overshoot_ctrl to true or should I just use #define overshoot_ctrl false? @CrHasher

@igv
Copy link
Author

igv commented Jun 13, 2022

And to always apply luma version of AS after FSRCNNX just add
//!BIND SUBCONV1 instead of //!WHEN... (doesn't work with gpu-next)

And in your tests did you changed #define overshoot_ctrl to true

No.

@yeezylife
Copy link

yeezylife commented Jun 13, 2022

And I don't seems to find linear_laplace in your latest revision of AS...
Is linear_laplace true necessary or should I just use your latest AS without it?
I mean CrHasher's latest test was in May. Shouldn't have linear_laplace in that version neither...(HolyWu's latest version with linear_laplace was 2 months ago)

@yeezylife
Copy link

yeezylife commented Jun 13, 2022

//!WHEN OUTPUT.w LUMA.w / 1.000 < OUTPUT.h LUMA.h / 1.000 < *
Still have a problem... When FSRCNNX upscales in twice and then downscale, adaptivesharpen will not take effect...

And with //!BIND SUBCONV1:

!BIND SUBCONV1

@igv
Copy link
Author

igv commented Jun 13, 2022

I said it doesn't work with gpu-next

linear_laplace

Don't need anymore

@yeezylife
Copy link

yeezylife commented Jun 13, 2022

Sorry I misunderstood.
Now I added //!WHEN OUTPUT.w OUTPUT.h * LUMA.w LUMA.h * / 1.0 <
and it behaves just like SSIMD, which kicks in when downscaling is needed (after FSRCNNX). And won't kick in if a video only needs upscale.Works well with gpu-next.Perfection!

@igv
Copy link
Author

igv commented Jun 14, 2022

@yeezylife does //!WHEN SUBCONV1.h 0 > work with gpu-next? (I can't test currently)

@yeezylife
Copy link

yeezylife commented Jun 14, 2022

//!HOOK LUMA
//!WHEN SUBCONV1.h 0 >
//!BIND HOOKED
...

222
Doesn't seem to work.

@CrHasher
Copy link

CrHasher commented Jun 14, 2022

What does your latest change do, defaults to LUMA if LUMA_tex is defined guess its defined by mpv right? And why does it not work with gpu-next?

I used previous version with gpu-next or at least I was not seeing errors

@igv
Copy link
Author

igv commented Jun 14, 2022

What does your latest change do, defaults to LUMA if LUMA_tex is defined

LUMA_tex is defined by mpv only if you hook LUMA

And why does it not work with gpu-next?

//!BIND SUBCONV1 does not work with gpu-next

@CrHasher
Copy link

CrHasher commented Jun 14, 2022

What does your latest change do, defaults to LUMA if LUMA_tex is defined

LUMA_tex is defined by mpv only if you hook LUMA

But you have to change the shader right with //!HOOK LUMA ?

And why does it not work with gpu-next?

//!BIND SUBCONV1 does not work with gpu-next

Understood

@CrHasher
Copy link

CrHasher commented Jun 14, 2022

I unsubscribed this gist because I thought I don't need adaptive-sharpen anymore...I was using FSRCNNX+SSimDownscaler (dscale=spline36 and correct-downscaling). Anyway after I read about CrHasher's tests and igv's coments now,I was thinking maybe there's way to make Adaptivesharpen only works when downscaling.

All my tests are only for upscaling not downscaling. No idea how to test downscale... You need a reference image and what would that be in case of a downscale? All I can think of is upscale reference with a quality algo and then use downscale see if you can get close to reference, very unlikely.

And I recommend using no sharpening when downscaling directly with no upscale beforehand example going from 4k to 1080p, or use SSimD in that case.
Whenever FSRCNNX kicks in use a little sharpening aka. AS

@igv
Copy link
Author

igv commented Jun 14, 2022

But you have to change the shader right with //!HOOK LUMA ?

yes, see

@igv
Copy link
Author

igv commented Jun 17, 2022

Whenever FSRCNNX kicks in use a little sharpening aka. AS

SSSR is probably better than AS when upscaling.

@CrHasher
Copy link

CrHasher commented Jun 17, 2022

Whenever FSRCNNX kicks in use a little sharpening aka. AS

SSSR is probably better than AS when upscaling.

What do you mean, what type of upscaling?

@igv
Copy link
Author

igv commented Jun 17, 2022

Like 480p -> 1440p
FSRCNNX+SSSR > FSRCNNX+AS

@CrHasher
Copy link

CrHasher commented Jun 17, 2022

Like 480p -> 1440p FSRCNNX+SSSR > FSRCNNX+AS

Definitely

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment