Requires sigmoid-upscaling=no. To use it on-demand add the following line to input.conf: n change-list glsl-shaders toggle "~~/adaptive-sharpen.glsl"

adaptive-sharpen.glsl

// Copyright (c) 2015-2020, bacondither
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions
// are met:
// 1. Redistributions of source code must retain the above copyright
//    notice, this list of conditions and the following disclaimer
//    in this position and unchanged.
// 2. Redistributions in binary form must reproduce the above copyright
//    notice, this list of conditions and the following disclaimer in the
//    documentation and/or other materials provided with the distribution.
//
// THIS SOFTWARE IS PROVIDED BY THE AUTHORS ``AS IS'' AND ANY EXPRESS OR
// IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
// OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
// IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
// INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
// NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
// THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

// Adaptive sharpen - version 2020-11-14
// Tuned for use post-resize, EXPECTS FULL RANGE GAMMA LIGHT (requires ps >= 3.0)

//!HOOK SCALED
//!BIND HOOKED
//!DESC adaptive-sharpen

//--------------------------------------- Settings ------------------------------------------------

#define curve_height    1.0                  // Main control of sharpening strength [>0]
                                             // 0.3 <-> 2.0 is a reasonable range of values

#define anime_mode      false                // Only darken edges

#define overshoot_ctrl  false                // Allow for higher overshoot if the current edge pixel
                                             // is surrounded by similar edge pixels

#define video_level_out false                // True to preserve BTB & WTW (minor summation error)
                                             // Normally it should be set to false

// Defined values under this row are "optimal" DO NOT CHANGE IF YOU DO NOT KNOW WHAT YOU ARE DOING!

#define curveslope      0.5                  // Sharpening curve slope, high edge values

#define L_overshoot     0.003                // Max light overshoot before compression [>0.001]
#define L_compr_low     0.167                // Light compression, default (0.167=~6x)
#define L_compr_high    0.334                // Light compression, surrounded by edges (0.334=~3x)

#define D_overshoot     0.009                // Max dark overshoot before compression [>0.001]
#define D_compr_low     0.250                // Dark compression, default (0.250=4x)
#define D_compr_high    0.500                // Dark compression, surrounded by edges (0.500=2x)

#define scale_lim       0.1                  // Abs max change before compression [>0.01]
#define scale_cs        0.056                // Compression slope above scale_lim

#define pm_p            0.5                  // Power mean p-value [>0-1.0]
//-------------------------------------------------------------------------------------------------

#define max4(a,b,c,d)  ( max(max(a, b), max(c, d)) )

// Soft if, fast linear approx
#define soft_if(a,b,c) ( sat((a + b + c + 0.025455)/(maxedge + 0.013636) - 0.85) )

// Soft limit, modified tanh approx
#define soft_lim(v,s)  ( sat(abs(v/s)*(27.0 + pow(v/s, 2.0))/(27.0 + 9.0*pow(v/s, 2.0)))*s )

// Weighted power mean
#define wpmean(a,b,w)  ( pow(w*pow(abs(a), pm_p) + abs(1.0-w)*pow(abs(b), pm_p), (1.0/pm_p)) )

// Get destination pixel values
#define get(x,y)       ( sat(HOOKED_texOff(vec2(x, y)).rgb) )
#define sat(x)         ( clamp(x, 0.0, 1.0) )
#define dxdy(val)      ( length(fwidth(val)) ) // edgemul = 2.2

#define CtL(RGB)       ( dot(RGB*RGB, vec3(0.2126, 0.7152, 0.0722)) )

#define b_diff(pix)    ( abs(blur-c[pix]) )

vec4 hook() {

    // [                c22               ]
    // [           c24, c9,  c23          ]
    // [      c21, c1,  c2,  c3, c18      ]
    // [ c19, c10, c4,  c0,  c5, c11, c16 ]
    // [      c20, c6,  c7,  c8, c17      ]
    // [           c15, c12, c14          ]
    // [                c13               ]
    vec3 c[25] = vec3[](get( 0, 0), get(-1,-1), get( 0,-1), get( 1,-1), get(-1, 0),
                        get( 1, 0), get(-1, 1), get( 0, 1), get( 1, 1), get( 0,-2),
                        get(-2, 0), get( 2, 0), get( 0, 2), get( 0, 3), get( 1, 2),
                        get(-1, 2), get( 3, 0), get( 2, 1), get( 2,-1), get(-3, 0),
                        get(-2, 1), get(-2,-1), get( 0,-3), get( 1,-2), get(-1,-2));

    float e[13] = float[](dxdy(c[0]),  dxdy(c[1]),  dxdy(c[2]),  dxdy(c[3]),  dxdy(c[4]),
                          dxdy(c[5]),  dxdy(c[6]),  dxdy(c[7]),  dxdy(c[8]),  dxdy(c[9]),
                          dxdy(c[10]), dxdy(c[11]), dxdy(c[12]));

    // Blur, gauss 3x3
    vec3  blur   = (2.0 * (c[2]+c[4]+c[5]+c[7]) + (c[1]+c[3]+c[6]+c[8]) + 4.0 * c[0]) / 16.0;

    // Contrast compression, center = 0.5, scaled to 1/3
    float c_comp = sat(0.266666681f + 0.9*exp2(dot(blur, vec3(-7.4/3.0))));

    // Edge detection
    // Relative matrix weights
    // [          1          ]
    // [      4,  5,  4      ]
    // [  1,  5,  6,  5,  1  ]
    // [      4,  5,  4      ]
    // [          1          ]
    float edge = length( 1.38*b_diff(0)
                       + 1.15*(b_diff(2) + b_diff(4) + b_diff(5) + b_diff(7))
                       + 0.92*(b_diff(1) + b_diff(3) + b_diff(6) + b_diff(8))
                       + 0.23*(b_diff(9) + b_diff(10) + b_diff(11) + b_diff(12)) ) * c_comp;

    vec2 cs = vec2(L_compr_low,  D_compr_low);

    if (overshoot_ctrl) {
        float maxedge = max4( max4(e[1],e[2],e[3],e[4]), max4(e[5],e[6],e[7],e[8]),
                              max4(e[9],e[10],e[11],e[12]), e[0] );

        // [          x          ]
        // [       z, x, w       ]
        // [    z, z, x, w, w    ]
        // [ y, y, y, 0, y, y, y ]
        // [    w, w, x, z, z    ]
        // [       w, x, z       ]
        // [          x          ]
        float sbe = soft_if(e[2],e[9], dxdy(c[22]))*soft_if(e[7],e[12],dxdy(c[13]))  // x dir
                  + soft_if(e[4],e[10],dxdy(c[19]))*soft_if(e[5],e[11],dxdy(c[16]))  // y dir
                  + soft_if(e[1],dxdy(c[24]),dxdy(c[21]))*soft_if(e[8],dxdy(c[14]),dxdy(c[17]))  // z dir
                  + soft_if(e[3],dxdy(c[23]),dxdy(c[18]))*soft_if(e[6],dxdy(c[20]),dxdy(c[15])); // w dir

        cs = mix(cs, vec2(L_compr_high, D_compr_high), sat(2.4002*sbe - 2.282));
    }

    // RGB to luma
    float luma[25] = float[](CtL(c[0]), CtL(c[1]), CtL(c[2]), CtL(c[3]), CtL(c[4]), CtL(c[5]), CtL(c[6]),
                             CtL(c[7]),  CtL(c[8]),  CtL(c[9]),  CtL(c[10]), CtL(c[11]), CtL(c[12]),
                             CtL(c[13]), CtL(c[14]), CtL(c[15]), CtL(c[16]), CtL(c[17]), CtL(c[18]),
                             CtL(c[19]), CtL(c[20]), CtL(c[21]), CtL(c[22]), CtL(c[23]), CtL(c[24]));

    float c0_Y = sqrt(luma[0]);

    // Precalculated default squared kernel weights
    const vec3 w1 = vec3(0.5,           1.0, 1.41421356237); // 0.25, 1.0, 2.0
    const vec3 w2 = vec3(0.86602540378, 1.0, 0.54772255751); // 0.75, 1.0, 0.3

    // Transition to a concave kernel if the center edge val is above thr
    vec3 dW = pow(mix( w1, w2, sat(2.4*edge - 0.82)), vec3(2.0));

    // Use lower weights for pixels in a more active area relative to center pixel area
    // This results in narrower and less visible overshoots around sharp edges
    float modif_e0 = 3.0 * e[0] + 0.0090909;

    float weights[12]  = float[](( min(modif_e0/e[1],  dW.y) ),
                                 ( dW.x ),
                                 ( min(modif_e0/e[3],  dW.y) ),
                                 ( dW.x ),
                                 ( dW.x ),
                                 ( min(modif_e0/e[6],  dW.y) ),
                                 ( dW.x ),
                                 ( min(modif_e0/e[8],  dW.y) ),
                                 ( min(modif_e0/e[9],  dW.z) ),
                                 ( min(modif_e0/e[10], dW.z) ),
                                 ( min(modif_e0/e[11], dW.z) ),
                                 ( min(modif_e0/e[12], dW.z) ));

    weights[0] = (max(max((weights[8]  + weights[9])/4.0,  weights[0]), 0.25) + weights[0])/2.0;
    weights[2] = (max(max((weights[8]  + weights[10])/4.0, weights[2]), 0.25) + weights[2])/2.0;
    weights[5] = (max(max((weights[9]  + weights[11])/4.0, weights[5]), 0.25) + weights[5])/2.0;
    weights[7] = (max(max((weights[10] + weights[11])/4.0, weights[7]), 0.25) + weights[7])/2.0;

    // Calculate the negative part of the laplace kernel and the low threshold weight
    float lowthrsum   = 0.0;
    float weightsum   = 0.0;
    float neg_laplace = 0.0;

    for (int pix = 0; pix < 12; ++pix)
    {
        float lowthr = clamp((29.04*e[pix + 1] - 0.221), 0.01, 1.0);

        neg_laplace += luma[pix+1] * weights[pix] * lowthr;
        weightsum   += weights[pix] * lowthr;
        lowthrsum   += lowthr / 12.0;
    }

    neg_laplace = inversesqrt(weightsum / neg_laplace);

    // Compute sharpening magnitude function
    float sharpen_val = curve_height/(curve_height*curveslope*pow(edge, 3.5) + 0.625);

    // Calculate sharpening diff and scale
    float sharpdiff = (c0_Y - neg_laplace)*(lowthrsum*sharpen_val + 0.01);

    // Calculate local near min & max, partial sort
    float temp;

    for (int i1 = 0; i1 < 24; i1 += 2)
    {
        temp = luma[i1];
        luma[i1]   = min(luma[i1], luma[i1+1]);
        luma[i1+1] = max(temp, luma[i1+1]);
    }

    for (int i2 = 24; i2 > 0; i2 -= 2)
    {
        temp = luma[0];
        luma[0]    = min(luma[0], luma[i2]);
        luma[i2]   = max(temp, luma[i2]);

        temp = luma[24];
        luma[24] = max(luma[24], luma[i2-1]);
        luma[i2-1] = min(temp, luma[i2-1]);
    }

    for (int i1 = 1; i1 < 24-1; i1 += 2)
    {
        temp = luma[i1];
        luma[i1]   = min(luma[i1], luma[i1+1]);
        luma[i1+1] = max(temp, luma[i1+1]);
    }

    for (int i2 = 24-1; i2 > 1; i2 -= 2)
    {
        temp = luma[1];
        luma[1]    = min(luma[1], luma[i2]);
        luma[i2]   = max(temp, luma[i2]);

        temp = luma[24-1];
        luma[24-1] = max(luma[24-1], luma[i2-1]);
        luma[i2-1] = min(temp, luma[i2-1]);
    }

    float nmax = (max(sqrt(luma[23]), c0_Y)*2.0 + sqrt(luma[24]))/3.0;
    float nmin = (min(sqrt(luma[1]),  c0_Y)*2.0 + sqrt(luma[0]))/3.0;

    float min_dist  = min(abs(nmax - c0_Y), abs(c0_Y - nmin));
    float pos_scale = min_dist + L_overshoot;
    float neg_scale = min_dist + D_overshoot;

    pos_scale = min(pos_scale, scale_lim*(1.0 - scale_cs) + pos_scale*scale_cs);
    neg_scale = min(neg_scale, scale_lim*(1.0 - scale_cs) + neg_scale*scale_cs);

    // Soft limited anti-ringing with tanh, wpmean to control compression slope
    sharpdiff = (anime_mode ? 0. :
                wpmean(max(sharpdiff, 0.0), soft_lim( max(sharpdiff, 0.0), pos_scale ), cs.x ))
              - wpmean(min(sharpdiff, 0.0), soft_lim( min(sharpdiff, 0.0), neg_scale ), cs.y );

    float sharpdiff_lim = sat(c0_Y + sharpdiff) - c0_Y;
    float satmul = (c0_Y + max(sharpdiff_lim*0.9, sharpdiff_lim)*1.03 + 0.03)/(c0_Y + 0.03);
    vec3 res = c0_Y + (sharpdiff_lim*3.0 + sharpdiff)/4.0 + (c[0] - c0_Y)*satmul;

    return vec4(video_level_out == true ? res + HOOKED_texOff(0).rgb - c[0] : res, HOOKED_texOff(0).a);
}

Hi igv,

first of all great job on your AdaptiveSharpen modifications! Looks better to my eyes than the original version.

FYI, I had replaced the original AdaptiveSharpen version with your's (the Sigmoid version) in madVR last week, but then some madVR users have complained that they liked the original version better than your's. After doing some tests and comparisons, I think there are 2 reasons for that:

1. The Sigmoid version sharpened grain too much. Your latest changes from yesterday already fixed that (thanks).
2. I was running the original version in gamma light, while your's wants to be run (as far as I understand) in linear light. It seems that especially for Anime content, gamma light might work better. Here's is a screenshot comparison, using bjin's "hires" image from here:

https://github.com/bjin/mpv-prescalers/wiki/Comparison

Upscaled 200% with NGU Anti-Alias, then post resize AdaptiveSharpen, with gamma light vs linear light. I've used your latest script (from yesterday) for both images. In order to produce the gamma light image I've simply changed "vec3 res = GammaInv(Gamma(c[0].rgb) + sharpdiff);" to "vec3 res = c[0].rgb + sharpdiff;", and of course shader input and output then needs to be gamma light.

To my eyes the gamma light image looks slightly "better", which is probably due to the lines appearing slightly thinner. I'm not completely sure yet if gamma light always looks better, or if maybe this is specific to Anime content. For now I've added an option to let madVR users choose between gamma light and linear light sharpening. Based on user feedback I'll decide whether to keep the linear light option or remove it.

Anyway, just wanted to get this information back to you, in case you also want to give mpv users the choice between gamma and linear light sharpening.

Hello madshi.

first of all great job on your AdaptiveSharpen modifications! Looks better to my eyes than the original version.

Thanks.

In order to produce the gamma light image I've simply changed "vec3 res = GammaInv(Gamma(c[0].rgb) + sharpdiff);" to "vec3 res = c[0].rgb + sharpdiff;"

You also need to linearize RGB.rgb on line 71 in that case.
And using precise luma calculation might help as well
#define CtL(RGB) sqrt(dot(vec3(0.2558, 0.6511, 0.0931), pow(GammaInv(RGB.rgb), vec3(2.0)) ))

Oh, and don't use default fast GammaInv for this, pass TRC to the shader.

Thanks. Hmmmm... Do you really mean "pow(GammaInv(RGB.rgb), vec3(2.0))" with brackets set like that? That's doing Gamma -> Linear on the gamma light RGB pixel twice in a row. That seems weird to me? #;-O I'm not actually sure, though, maybe it has a purpose?

Do you really mean "pow(GammaInv(RGB.rgb), vec3(2.0))" with brackets set like that?

Yes, sqrt( .299*R^2 + .587*G^2 + .114*B^2 ), and you always calculate luma from linear light (incorrect luma calculation is one of the main reasons there are ringing artifacts with original AS).

I'm sorry, I'm probably being stupid, but what I find confusing is that your latest shader just does this:

#define CtL(RGB) ( dot(vec3(0.2558, 0.6511, 0.0931), RGB.rgb) )

There's no funky sqrt( ^2 + ^2 + ^2 ) going on in your original CtL define. Of course in my case I'm now feeding the shader with gamma light instead of linear light, so in order to calculate linear light CtL, I may have to convert the pixel from gamma light to linear light first, but where does this need for sqrt( ^2 + ^2 + ^2 ) come from? Shouldn't I simply convert my gamma light "RGB.rgb" to linear light, and then feed it into your unmodified CtL define? So shouldn't it be simply the following?

#define CtL(RGB) ( dot(vec3(0.2558, 0.6511, 0.0931), GammaInv(RGB.rgb)) )

I guess my math abilities are leaving me, or maybe it's too late in the evening... :(

Yeah, I use fast RGB to Luma formula.
See https://stackoverflow.com/questions/596216/formula-to-determine-brightness-of-rgb-color
For this shader it looks like there is no much difference between these 2 formulas, but, for example, SSimSuperRes looks much better with more precise (perceived option 2) RGB->Luma formula. I'm just suggesting you to try precise one, maybe you will like it better.

Ah, now I fully understand! :) Thanks for pointing me to the "perceived option 2" formula, I don't think I've seen it before.

madshi, you can make this part optional for anime to make edges darker.

	float sharpdiff_lim = saturate(c0_Y + sharpdiff) - c0_Y;
	float satmul = (c0_Y + max(sharpdiff_lim*0.9, sharpdiff_lim) + 0.03)/(c0_Y + 0.03);
	float3 res = c0_Y + (sharpdiff_lim*3 + sharpdiff)/4 + (c[0].rgb - c0_Y)*satmul;

@igv, thank you, I appreciate that!

I still have on my to do list to re-investigate all the various AdaptiveSharpen versions. Some anime users have complained that they liked the original version better (although it had ringing problems). Maybe they will be satisfied with your edge darkening suggestion. I'll update you when I reach a "final" AdaptiveSharpen version that my users are happy with. But it could take a while, sooo busy...

(Didn't get notification about your new comment, for some reason, so my reply is late.)

How do you make edges in anime darker in the current version?

You can't. My advice above doesn't work. In upstream adaptive-sharpen edges are darker because of a bug - information (c_edge in particular) gets truncated between passes.

Would you be willing to port AMDs Contrast Adaptive Sharpening into a glsl file? I think most of the work is already done here. (Though I personally couldn't get it to work with mpv)

No, why downgrade? Adaptive-sharpen is much better. CAS is just an unsharp masking filter (sharpen in mpv) that adaptively calculates this weights.
Ask mpv maintainers maybe to upgrade sharpen to CAS.

Alright. Thanks for the insight. I will ask the mpv devs to update their sharpening filter.

Here's my mpv shader port of CAS: https://github.com/deus0ww/mpv-conf/tree/master/shaders/cas

In most cases, I would NOT use this over igv's Adaptive Sharpen, especially on noisy sources.

Here's my mpv shader port of CAS: https://github.com/deus0ww/mpv-conf/tree/master/shaders/cas

In most cases, I would NOT use this over igv's Adaptive Sharpen, especially on noisy sources.

Oh hey. Thanks.

Is there a reason why FSRCNNX 16-0-4-1 is in archive? Should we just use FSRCNNX_x2_8-0-4-1 from now on?

No, but yes.

Any plans to release an improved version for 16-0-4-1? That would be great. According to the tests here, https://artoriuz.github.io/blog/mpv_upscaling.html, the 16-0-4-1 model is still better albeit has a much higher performance cost. Thanks for your continual work on this.

Difference <0.5 dB (PSNR(-HMA)) is invisible without zooming in.

What does enabling overshoot_ctrl do, exactly? (I'm not familiar with image processing/DSP terms.)
Does it increase sharpening strength near thicker edge pixel clusters, reduce sharpening strength in edge-less, "texture" areas, or something else?

For example pixels inside of a bright circle like edge will be made darker.
L_compr_high and D_compr_high params is maximal strength of overshoot.