ClaimsMappingPolicy-GroupFilter.ps1

# First, connect to Azure AD
Connect-AzureAD

# Make sure there is no same policy already defined
# Get your policies and inspect them
Get-AzureADPolicy

# modify your gorup filter as appropriate
# ref: https://learn.microsoft.com/en-us/azure/active-directory/develop/reference-claims-mapping-policy-type#group-filter
$PolicyDefinitionString = "{
  `"ClaimsMappingPolicy`":
    {
      `"Version`":1,
      `"IncludeBasicClaimSet`":`"true`", 
      `"GroupFilter`": 
        {
          `"MatchOn`":`"displayname`",
          `"Type`":`"prefix`",
          `"Value`":`"r_k8s`"
        }
    }
  }"

# create the claims mapping policy with group filter
$policy = New-AzureADPolicy -Definition  $PolicyDefinitionString `
            -DisplayName "GroupFilter_r_k8s" `
            -Type "ClaimsMappingPolicy"

# get the service principal for which you want to apply the policy
# make sure you use the object id of the service pricipal, not the app registration!
$sp = Get-AzureADServicePrincipal -ObjectId fa63c4fd-0ce4-45cb-a72e-5f4517ea94fc

# map the policy to the service principal
Add-AzureADServicePrincipalPolicy -Id $sp.objectId -RefObjectId $policy.Id

On line 34 (Add-AzureADServicePrincipalPolicy) we are getting an access denied error. The user running the script is an owner of the app registration. Do we need to have elevated privileges to enable this preview feature, or admin on the Azure AD tenant, something more than app registration owner?

You must be an owner of the service principal too. Owner of the app registration is not owner of the service principal.