You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Description of approximate conversion algorithm from score V1 to new standardised (score V2)
Rationale
The combo portions of stable's score V1 and score V2 differ in rate of progression. While score V1's combo portion is quadratic in progression in relation to current combo, score V2 is initially logarithmic until 200 combo, after which it is linear. This was demonstrated in prior work in #24823 and #24924, and is apparent on the screenshot below taken from catch's TestSceneScoring:
Notably, catch is the only ruleset wherein the rate of ascent differs between score V1 and score V2. This means, that for all other rulesets, it was completely fine to estimate the combo portion of score V2 / "standardised" scoring by just taking the linear ratio of combo score achieved on the beatmap to maximum combo score achievable on the beatmap.
This approach however breaks down badly in edge cases in catch due to the unequal rates of progression. The naive implementation would disproportionately tank scores with lots of short combos, especially on marathon maps. Thus, the intent of this document is to describe a method of converting score V1's combo portion to score V2.
The remaining details of the implementation do not differ from the gameplay implementation of "standardised" score / score V2, and as such are omitted from this document. We will focus on the derivation of estimation of combo score explicitly.
Derivation
Note that an accurate recomputation here is infeasible; we do not have the data or processing power to do so. Therefore, things will be rather loose and ballpark from here on now, but experimental results showed the method to be performing adequately in most cases.
To begin with, let's note a few things:
For each score, we have the user's maximum combo. Therefore, we know that the user has achieved a combo streak this long.
We do not and cannot know which objects the combo streak consisted of. This matters because fruits and large droplets have different combo score weighting.
Simplification: We will ignore this out of necessity and assume equal weighting.
We do not and cannot know the length of the other combo streaks in the score.
Simplification: We will assume the worst case and presume that the lengths of remaining combo streaks are evenly distributed across the map (excluding the known maximum combo streak, and number of misses). The intuition behind this being the worst case can be gathered by looking at the log-then-linear behaviour of combo score: the longer a combo streak is before a miss, the more points is cumulatively lost after the miss.
This is also apparent on the following screenshots:
10 misses in sequence after 500 combo
10 misses every 100 objects after 500 combo
We will also make a worst-case assumption that every time the player dropped combo, they had already reached 200 combo, therefore reaching the linear part of the curve and causing the maximum loss of combo score. This will punish scores on short maps harsher, where this assumption will generally not hold.
The above simplifications allow us to produce a semblance of estimate of the combo score.
Assuming equal weighting of objects, the combo score for a single object, as a function of current combo, can be approximated by
$$ f(x) = \begin{cases}
0.5 & x < 2 \\
\log_4(x) & 2 \leq x < 200 \\
\log_4(200) & 200 \leq x
\end{cases} $$
For convenience later, let's also denote
$$ F(x) = \int_0^x f(t) dt $$
In human language, large $F$ is the total combo score achieved on a combo of length $x$.
Thus, we can roughly approximate the number of combo score points lost by dropping combo after reaching 200 on a combo streak as
This is generally a bit annoying to compute given the piecewise construction of $f(x)$, so let's just discard the $x < 2$ case. The $x \geq 200$ case actually simplifies itself, as then we have
And therefore, if $x_{max}$ is the maximum possible combo length on the beatmap, $\overline{x}$ is the assumed average length of all remaining combo streaks and $n$ is the number of them,
$$ 1 - \frac{n \cdot \delta(x)}{F(x_{max})} $$
is our ballpark, pessimistic estimate of the ratio of combo points lost by the player on that particular play.
One exception is used when after subtracting the max combo streak and number of misses from max combo, there are not enough objects left to divide them to at-least one-long combo streaks; in that case
$$ \frac{F(x_{max}) - F(x_{play})}{F(x_{max})} $$
where $x_{play}$ is the length of the longest combo streak in the play, is used as the estimation.
Inspection of performance of catch scoring conversion algorithm for selected outlier scores
The general goal here is to reduce the number of incorrect conversions by examining gross outliers and eliminating them wherever possible.
The secondary goal is, whenever it is infeasible to get a somewhat precise estimate of standardised score from score V1, to prefer underestimating standardised score rather than overestimating, to ensure lazer scores can beat imported stable scores.
Most attention was given to:
big relative and absolute increases
big relative and absolute decreases
scores crossing the 1M threshold in either direction
Key examples driving iterations of changes
At the time of writing, the PR went through 3 iterations of changes. Each iteration was driven by a significant shortcoming in the previous iteration. This section will present key examples motivating why the changes were made.
After implementing the naive version of the conversion (by just adjusting the combo and "accuracy" portions to match score V2) this revealed that the combo estimation could be way off for catch scores, due to a difference between the growth rate of combo score between score V1 and score V2 on stable.
Stable's score V1 is quadratic in nature, while score V2 is initially logarithmic and then linear. This means that on scores with a lot of combo breaks, linearly rescaling the achieved combo score relative to the maximum achievable combo score would massively underweight scores that had multiple combo breaks, because of the influence of the quadratic.
This spurred the implementation of the alternative method of estimating the combo portion for catch. Notably, this is a catch-specific issue, as it is the only ruleset in which the growth rate of combo score noticeably differs between score V1 and score V2:
osu! combo score is quadratic both in score V1 and score V2.
taiko and mania combo score is linear both in score V1 and score V2 (to a degree - it's actually logarithmic and then linear in score V2, but it will not cause such a massive difference in practice).
This was counteracted by a separate estimation of the combo portion for catch.
The first iteration of the estimation of combo portion mentioned in the previous section was optimistic, by assuming that the player had hit as many combo streaks of length equal to the max combo as possible. This ended up overestimating scores that had many misses, by grossly overestimating how many combo streaks they achieved.
This spurred a change to the algorithm to err on the pessimistic side: the second implementation assumed that the player hit one combo streak of the length equal to max combo, and the remaining combo streaks are divided equally in length to fill out max_beatmap_combo - max_score_combo - score_miss_count. (If that turns out to floor to zero, then it is presumed that the player hit a single combo streak of length equal to max combo, then missed everything else.)
The above method is the pessimistic estimate, as missing after a long combo streak is more costly than missing multiple objects in quick succession. (You can use the catch's TestSceneScoring to experimentally verify this.)
Final score after replay playback with score V2 active (stable)1
N/A (no replay)
Final score after replay playback (lazer)
N/A (no replay)
Not able to accurately investigate this one due to lack of replay, but there are some hints, namely that the score is a FC but has lots of droplets missed.
I decided to investigate this one by setting a similar score with nomod on stable wherein I made sure to get FC but miss as many droplets as feasible. I got:
37,528 score on stable at 44.44% accuracy
748,130 score when above is imported to lazer
745,374 score when playing back said imported replay in lazer
779,646 score when taking said initial replay, binary-modifying it to specify Score V2, and playing that back again in stable with Score V2 active
748,460 score when applying 0.96x multiplier to above
Causes of behaviour:
Correct modeling of portions of stable score V2
Correct estimation of combo portion in terms of matching score V2
Causes of behaviour:
This was and continues to be wrong, as it should not really be possible for any score to exceed 1M after conversion. The root cause is incorrect legacy score attribute calculation, which spurs from the non-matching calculation of difficultyPeppyStars between stable and lazer. The unrounded value for this beatmap is 4.5, which:
stable rounds up to 5
lazer rounds down to 4
This has a knock-on effect of majorly underestimating maximum achievable combo portion, and thus majorly overestimating achieved bonus portion, which ends up amounting to almost 40 million points.
In order to match score V2, this pull increases the value of bananas caught from 50 to 200 points. This has the effect of increasing BonusScore and BonusScoreRatio in legacy scoring attributes fourfold, and thus explains the difference.
This is likely to be an issue across all rulesets and as such will not be addressed in this pull.
Verdict: Was and continues to be broken, fix may be attempted independently as it may impact beatmaps in all rulesets
wherein stable rounds 3.5 down to 3, and lazer rounds it up to 4. This causes the combo portion to be overestimated by lazer, but due to clamping applied to fix previous issues (estimated bonus portion is clamped from below to 0), scores receive much more correct (yet still not completely correct) totals.
Causes of behaviour:
This is a 23-minute long gimmick catch map with tiny droplet spam, a long combo streak in the middle that literally plays itself (catcher is in one position for minutes on end), and NaN abuse in control point specs. Map also looks wildly differently between stable and lazer (has a weird juice stream on far right of playfield at start that's seemingly not there on stable).
Verdict:
I honestly can't find it in myself to try and debug this one. Map is broken, should not be allowed to have a leaderboard.
Causes of behaviour:
This is a beatmap-specific issue.
The beatmap has a difficultyPeppyStars of 0.
Therefore, the combo portion as estimated by CatchLegacyScoreSimulator was 0.
Therefore, the comboProportion in the standardised score migration code was also estimated as 0 - but actually should have been 1, as the score is an FC.
Therefore, previously only $400000 \cdot 0.96 = 384000$ points would be awarded rather than the correct 960000.
Verdict: Working as intended. The estimation of comboProportion for remaining rulesets likely needs to be revised to anticipate this scenario happening on other beatmaps, likely by checking for active mods (if relax/autopilot active, assume combo portion of 0, else 1).
Similar cases that will not be discussed further:
All scores on this map.
Causes of behaviour:
The explanation for this one is simple. This is a gimmick map, which consists only of banana showers. Thus, it has 0x maximum combo. (Notably, on stable, score V2 totally does not handle this, and refuses to grant any points even if hitting bananas.)
With the current implementation, this means that the player will get 1M score automatically for free, and then any bananas caught will give 200 points per banana. So the old score automatically looks wrong.
Now, given the special circumstances, we have the following info:
Stable shows 6,600 total score. Each banana gives 1,100 score on stable given score V1. This means 6 bananas were hit.
In score V2, and thus in lazer standardised, each banana gives 200 points.
Classic mod has 0.96x multiplier.
Thus,
$$ (1000000 + 6 \cdot 200) \cdot 0.96 = 961152 $$
QED.
Verdict: Working as intended.
At this point I concluded that the "gain" side of the sheet looks solid enough and I don't see anything obviously broken.
Causes of behaviour:
Complete mess.
Beatmap uses NaN lengths on some slider objects in .osu file.
Leading to some objects being straight up not parsed by lazer.
Thus, the combo portion estimation actually goes negative, but is saved by the clamp to [0,1].
Verdict:
Well this is one of the truly unlucky ones.
Score got majorly hammered by the negative combo estimation.
Not sure it's worth doing much to salvage a C rank score.
Verdict:
Another harsh hit from combo estimation, as above.
For rank D, seems like it's probably fine to keep as is, though.
Footnotes
This was achieved by binary modifying the .osr file to specify score V2 as an active mod for the replay, playing it back until the end, and observing the final total score. ↩↩2↩3↩4↩5↩6↩7↩8↩9↩10↩11↩12↩13↩14↩15
lazer applies 0.96x multiplier due to classic mod being attached to stable scores, so this is applied onto stable total for comparison purposes. ↩↩2↩3↩4↩5↩6↩7↩8↩9↩10↩11↩12↩13↩14
While this is provided for general reference to ensure that no score value from actual playback clearly deviates from the rest, the score value after lazer's replay playback may be inaccurate due to inaccuracies of the actual replay playback. Hit statistics after playing back a replay routinely do not match the statistics stored to the replay. ↩↩2↩3↩4↩5↩6↩7↩8↩9↩10↩11↩12↩13↩14
