This is a fix for the formula from: https://en.xen.wiki/w/Color_notation#Converting_a_Ratio_to%2Ffrom_a_Color_Name. Adapted from a code comment I wrote in microtonal-utils/color.js.
Suppose we're converting an interval with stepspan S and magnitude M.
Our monzo is [a b c d e ...>, where a and b are unknown and c, d, e, ... are known.
Let X = stepspan([0 0 c d e ...>) = 16*c + 20*d + 24*e + ...
Y = magnitude([0 2(S-X) c d e ...>) = round((2(S-X) + c + d + e + ...) / 7)
By the definition of stepspan we have:
S = 7*a + 11*b + 16*c + 20*d + 24*e + ...
= 7*a + 11*b + X
And thus: S-X = 7*a + 11*b
This is a linear diophantine equation, all its solutions have the form:
a = -3(S-X) - 11k
b = 2(S-X) + 7k for some k
(Source: https://en.wikipedia.org/wiki/Diophantine_equation#One_equation)
By the definition of magnitude we have:
M = round((b + c + d + e + ...) / 7)
= round((2(S-X) + 7k + c + d + e + ...) / 7)
= round((2(S-X) + c + d + e + ...) / 7) + k
= Y + k
And thus: k = M - Y
So, we conclude:
a = -3(S-X) - 11(M-Y)
b = 2(S-X) + 7(M-Y)
Note that the above does not take into account "co" prefixes, which are independent of S and M.
If C = the number of "co" prefixes, the formula is instead:
a = -3(S-X) - 11(M-Y) + C
b = 2(S-X) + 7(M-Y)