Well, ultimately music is always subjective. You can't really argue with someone who likes some particular music – they like what they like, and it would be dogmatism to condemn some composition just because it violates the theoretical frameworks† that you happen to know.
That said, I personally would probably agree with your assessment. I don't think much of music that's just too arbitrary – not because it's bad to violate the standard theory, but because music should have a purpose. This is regardless of whether that purpose can be explained within Common Practice theory, with deliberately violating the common rules, with providing a completely new theoretical framework, or something else.
In other words, “Why it couldn't go there anyways?” is just not the right question to ask to rebut criticism. The right rebuttal is to give a reason why it would go there.
If he picked that particular harmony for a reason – whatever reason – then there's that. In doubt, the reason could simply be “I li