This issue revolves around two aspects: the perception of the downvotee, and the behavior of the downvoter.
Removing the "You" from the title will help to avoid the downvotee assuming it's a personal attack. However, this only works if the downvoter keeps the same intent.
I'm a Room Owner of the SOCVR room (Aralun is one of the regulars). When our room started we were just 3 members hoping to make a dent in the CV queue. Now, we've got 10+ regulars and have gained some serious power. We can close any question on-demand.
This is a serious responsibility and had a big meeting recently to address it. The big take away is to ensure that we don't ever target a user. Aralun talked at length about that here in the first post.
Along with addressing how we close vote, the other main aspect from the meeting was user communication behavior, whether in the comments or in chat.
How we say something, especially in an environment where it is difficult to correctly convey emotions, is critical. We should be thrilled that a user hopped in chat. Now you have a chance to guide them, not criticize them to the ground. Don't show them the error of their ways, show them how to get back on the path. This won't be an easy thing; we know first hand how much unsalvageable content is on the site. Just remember that everyone who posts stuff here is human. Not everyone is Jon Skeet, but you should treat them like they were.
In your first dialog example:
- "You downvoted me! Why did you do that, you jerk! This is important!"
- "No, I didn't downvote you, I downvoted this question of yours. See, I don't hate you, I hate your post. Here's a link you can use to fix it. I'll remove my downvote if you do."
- "You... Wow, thanks, hero."
Yes, it's easy to take a pure defencive stance when someone accuses you of doing something wrong. However, it should be our goal to avoid argument, and focus first on editing the content.
Instead of a defensive stance for the downvotee, they could have a positive attitude.
"I downvoted your post because it's X, Y, and Z. Here's a link you can use to fix it. I'll remove my downvote if you do."
"Here's a link you should read through. It will help you fix some problems in your post, like X and Y."
The rules of Stack Overflow is to downvote bad content. Don't punish the downvotee for that, but they need to be accountable for user interactions if they comment.
To sum it up: I think the "you" should be dropped, but the downvotee needs to be the bigger person. We can't expect this site to fix that behavior, but we can get halfway there.
/me appends Spearrel seal of Approval