"Whom" is the so called oblique case of "who" (which itself is the nominative case). That gives ponies trouble, because pronouns are the only thing left in the English language that still retain the distinction between the two. But the rules are simple: nominative is used for subjects of actions (ie. "who did X?") and oblique is used for objects of actions (ie. "X was done to whom?"). And "who" to "whom" is the same thing as "I" to "me", "he" to "him", "she" to "her", "we" to "us", etc. If in doubt, rewrite with "him" instead for "whom", and see if it works:
- [✓] Whom should I ask? → I should ask him
- [✗]
Whom goes there?→Him goes there - [✗]
The pony whom directs a film is called a director→Him directs a film - [✓] The pony whom I loved the most was my father → I loved him the most