Partial Command History is pretty basic in any *NIX system, but for some reason in Git BASH it's not a default.
This means that when you type partial command in bash and use arrow up/down keys, it goes through only those commands starting with what you have already typed.
In your ~/.inputrc
file you just need to add key bindings:
"\e[A": history-search-backward
"\e[B": history-search-forward
Next time you start a new bash it will work nicely <3