BASH is a text-based shell for controlling your computer (or operating system). Bash is actually an acronym which stands for Bourne-again shell. It replaced the Bourne shell and "bashed together" the unix programs sh, csh and ksh. From it you can navigate the files on your computer and execute programs. You can also connect to other computers and basically do everything you can do in your GUI Operating System (like OS X or Windows).
When you open a terminal, you're basically within your file system, or in a directory, just like you are when you open a Finder window or an Explorer window.