Use chmod +X (as opposed to +x) which will make only directories executable and leave files alone.
This is great for recursively fixing a Drupal files directory. For example:
chmod -R 664 sites/default/files && chmod -R a+X sites/default/files
That will make all your files writeable by you and the server, but not executable, but will also make directories executable (i.e., listable).