wpPERFORM.com's recommended robots.txt file

robots.txt

User-agent: *
Allow: /*.js*
Allow: /*.css*
Disallow: /*?
Disallow: /feed/
Disallow: */feed/
Disallow: /comments/
Disallow: */comments/
Disallow: /trackback/
Disallow: */trackback/
Disallow: /wp-content/upgrade/
# WP admin directories
# GoogleBot needs access to js & css, and Google's order of precedence blocks them if they are disallowed here 
#Disallow: /wp-admin/
#Disallow: /wp-includes/
#Disallow: /wp-content/themes/
#Disallow: /wp-content/plugins/
#Disallow: /wp-content/mu-plugins/

# if you use NextGen Gallery, uncomment the next line and replace {siteID} with your numeric site ID
#Disallow: /wp-content/uploads/sites/{siteID}/gallery/cache

# if you use the WP-Print plugin, uncomment the next 2 lines
#Disallow: /print/
#Disallow: */print/

Allow: /

# if you want to block the Internet Archive crawler, https://archive.org/details/archive.org_bot, uncomment the next 2 lines
#User-agent: archive.org_bot*
#Disallow: /