Skip to content

Instantly share code, notes, and snippets.

@cr0hn

cr0hn/agents.md

Forked from Paratron/agents.md
Last active October 28, 2020 03:17
Show Gist options
  • Save cr0hn/ff254d47376f752795d3 to your computer and use it in GitHub Desktop.
Save cr0hn/ff254d47376f752795d3 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
agents.md

#Social Network Crawler User Agents Users can post URLs on a lot of different platforms nowadays. Most of those platforms will send a request to that URL to generate some preview data from it.

These are a couple of user agents I quickly tested out.

##Facebook

facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)

##Google+

Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20110814 Firefox/6.0 Google (+https://developers.google.com/+/web/snippet/)

##Xing

XING-contenttabreceiver/2.0

##LinkedIn

LinkedInBot/1.0 (compatible; Mozilla/5.0; Jakarta Commons-HttpClient/3.1 +http://www.linkedin.com)

##Delicious

Java/1.7.0_45

##Twitter

Twitterbot

Pinterest

Pinterest/0.1 +http://pinterest.com/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment