Skip to content

Instantly share code, notes, and snippets.

@Microtribute

Microtribute/bots.txt

Last active November 23, 2020 19:30
Show Gist options
  • Save Microtribute/6673ca0a626ab54534e366cefad2f986 to your computer and use it in GitHub Desktop.
Save Microtribute/6673ca0a626ab54534e366cefad2f986 to your computer and use it in GitHub Desktop.
Download ZIP
User Agents
Raw
bots.txt
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Flipboard/4.2.85
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Flipboard/4.2.90
Mozilla/5.0 (compatible; proximic; +https://www.comscore.com/Web-Crawler)
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
[FBAN/FBIOS;FBDV/iPhone11,6;FBMD/iPhone;FBSN/iOS;FBSV/13.6.1;FBSS/3;FBID/phone;FBLC/en_US;FBOP/5]
Java/1.8.0_171
Mozilla/5.0 (PlayStation; PlayStation 4/8.00) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Safari/605.1.15
Mozilla/5\.0 \(Linux; Android 9; SM\-N950F Build/PPR1\.180610\.011; wv\) AppleWebKit/537\.36 \(KHTML, like Gecko\) Version/4\.0 Chrome/84\.0\.4147\.125 Mobile Safari/537\.36 Instagram 156\.0\.0\.26\.109 Android \(28/9; 560dpi; 1440x2792; samsung; SM\-N950F; greatlte; samsungexynos8895; en_GB; 240726484\)
Mozilla/5.0 (compatible; adidxbot/2.0; +http://www.bing.com/bingbot.htm)
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.268
News Republic/12.7.0 (Linux; U; Android 24; en-us) Mobile Safari
Mozilla/5\.0 \(Linux; Android 7\.0; 5049S\) AppleWebKit/537\.36 \(KHTML, like Gecko\) Chrome/74\.0\.3729\.157 Mobile Safari/537\.36
Mozilla/5\.0 \(Linux; Android 10; J8110 Build/55\.1\.A\.3\.149; wv\) AppleWebKit/537\.36 \(KHTML, like Gecko\) Version/4\.0 Chrome/85\.0\.4183\.81 Mobile Safari/537\.36 \[FB_IAB/FB4A;FBAV/285\.0\.0\.43\.120;\]
Mozilla/5\\.0 \\(Linux; Android 10; J8110 Build/55\\.1\\.A\\.3\\.149; wv\\) AppleWebKit/537\\.36 \\(KHTML, like Gecko\\) Version/4\\.0 Chrome/85\\.0\\.4183\\.81 Mobile Safari/537\\.36 \\[FB_IAB/FB4A;FBAV/285\\.0\\.0\\.43\\.120;\\]
Mozilla/5\.0 \(Linux; Android 7\.0; 5049S\) AppleWebKit/537\.36 \(KHTML, like Gecko\) Chrome/74\.0\.3729\.157 Mobile Safari/537\.36
YelpWebView/1.5 iphone/14_0_1 YelpApp/12.74.0 (x-screen-size 3.0; YelpApplePay/3;
Mozilla/5.0 (en-US) AppleWebKit/537.36 (KHTML, like Gecko; Widget Server) Chrome/85.0.4183.140 Safari/537.36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment