Skip to content

Instantly share code, notes, and snippets.

@eliasdabbas

eliasdabbas/user_agent_to_data_frame.py

Created November 30, 2021 16:11
Show Gist options
  • Save eliasdabbas/6fae36e14c563a83620f785bb91eb8e3 to your computer and use it in GitHub Desktop.
Save eliasdabbas/6fae36e14c563a83620f785bb91eb8e3 to your computer and use it in GitHub Desktop.
Download ZIP
From a list of user agents to a DataFrame of parsed UA's
Raw
user_agent_to_data_frame.py
user_agents = [
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36',
'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36',
'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36',
'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0',
'Mozilla/5.0 (Windows NT 6.1; rv:64.0) Gecko/20100101 Firefox/64.0',
'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0',
'Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)',
'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
'Mozilla/5.0 (iPhone; CPU iPhone OS 12_1_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1',
'Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36',
'Googlebot-Image/1.0',
'Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0',
'Mozilla/5.0 (Windows NT 5.1; rv:52.0) Gecko/20100101 Firefox/52.0',
'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36',
'Mozilla/5.0 (compatible; AhrefsBot/6.1; +http://ahrefs.com/robot/)',
'Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:16.0)Gecko/16.0 Firefox/16.0',
'Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko',
'Mozilla/5.0 (Windows NT 6.3; rv:64.0) Gecko/20100101 Firefox/64.0',
'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36'
]
from ua_parser import user_agent_parser
import pandas as pd
pd.options.display.max_columns = None
pd.json_normalize([user_agent_parser.Parse(ua) for ua in user_agents])
@eliasdabbas
Copy link
Author

Running the above script would produce the following table:

string user_agent.family user_agent.major user_agent.minor user_agent.patch os.family os.major os.minor os.patch os.patch_minor device.family device.brand device.model
0 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36 Chrome 71 0 3578 Windows 10 Other
1 Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36 Chrome 71 0 3578 Windows 7 Other
2 Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36 Chrome 71 0 3578 Windows 7 Other
3 Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0 Firefox 64 0 Windows 10 Other
4 Mozilla/5.0 (Windows NT 6.1; rv:64.0) Gecko/20100101 Firefox/64.0 Firefox 64 0 Windows 7 Other
5 Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Googlebot 2 1 Android 6 0 1 Spider Spider Smartphone
6 Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0 Firefox 64 0 Windows 7 Other
7 Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) bingbot 2 0 Other Spider Spider Desktop
8 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Googlebot 2 1 Other Spider Spider Desktop
9 Mozilla/5.0 (iPhone; CPU iPhone OS 12_1_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1 Mobile Safari 12 0 iOS 12 1 2 iPhone Apple iPhone
10 Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36 Chrome 71 0 3578 Windows 8 1 Other
11 Googlebot-Image/1.0 Googlebot-Image 1 0 Other Spider Spider Desktop
12 Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0 Firefox 64 0 Windows 8 1 Other
13 Mozilla/5.0 (Windows NT 5.1; rv:52.0) Gecko/20100101 Firefox/52.0 Firefox 52 0 Windows XP Other
14 Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36 Chrome 49 0 2623 Windows XP Other
15 Mozilla/5.0 (compatible; AhrefsBot/6.1; +http://ahrefs.com/robot/) AhrefsBot 6 1 Other Spider Spider Desktop
16 Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:16.0)Gecko/16.0 Firefox/16.0 Firefox 16 0 Windows 8 Other
17 Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko IE 11 0 Windows 7 Other
18 Mozilla/5.0 (Windows NT 6.3; rv:64.0) Gecko/20100101 Firefox/64.0 Firefox 64 0 Windows 8 1 Other
19 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36 Chrome 71 0 3578 Windows 7 Other

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment