Skip to content

Instantly share code, notes, and snippets.

@vladox
Forked from bubenkoff/download_sentry_data.py
Last active April 13, 2024 01:00
Show Gist options
  • Save vladox/f401db999c5efdf2f25d38683fde2813 to your computer and use it in GitHub Desktop.
Save vladox/f401db999c5efdf2f25d38683fde2813 to your computer and use it in GitHub Desktop.
Download all sentry events for a project. Useful for data processing
"""Download sentry data.
usage:
python download_sentry_data.py <org>/<project> <api_key>
"""
import requests
import csv
import sys
if __name__ == '__main__':
with open('data.csv', 'w', encoding='utf-8') as csvfile:
fieldnames = ['culprit', 'count', 'lastSeen', 'logger']
writer = csv.DictWriter(csvfile, fieldnames=fieldnames, extrasaction='ignore')
writer.writeheader()
url = 'https://app.getsentry.com/api/0/projects/{0}/issues/'.format(sys.argv[1])
while True:
response = requests.get(
url,
headers={'Authorization': 'Bearer {TOKEN}'.format(TOKEN=sys.argv[2])}
)
data = response.json()
for event in data:
# tags = {item['key']: item['value'] for item in event['tags']}
writer.writerow(dict(event))
link = response.headers.get('Link')
print("Last event date: {0}".format(data[-1]['lastSeen']))
if link and '"next"' in link:
print("Getting next page...")
url = link.split()[4][1:-2]
else:
break
@vladox
Copy link
Author

vladox commented Jul 4, 2019

updated to code to new Authorization scheme and to retrieve issues instead of events

@bubenkoff
Copy link

cool, thanks

@viacheslav-gurin
Copy link

Hi! Thank you so much. Though I'm not a developer, your code could be very helpful for my purposes. Trying to use it. And the process of getting data from Sentry seems to be going very well in the beginning, but in the end I get something like this:

Getting next page...
Last event date: 2019-08-20T14:23:16.284Z
Getting next page...
Traceback (most recent call last):
  File "C:\Personal\download_sentry_data.py", line 26, in <module>
    print("Last event date: {0}".format(data[-1]['lastSeen']))
IndexError: list index out of range

Could you please help me to figure out what the problem is? To my shame, I'm not skilled enough for reading and understanding the Python syntax. :(

@bubenkoff
Copy link

probably something is wrong with getting of the data (Authorization)?
add print(data) before that line to check

@vladox
Copy link
Author

vladox commented Sep 19, 2019

Hi! Thank you so much. Though I'm not a developer, your code could be very helpful for my purposes. Trying to use it. And the process of getting data from Sentry seems to be going very well in the beginning, but in the end I get something like this:

Getting next page...
Last event date: 2019-08-20T14:23:16.284Z
Getting next page...
Traceback (most recent call last):
  File "C:\Personal\download_sentry_data.py", line 26, in <module>
    print("Last event date: {0}".format(data[-1]['lastSeen']))
IndexError: list index out of range

Could you please help me to figure out what the problem is? To my shame, I'm not skilled enough for reading and understanding the Python syntax. :(

Hi @viacheslav-gurin How are you calling the script? What do you see in your data.csv file? It's empty?

@viacheslav-gurin
Copy link

@bubenkoff, @vladox, hi!
I'm calling the script via Anaconda prompt:

python C:\Personal\download_sentry_data.py spectrum-data/react [our API key]

where

  • spectrum-data is the actual organization name in Sentry
  • react is the actual project name in Sentry
  • [our API key] is the key I've created in Sentry (API scopes are: event:admin, event:read, member:read, org:read, project:read, project:releases, team:read).

And I don't see data.csv at all.

@viacheslav-gurin
Copy link

Also tried to call the script from my laptop at home like this:

python C:\Users\Lenovo\Desktop\Sentry\download_sentry_data.py spectrum-data/react [our API key]

and received the following:

Traceback (most recent call last):
  File "C:\Users\Lenovo\Desktop\Sentry\download_sentry_data.py", line 24, in <module>
    writer.writerow(dict(event))
ValueError: dictionary update sequence element #0 has length 1; 2 is required

BTW the Python version installed both on my work PC and on my laptop is 3.6.5.

@vvolodko
Copy link

vvolodko commented Dec 11, 2019

It's a pagination issue: the link has results="false".

See https://docs.sentry.io/api/pagination/

@pranith-bm-ai
Copy link

ValueError: dictionary update sequence element #0 has length 1; 2 is required

I added a few print statements, turns out I was getting an authorisation error.

https://sentry.io/settings/account/api/auth-tokens/
My auth token did not have the correct scope.

@siiramone
Copy link

Hi @vladox, thank you so much for useful script.
I improved paging and added date option and more.
https://gist.github.com/siiramone/0d2ed5fe9a78b4f34f70d60d6a9f80b2

@ssjoyo
Copy link

ssjoyo commented Oct 21, 2020

Thank you, this helped.

@juandspy
Copy link

juandspy commented Mar 7, 2022

This snippet doesn't deal with pagination. I found that for large number of events (~15000) it only returns (~1500).

@mezotv
Copy link

mezotv commented Nov 16, 2023

Does not work anymore

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment