Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Socorro Lite

Socorro (Lite)

  1. Create a place to hold crashes
mkdir crashreports
cd crashreports
  1. Build stackwalker
git clone https://github.com/rhelmer/minidump-stackwalk
cd minidump-stackwalk
make
cd ../
  1. Install Socorro
mkvirtualenv crashreports
pip install https://people.mozilla.org/~rhelmer/socorro-master.tar.gz
  1. Run the Collector and Processor:
socorro collector &
socorro processor &
  1. Now try submitting a test crash:
curl 'https://raw.githubusercontent.com/mozilla/socorro/master/testcrash/raw/7d381dc5-51e2-4887-956b-1ae9c2130109.dump' > testcrash.dump 
curl -F ProductName=TestApp \
  -F Version=1.0 \
  -F upload_file_minidump=@testcrash.dump \
  http://127.0.0.1:8882/submit
  1. You should see a CrashID returned:
CrashID=bp-d1ffc5df-f26f-40d3-aa11-6d4ea2150113

Both the raw (.json, .dmp) and processed (.jsonz) files will be available under ./crashes:

$ find ./crashes -name "d1ffc5df-f26f-40d3-aa11-6d4ea2150113.*"
./crashes/20150113/name/d1/ff/d1ffc5df-f26f-40d3-aa11-6d4ea2150113.dump
./crashes/20150113/name/d1/ff/d1ffc5df-f26f-40d3-aa11-6d4ea2150113.json
./crashes/20150113/name/d1/ff/d1ffc5df-f26f-40d3-aa11-6d4ea2150113.jsonz

Production Crash Collection

For production use, run the collector app under uwsgi:

uwsgi -H ~/.virtualenvs/crashreports -w socorro.wsgi.collector -s my_socket

Then configure your webserver to pass connections to that socket.

Nginx: http://uwsgi-docs.readthedocs.org/en/latest/Nginx.html

Apache: http://uwsgi-docs.readthedocs.org/en/latest/Apache.html

Storing crashes in ElasticSearch

Fetch the Socorro index mappings:

  curl 'https://raw.githubusercontent.com/mozilla/socorro/master/socorro/external/elasticsearch/mappings/socorro_index_settings.json' > socorro_index_settings.json

Stop the socorro processor, and run it with the following options:

socorro processor \
  --destination.crashstorage_class='
    socorro.external.crashstorage_base.PolyCrashStorage' \
  --destination.storage_classes='
    socorro.external.fs.crashstorage.FSLegacyDatedRadixTreeStorage,
    socorro.external.elasticsearch.crashstorage.ElasticSearchCrashStorage' \
  --destination.storage1.elasticsearch_base_settings=./socorro_index_settings.json

This configures Socorro to store processed crashes using the PolyCrashStorage for the destination crashstorage_class, which stores crashes in multiple places. The destination storage_classes are set to save crashes both to the filesystem (FSLegacyDatedRadixTreeStorage) and also in ElasticSearch (ElasticSearchCrashStorage).

The ElasicSearch index is expected to be in ./socorro_index_settings.json.

The processor will by default use an ElasticSearch instance at ```localhost:9200``, this can be changed with:

  --destination.storage1.elasticsearch_urls='http://...:9200'

Try passing --help along with options to see all of these defaults.

Using Kibana to view crashes in ElasticSearch

Kibana is a web-based tool for exploring data in ElasticSearch and making dashboards: http://www.elasticsearch.org/overview/kibana/installation/

You can find crashes in the socorro index.

Storing crashes in Amazon S3

Stop the Socorro processor, and run it with the following options:

socorro processor \
  --destination.crashstorage_class='
    socorro.external.crashstorage_base.PolyCrashStorage' \
  --destination.storage_classes='
    socorro.external.fs.crashstorage.FSLegacyDatedRadixTreeStorage,
    socorro.external.boto.crashstorage.BotoS3CrashStorage' \
  --destination.storage1.access_key=YOUR_ACCESS_KEY \
  --destination.storage1.secret_access_key=YOUR_SECRET_ACCESS_KEY \
  --destination.storage1.bucket_name=YOUR_BUCKET_NAME

This configures Socorro to store processed crashes using the PolyCrashStorage for the destination crashstorage_class, which stores crashes in multiple places. The destination storage_classes are set to save crashes both to the filesystem (FSLegacyDatedRadixTreeStorage) and also in Amazon S3 (BotoS3CrashStorage).

Make sure to set YOUR_ACCESS_KEY, YOUR_SECRET_ACCESS_KEY and YOUR_BUCKET_NAME.

Storing crashes in PostgreSQL

Create the default breakpad database for Socorro to use:

  # FIXME need better way to get raw_sql
  mkdir -p socorro/external/postgresql/raw_sql/
  cp -rp ~/src/socorro/socorro/external/postgresql/raw_sql/* \
         socorro/external/postgresql/raw_sql/
  cp -rp ~/src/socorro/alembic .
  rm alembic/versions/*.py
  
  socorro setupdb --database_name=breakpad

Create PostgreSQL partitions (this must be run on a weekly basis):

  # FIXME doesn't actually work yet
  # socorro crontabber --job=weekly-reports-partitions --force
python ~/.virtualenvs/crashreports/lib/python2.7/site-packages/socorro/cron/crontabber_app.py \
  --job=weekly-reports-partitions --force

Stop the processor and run with the following options:

socorro processor \
  --destination.crashstorage_class='
    socorro.external.crashstorage_base.PolyCrashStorage' \
  --destination.storage_classes='
    socorro.external.fs.crashstorage.FSLegacyDatedRadixTreeStorage,       
    socorro.external.postgresql.crashstorage.PostgreSQLCrashStorage'

This uses PolyCrashStorage to store crashes in both the filesystem (FSLegacyDatedRadixTreeStorage) and PostgreSQL (PostgreSQLCrashStorage).

Using the Crash-Stats UI to view crashes

If you wish to have a UI like https://crash-stats.mozilla.com, you can use the same Django-based project that Mozilla does.

https://gist.github.com/rhelmer/ce2354e7c11eb2848c2e

@peterbe

This comment has been minimized.

Copy link

peterbe commented Jan 14, 2015

It's missing the webapp.

@rhelmer

This comment has been minimized.

Copy link
Owner Author

rhelmer commented Jan 17, 2015

@peterbe Yes it is :) - the idea is to make a "minimum viable crashserver", so things like ES, S3, Postgres, webapp, etc. are optional that you can turn on later. Our webapp currently depends on Postgres, ES, so I'll show how to enable that later after those are set up.

By making the webapp optional, I am thinking that someone could e.g. just turn on ES support in processor, and then start using Kibana and never have to use our webapp if they don't want to. Or enable S3 and use one of the many tools for analyzing data stored there.

Our webapp is a bit complex (and wedded to our release process in deep ways), but the core collector+processor of Socorro are not, and it'd be pretty easy to do custom reporting on top of HBase or S3 or ES and still get a lot of value out of (Socorro - webapp)

@rhelmer

This comment has been minimized.

Copy link
Owner Author

rhelmer commented Jan 18, 2015

@peterbe there's a link for setting up the webapp now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.