Legacy pysec
What I see as the biggest problem of pysec is that is not really a completed project. Its XBRL API is probably fine, but the rest related to the Django app is not more than a proof of concept. The following gives instruction on how to use the legacy pysec.
Install django
. The following would install the packaged version, which may be somewhat obsolette:
apt-get install python-django
Clone the pysec repo:
git clone https://github.com/lukerosiak/pysec.git
cd pysec
Update the settings:
# Create the local settings file from the example one.
cp local_settings-example.py local_settings.py
# Chnage DB backend to sqlite (sqlite is very easy for DB querying outside of the app)
sed -i "s/^\(\s*'ENGINE'\s*:\).*/\1 'django.db.backends.sqlite3',/" local_settings.py
sed -i "s/^\(\s*'NAME'\s*:\).*/\1 os.path.join(PROJECT_ROOT, 'db.sqlite3'),/" local_settings.py
# Bash function to generate a SECRET_KEY. Comes from https://gist.github.com/ndarville/3452907
function mk_dj_secret() { python -c "import random,string;print 'SECRET_KEY=\"%s\"'%''.join([random.SystemRandom().choice(\"{}{}{}\".format(string.ascii_letters, string.digits, string.punctuation)) for i in range(63)])" ;}
# Update the secret key
sed -i '/^SECRET_KEY/d' local_settings.py
mk_dj_secret >> local_settings.py
# Create data directory (the one defined in the local_settings.py)
mkdir pysec/data
Start up:
# Create the DB
python manage.py syncdb
# Populate DB with some sample data
python manage.py sec_import_index
Create a sample application (modified from the pysec/example.py
and put under pysec/management/commands/
):
from pysec.models import *
from django.core.management.base import NoArgsCommand
from django.conf import settings
class Command(NoArgsCommand):
help = "..."
def handle_noargs(self, **options):
#"""get a file from the index. it may or may not be present on our hard disk. if it's not, it will be downloaded
#the first time we try to access it, or you can call .download() explicitly"""
filing = Index.objects.filter(form='10-Q',cik=1090872).order_by('-date')[0]
print filing.name
#"""initialize XBRL parser and populate an attribute called fields with a dict of 50 common terms"""
x = filing.xbrl()
print x.fields['FiscalYear']
print x.fields
#"""fetch arbitrary XBRL tags representing eiter an Instant or a Duration in time"""
print 'Tax rate', x.GetFactValue('us-gaap:EffectiveIncomeTaxRateContinuingOperations','Duration')
if x.loadYear(1):
# """Most 10-Ks have two or three previous years contained in them for the major values. This call switches the contexts
# to the prior year (set it to 2 or 3 instead of 1 to go back further) and reloads the fundamental concepts.
# Any calls to GetFactValue will use that year's value from that point on."""
print x.fields['FiscalYear']
print x.fields
print 'Tax rate', x.GetFactValue('us-gaap:EffectiveIncomeTaxRateContinuingOperations','Duration')
Run the above example command:
python manage.py example
The original project has been forked many times and some of the forks offer more complete apps. A notable example is django-sec.
Create a sample Django project:
# Creating the 'mysite' project (may test with `python manage.py runserver`)
django-admin startproject mysite
cd mysite
# Edit mysite/settings.py and add 'django-sec' into 'INSTALLED_APPS'
...
# Create the project's DB (use 'migrate' or 'syncdb' based on your Django version
# Create a DB user when asked to!
python manage.py migrate django_sec
python manage.py syncdb
# Start the server
python manage.py runserver
Administer the application from another terminal:
# Note that <year1> has to be less than <year2>
# Results of processed indexes can be observed at http://localhost:8000/admin/django_sec
# (use the user/password used when creating the project's DB)
python manage.py sec_import_index --start-year=<year1> --end-year=<year2>
... to be continued
any way you can continue this tutorial and explain how to extract data from 10-q filings?