brabect1/working_with_pysec.md

## working_with_pysec.md

      
    Raw
  

              working_with_pysec.md
            
          
    Legacy pysec

What I see as the biggest problem of pysec is that is not really a completed project. Its XBRL API is probably fine, but the rest related to the Django app is not more than a proof of concept. The following gives instruction on how to use the legacy pysec.
Install django. The following would install the packaged version, which may be somewhat obsolette:
apt-get install python-django

Clone the pysec repo:
git clone https://github.com/lukerosiak/pysec.git
cd pysec

Update the settings:
# Create the local settings file from the example one.
cp local_settings-example.py local_settings.py

# Chnage DB backend to sqlite (sqlite is very easy for DB querying outside of the app)
sed -i "s/^\(\s*'ENGINE'\s*:\).*/\1 'django.db.backends.sqlite3',/" local_settings.py
sed -i "s/^\(\s*'NAME'\s*:\).*/\1 os.path.join(PROJECT_ROOT, 'db.sqlite3'),/" local_settings.py

# Bash function to generate a SECRET_KEY. Comes from https://gist.github.com/ndarville/3452907
function mk_dj_secret() { python -c "import random,string;print 'SECRET_KEY=\"%s\"'%''.join([random.SystemRandom().choice(\"{}{}{}\".format(string.ascii_letters, string.digits, string.punctuation)) for i in range(63)])" ;}

# Update the secret key
sed -i '/^SECRET_KEY/d' local_settings.py
mk_dj_secret >> local_settings.py

# Create data directory (the one defined in the local_settings.py)
mkdir pysec/data

Start up:
# Create the DB
python manage.py syncdb

# Populate DB with some sample data
python manage.py sec_import_index


Create a sample application (modified from the pysec/example.py and put under pysec/management/commands/):
from pysec.models import *
from django.core.management.base import NoArgsCommand
from django.conf import settings

class Command(NoArgsCommand):
    help = "..."
    
    
    def handle_noargs(self, **options):
        #"""get a file from the index. it may or may not be present on our hard disk. if it's not, it will be downloaded
        #the first time we try to access it, or you can call .download() explicitly"""
        filing = Index.objects.filter(form='10-Q',cik=1090872).order_by('-date')[0]
        print filing.name
        
        #"""initialize XBRL parser and populate an attribute called fields with a dict of 50 common terms"""
        x = filing.xbrl()

        print x.fields['FiscalYear']

        print x.fields

        #"""fetch arbitrary XBRL tags representing eiter an Instant or a Duration in time"""
        print 'Tax rate', x.GetFactValue('us-gaap:EffectiveIncomeTaxRateContinuingOperations','Duration')

        if x.loadYear(1): 
        #    """Most 10-Ks have two or three previous years contained in them for the major values. This call switches the contexts
        #    to the prior year (set it to 2 or 3 instead of 1 to go back further) and reloads the fundamental concepts.
        #    Any calls to GetFactValue will use that year's value from that point on."""
                    
            print x.fields['FiscalYear']

            print x.fields

            print 'Tax rate', x.GetFactValue('us-gaap:EffectiveIncomeTaxRateContinuingOperations','Duration')
Run the above example command:
python manage.py example

Improved Versions

The original project has been forked many times and some of the forks offer more complete apps. A notable example is django-sec.
django-sec

Create a sample Django project:
# Creating the 'mysite' project (may test with `python manage.py runserver`)
django-admin startproject mysite
cd mysite

# Edit mysite/settings.py and add 'django-sec' into 'INSTALLED_APPS'
...

# Create the project's DB (use 'migrate' or 'syncdb' based on your Django version
# Create a DB user when asked to!
python manage.py migrate django_sec
python manage.py syncdb

# Start the server
python manage.py runserver

Administer the application from another terminal:
# Note that <year1> has to be less than <year2>
# Results of processed indexes can be observed at http://localhost:8000/admin/django_sec
# (use the user/password used when creating the project's DB)
python manage.py sec_import_index --start-year=<year1> --end-year=<year2>

... to be continued