Skip to content

Instantly share code, notes, and snippets.

@reginafcompton
Last active January 22, 2018 17:32
Show Gist options
  • Save reginafcompton/2cb4d690c0253a22305929a334753959 to your computer and use it in GitHub Desktop.
Save reginafcompton/2cb4d690c0253a22305929a334753959 to your computer and use it in GitHub Desktop.
Periodically, DataMade needs to manually remove a duplicate or mistakenly added entry in Councilmatic. This gist outlines the process for doing so.

How to remove an entry from Councilmatic

Note: cronjobs can collide with manual data removal from the databases. Please turn off the cron before intervening with the database, or ensure that a cron job is not running at the time of deletion.

Remove the entry from the OCD API

Login to the OCD API server (i.e., ssh ubuntu@ocd.datamade.us), and go to the OCD app: api.opencivicdata.org. Shell into the database via Django (workon ocd-api + python manage.py shell), find the entry, and remove it:

# Example: remove a Bill in the python shell
from opencivicdata.legislative.models import Bill
b = Bill.objects.get(id='ocd-bill/d7d98f30-29ba-45ed-b1ed-d9f658aa62c8')
b.delete()

Remove the entry from the Councilmatic downloads

Login to the Councilmatic server (e.g., ssh ubuntu@lametro.datamade.us), go to the relevant Councilmatic server, and delete the contents of the downloads:

# Example: remove bills from Metro councilmatic downloads
sudo su - datamade
cd lametro
rm -rf downloads/bills/*

Remove the entry from the Councilmatic database

While still in the Metro repository, shell into the database via Django (workon lametro + python manage.py shell), find the entry, and remove it:

# Example: remove a bills from the Metro councilmatic database
from councilmatic_core.models import Bill
b = Bill.objects.get(ocd_id='ocd-bill/d7d98f30-29ba-45ed-b1ed-d9f658aa62c8')
b.delete()

(ONLY FOR BILLS) Remove the entry from Solr

Solr cannot detect the absence of bills once present. If you delete a bill, manually remove it from Solr:

# Example: remove a bill from Metro solr
import pysolr
# You can find the correct URL in the settings_deployment
solr = pysolr.Solr('http://127.0.0.1:8983/solr/lametro')
results = solr.search(q='django_id:ocd-bill/d7d98f30-29ba-45ed-b1ed-d9f658aa62c8')
# Is this the right bill? Check.
[r['friendly_name'] for r in results]
solr.delete(q='django_id:ocd-bill/d7d98f30-29ba-45ed-b1ed-d9f658aa62c8')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment