edX Analytics Combined Devstack.md

As of: 2015-08-19

This is how to set up a combined devstack/insights virtualbox.

Run these commands on your host computer, in a new "combined-devstack" folder:

mkdir combined-devstack
cd combined-devstack
curl -L https://raw.githubusercontent.com/edx/configuration/master/vagrant/release/analyticstack/Vagrantfile > Vagrantfile
CONFIGURATION_VERSION=origin/master EDX_PLATFORM_VERSION=master vagrant up
vagrant ssh
cd /edx/app/edx_ansible/edx_ansible/
sudo su
git checkout master
git pull
source /edx/app/edx_ansible/venvs/edx_ansible/bin/activate
cd /edx/app/edx_ansible/edx_ansible/playbooks/edx-east
ansible-playbook -i localhost, -c local analytics_single.yml -e configuration_version=master
usermod -s /bin/bash analytics_api
usermod -s /bin/bash insights

If you get "An error occurred while downloading the remote file." then edit the Vagrantfile and where it says config.vm.box = "analyticstack-beta", add a line immediately after that says config.vm.box_url = "http://files.edx.org/vagrant-images/cypress-rc4-devstack.box".

Meanwhile, in a seprate terminal window on your host, you can set up the pipeline:

cd analytics-devstack
git clone https://github.com/edx/edx-analytics-pipeline
cd edx-analytics-pipeline/
virtualenv venv
source venv/bin/activate
make bootstrap
# There are currently a couple of bugs - the following may be required to fix:
pip uninstall edx.analytics.tasks
python setup.py develop

Once both of these sets of setup commands have completed, follow the usage instructions at https://github.com/mulby/edx-documentation/blob/gabe/analytics-devstack/en_us/install_operations/source/devstack/analytics_devstack.rst#using-the-analytics-devstack

To get Insights login working:

1. Go to http://localhost:8000/admin/oauth2/client/add/
2. Name: Insights Dashboard
3. URL: http://127.0.0.1:8110/
4. Redirect URL: http://127.0.0.1:8110/complete/edx-oidc/
5. Client type: Confidential
6. Save
7. vagrant ssh
8. sudo nano /edx/etc/insights.yml
9. Set SOCIAL_AUTH_EDX_OIDC_KEY to the "Client ID" value
10. Set SOCIAL_AUTH_EDX_OIDC_ID_TOKEN_DECRYPTION_KEY and SOCIAL_AUTH_EDX_OIDC_SECRET to the "Client Secret" value.
11. Save

Unfortunately, things seem to have shifted again, as these instructions don't work anymore. In addition to the issue with ansible 1.4.5 not working inside a virtualenv for me, I had to make the following modifications to the instructions:

1. I had to use a completely different playbook, namely configuration/playbooks/vagrant-analytics.yml. The one mentioned here doesn't install edx-platform.
2. Passing CONFIGURATION_VERSION and EDX_PLATFORM_VERSION in the environment when running vagrant up didn't have any effect for me. I had to pass them directly to ansible when running the playbook: -e configuration_version=origin/master -e edx_platform_version=origin/master
3. Some of the tasks in the playbook failed:
   • Compressing the Insights assets fails with a weird Java traceback. When rerunning the playbook, this succeeds for some reason.
   • Creating an OAuth2 client fails with the error message that the management command create_oauth2_client does not exist. I simply removed this task in a local branch and passed the name of that branch as configuration_version to ansible.

Thanks for that info, @smarnach! I will update these again once a box image is published which should make this all much easier.