oguya/installing_onadata.md

## installing_onadata.md

      
    Raw
  

              installing_onadata.md
            
          
    Installing OnaData(Classic ONA) on Ubuntu 16.04

The following are step-by-step instructions for installing Classic ONA(OnaData) on an Ubuntu 16.04 machine.


clone ona repo
  $ cd /opt
  $ git clone https://github.com/onaio/onadata.git
  $ mv onadata onadata-venv
  $ cd onadata-venv


checkout classic ONA commit
  $ git checkout 394f06e483c88cdfc7aab31673310e99742c5c3e
  $ git checkout -b classic_ona


From here the configuration is more or less the steps as described in https://github.com/onaio/onadata/blob/master/install.md with a few modifications


install system dependency packages
  $ sudo apt-get update && apt-get dist-upgrade -y
  $ sudo apt-get install -y wget lsb-release
  $ sudo echo "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list
  $ wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
  $ sudo apt-get update && sudo apt-get install -y postgresql-9.4-postgis-2.3 binutils libproj-dev gdal-bin memcached libmemcached-dev python-pip python-dev python-virtualenv git libssl-dev libpq-dev gfortran libatlas-base-dev libjpeg-dev libxml2-dev libxslt1-dev zlib1g-dev python-software-properties ghostscript openjdk-8-jre libffi-dev libssl-dev rabbitmq-server uwsgi


NOTE
Not sure about these(they are provided in Ona's documentation):
  sudo ln -s /usr/lib/`uname -i`-linux-gnu/libjpeg.so ~/virtualenv/python2.7/lib/
  sudo ln -s /usr/lib/`uname -i`-linux-gnu/libz.so ~/virtualenv/python2.7/lib/


PostgreSQL Database Setup


switch to postgres user
  $ sudo su - postgres
  $ psql


create PostgreSQL account & db
  postgres=# create user onadata with password 'onadata';
  postgres=# create database onadata owner onadata;


create postgis extension in the db created above
  postgres=# \c onadata 
  onadata=# create extension if not exists postgis;
  onadata=# create extension if not exists postgis_topology;


test credentials by connecting to the db
  $ PGPASSWORD=onadata psql -U onadata -d onadata -h localhost


Setup Python Virtual Environment


create virtual env.
  $ cd /opt/onadata-venv
  $ virtualenv .
  $ source ./bin/activate


create onadata/settings/local_settings.py containing postgresql db credentials created above
  $ cp onadata/settings/default_settings.py onadata/settings/local_settings.py


onadata/settings/local_settings.py content
  DATABASES = {
      'default': {
          'ENGINE': 'django.contrib.gis.db.backends.postgis',
          'NAME': 'onadata',
          'USER': 'onadata',
          'PASSWORD': 'onadata',
          'HOST': '127.0.0.1'
      }
  }


add --run-syncdb to python manage.py command in the Makefile
  $ sed -i 's/migrate$/migrate --run-syncdb/' Makefile


install python dependency packages and run make
  $ pip install -r requirements/base.pip
  $ make


NOTE
At this point you can start the core with:
  `bin/python manage.py runserver --nothreading`

OR continue with the setup

Compile API docs


compile the API documentation
  $ cd docs
  $ make html
  $ cd ..


copy static html files to static dir
  $ bin/python manage.py collectstatic --noinput


User Administration


create a super user that will be used for administration from the web UI
  $ bin/python manage.py createsuperuser


Setup UWSGI


install uwsgi system package system-wide
  $ sudo apt install uwsgi


configure the uwsgi process for onadata to run as www-data user
  $ sed -i 's/^uid=.*/uid=www-data/;s/^gid=.*/gid=www-data/' /opt/onadata-venv/uwsgi.ini


create a new uwsgi config file; /opt/onadata-venv/uwsgi.ini
  [uwsgi]
  http-socket=:3030
  socket=/tmp/ona.sock
  chmod-socket=777
  uid=www-data
  gid=www-data
  chdir=/opt/onadata-venv
  module=onadata.apps.main.wsgi:application
  master=True
  processes=12
  pidfile=/var/run/ona/ona.pid
  vacuum=True                 # clear environment on exit
  harakiri=120                 # respawn processes taking more than 2 minute
  max-requests=5000           # respawn processes after serving 5000 requests
  logto=/var/log/uwsgi/onadata.log
  virtualenv=/opt/onadata-venv
  static-map=/static=/opt/onadata-venv/onadata/static
  buffer-size=8192
  env=HTTPS=on
  stats=/tmp/onastats.sock
  plugins=python


create log dir. with necessary permissions
  $ sudo mkdir /var/log/uwsgi/
  $ sudo chown -R www-data:root /var/log/uwsgi/


create an upstart file for uwsgi to run onadata; /etc/init/onadata.conf
  #!upstart
  description "onadata upstart script"

  start on (local-filesystems and runlevel [2345])
  stop on runlevel [!2345]

  respawn

  script
      exec sudo /usr/local/bin/uwsgi --ini /opt/onadata-venv/uwsgi.ini --env DJANGO_SETTINGS_MODULE=onadata.settings.common
  end script

  pre-start script
      echo "[`date -u +%Y-%m-%dT%T.%3NZ`] Starting" >> /var/log/onadata_upstart.log
  end script

  pre-stop script
      echo "[`date -u +%Y-%m-%dT%T.%3NZ`] Stopping" >> /var/log/onadata_upstart.log
  end script


OR create a systemd service file for onadata; /etc/systemd/system/onadata.service
  [Unit]
  Description=OnaData server
  After=network.target auditd.service
  ConditionPathExists=/opt/onadata-venv/uwsgi.ini

  [Service]
  ExecStart=/usr/local/bin/uwsgi --ini /opt/onadata-venv/uwsgi.ini --env DJANGO_SETTINGS_MODULE=onadata.settings.common
  PIDFile=/var/run/ona/ona.pid
  RuntimeDirectory=ona
  Restart=on-failure
  Type=forking
  User=www-data
  Group=www-data

  [Install]
  WantedBy=multi-user.target


start onadata systemd service
  # systemctl daemon-reload
  # systemctl enable onadata
  # systemctl start onadata


Setup Celery


add celeryd defaults to /etc/default/rabbitmq-server & /etc/rabbitmq/rabbitmq-env.conf
  # Name of nodes to start, here we have a single node
  CELERYD_NODES="w1"

  # Where to chdir at start.
  CELERYD_CHDIR="/opt/onadata-venv"

  # Python interpreter from environment, if using virtualenv
  ENV_PYTHON="/opt/onadata-venv/bin/python"

  # How to call "manage.py celeryd_multi"
  CELERYD_MULTI="$ENV_PYTHON $CELERYD_CHDIR/manage.py celeryd_multi"

  # How to call "manage.py celeryctl"
  CELERYCTL="$ENV_PYTHON $CELERYD_CHDIR/manage.py celeryctl"

  # Extra arguments to celeryd
  #CELERYD_OPTS="--time-limit=300 --concurrency=1"

  # Name of the celery config module, don't change this.
  CELERY_CONFIG_MODULE="celeryconfig"

  # %n will be replaced with the nodename.
  CELERYD_LOG_FILE="/var/log/celery-venv/%n.log"
  CELERYD_PID_FILE="/var/run/celery-venv/%n.pid"

  # Workers should run as an unprivileged user.
  CELERYD_USER="www-data"
  CELERYD_GROUP="www-data"

  SECRET_KEY="uklsdfd%6olsd82jadfnsew"

  # Set any other env vars here too!
  PROJET_ENV="PRODUCTION"

  # Name of the projects settings module.
  # in this case is just settings and not the full path because it will change the dir to
  # the project folder first.
  export DJANGO_SETTINGS_MODULE="onadata.settings.common"


start & enable rabbitmq-server
  $ systemctl enable rabbitmq-server.service
  $ systemctl start rabbitmq-server.service


Setup NGINX


install nginx
  $ sudo install nginx


create a vhost config file for onadata; /etc/nginx/sites-available/onadata
  upstream ona-venv {
      server 127.0.0.1:3030;
  }

  server {
      listen 80 default_server;
      listen [::]:80 default_server;

      client_max_body_size 50m;

      access_log /var/log/nginx/onadata-venv.access.log;
      error_log /var/log/nginx/onadata-venv.error.log;

      root /opt/onadata-venv/onadata;
      location /static {
          alias /opt/onadata-venv/onadata/static;

          # if asset versioning is used
          if ($query_string) {
              expires max;
          }
      }

      # Proxying the connections connections
      location / {
          uwsgi_pass         ona-env;
          include            uwsgi_params;
          proxy_redirect     off;
          proxy_set_header   Host $host;
          proxy_set_header   X-Real-IP $remote_addr;
          proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
          proxy_set_header   X-Forwarded-Host $server_name;
          proxy_connect_timeout 60s;
          proxy_read_timeout 300s;

          proxy_pass http://ona-env;
      }
  }


enable the vhost & remove the de
  $ ln -s /etc/nginx/sites-available/onadata /etc/nginx/sites-enabled/onadata


test nginx config to confirm everything is OK
  $ sudo service nginx configtest


restart nginx
  $ sudo systemctl restart nginx


test connection using curl
  $ curl -Iv http://localhost/


Finally...

If all the above steps completed without any issues/errors, then the installation is complete! Access the web console using the URL http://your-server-ip/ or http://localhost/ if you installed onadata on your local PC.
Extra configurations

To further fine-tune onadata, you can make changes in /opt/onadata-venv/onadata/settings/common.py and then restart the uwsgi process running ona by executing the command systemctl reload onadata