greglinch/datawrapper-install-instructions-detailed.md

## datawrapper-install-instructions-detailed.md

      
    Raw
  

              datawrapper-install-instructions-detailed.md
            
          
    Datawrapper self-hosted setup instructions

Thanks to the following resources:

https://github.com/datawrapper/datawrapper/wiki/Installing-Datawrapper
https://github.com/washingtonpost/datawrapper#installation-instructions
https://www.digitalocean.com/community/tutorials/how-to-install-linux-apache-mysql-php-lamp-stack-on-ubuntu-14-04
https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-an-ubuntu-14-04-server

Create your environment

These instructions are based on setting up your own installation of Datawrapper using Amazon Web Services (AWS).
You will need:

EC2 instance running Ubuntu 14
S3 bucket

On the EC2 instance, you will need the following ports open:

22 (ssh)
80 (http)
443 (https) (optional)

You'll also need to have a dedicated domain or subdomain for Datawrapper, so be sure to update your DNS records and confirm they've propagated.
General notes

Any time you see angle brackets <value>, replace that with the correct value for your set up.
You might also see also <subdomain>.<domain>.com, but you can disregard the <subdomain> part if you're setting up Datawrapper on its own dedicated domain.
Prepare your server

Log in via ssh
ssh <username>@<domain>.com

Install Apache
sudo apt-get install apache2

Install MySQL
sudo apt-get install mysql-client-core-5.5
sudo apt-get install mysql-server php5-mysql

Set your username and password
Next setps for MySQL
sudo mysql_install_db
sudo mysql_secure_installation

Answer the ensuing prompt however you prefer. For example:
y (remove anon users)

n (disbale remote users)

y (remove test db and access)

y (reload privelege tables)

Install PHP
sudo apt-get install php5 libapache2-mod-php5 php5-mcrypt

Install Git
sudo apt-get install git

Initial application setup

Clone the repo
git clone https://github.com/datawrapper/datawrapper.git 

Change into that directory
cd ~/datawrapper

Install Composer
curl -sS https://getcomposer.org/installer | php

Install project dependencies
php composer.phar install

Install Node and npm
sudo apt-get install nodejs
sudo apt-get install nodejs-legacy
sudo apt-get install npm

Node legacy required because of a later error that occurs when running Grunt
https://github.com/nodejs/node-v0.x-archive/issues/3911#issuecomment-18951288

Install Grunt
sudo npm install -g grunt-cli

Install Node dependencies
sudo npm install

Run Grunt
sudo grunt

Permissions

Set the following permissions
sudo chown -R :www-data /home/ubuntu/datawrapper/charts/
sudo chown -R :www-data /home/ubuntu/datawrapper/tmp/
sudo chown -R :www-data /home/ubuntu/datawrapper/www/static/

sudo chmod -R g+w /home/ubuntu/datawrapper/charts/    
sudo chmod -R g+w /home/ubuntu/datawrapper/tmp/
sudo chmod -R 755 /home/ubuntu/datawrapper/www/static/


Apache configuration

Copy the example config
sudo cp /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/datawrapper.conf

Edit the new config
sudo vim /etc/apache2/sites-available/datawrapper.conf

Add/update these lines
ServerAdmin <your email>
ServerName <subdomain>.<domain>.com
ServerAlias <subdomain>.<domain>.com
DocumentRoot /home/ubuntu/datawrapper/www

Save and close
:wq

Copy that config file to another one for charts
sudo cp /etc/apache2/sites-available/datawrapper.conf /etc/apache2/sites-available/datawrapperchart.conf

Edit
sudo vim /etc/apache2/sites-available/datawrapperchart.conf

Update these lines
ServerName charts.<subdomain>.<domain>.com
ServerAlias charts.<subdomain>.<domain>com
DocumentRoot /home/ubuntu/datawrapper/charts/static

Save and close
Change into the sites-available dir
cd /etc/apache2/sites-available/

Enable the sites (a.k.a. symlink sites-available to sites-enabled)
sudo a2ensite datawrapper.conf
sudo a2ensite datawrapperchart.conf

Then reload the Apache server
service apache2 reload

Also, be sure to remove the default symlink
sudo rm /etc/apache2/sites-enabled/000-default.conf

Create/update yaml config

Change back to Datawrapper application directory
cd ~/datawrapper

Create a new yaml config file
cp config.template.yaml config.yaml

Edit that
vim config.yaml

Update/uncomment and update the following items:
domain: <subdomain>.<domain>.com
chart_domain: charts.<subdomain>.<domain>.com

auth_salt: <auth_salt>
secure_auth_key: <auth_secure_key>

email:
  # users with problems will be shown the support email
  support: <support email>
  # error messages will go here
  log: <debgug email>
  # sender
  sender: <your email or noreply@domain.com>

Save and close
MySQL setup

Log into MySQL and enter password when prompted
mysql -u <user> -p

Create a new database
CREATE DATABASE datawrapper;

Confirm it was created
SHOW DATABASES;

Close the MySQL console
Control + C

Initialize the database schema
mysql -u <user> -p datawrapper < lib/core/build/sql/schema.sql

Confirm it worked by logging in and entering your password when prompted; you can also use this command in the future to directly access the Datawrapper database
mysql -u <user> -p datawrapper

Then enter
SHOW TABLES;

And you should see
+-----------------------+
| Tables_in_datawrapper |
+-----------------------+
| action                |
| chart                 |
| job                   |
| organization          |
| organization_product  |
| plugin                |
| plugin_data           |
| plugin_organization   |
| product               |
| product_plugin        |
| session               |
| stats                 |
| user                  |
| user_organization     |
| user_product          |
+-----------------------+

Close MySQL console
Control + C

Install plugins

sudo php scripts/plugin.php install "*"

Datawrapper database config

Copy DataWrapper conf to new file
cp lib/core/build/conf/datawrapper-conf.php.master lib/core/build/conf/datawrapper-conf.php

Open/edit the file and update your database settings (dbname, user, password) according to your server configuration
vim lib/core/build/conf/datawrapper-conf.php

Save and close
Apache main config

Open the Apache config
sudo vim /etc/apache2/apache2.conf

Comment out everything except  and add this
<Directory /home/ubuntu/datawrapper/www/>
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>

Updated  to
Require all granted

Save and close
Enable mod_rewrite by running this command
sudo a2enmod rewrite

Create an empty rewrite config file
sudo touch /etc/apache2/mods-enabled/rewrite.conf

Add these lines to ~/datawrapper/www/.htaccess
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^vendor/(.*)$ /static/vendor/$1 [R=301,L]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^ index.php [QSA,L]

Restart Apache
sudo service apache2 restart

Fixes

Missing minified js files
cp ~/datawrapper/www/static/vendor/queue-async/queue.js ~/datawrapper/www/static/vendor/queue-async/queue.min.js

cp ~/datawrapper/dw.js/dw-2.0.js ~/datawrapper/dw.js/dw-2.0.min.js

cp ~/datawrapper/www/static/vendor/json-js/json2.js ~/datawrapper/www/static/vendor/json-js/json2.min.js

Try re-installing plugins and re-running grunt from dw.js dir, then restart server
Downloading additional plugins

Change into the plugin directory
cd ~/datawrapper/plugins

If you download a plugin from the Datawrapper Bitbucket repo, the easiest way is to include a new name without the plugin- prefix as part of the command
git clone <url> <plugin-name-sans-prefix> 

For example:
git clone https://bitbucket.org/jpp-cologne/plugin-publish-s3.git publish-s3

Install thses plugins
color-category-selector
color-gradient-selector
custom-tooltip
email-native
export-image
export-print-pdf
export-static-png
export-zip
phantomjs
publish-s3
snapshot

Change back to the main Datawrapper directory
cd ~/datawrapper

The plugin installation command is
php scripts/plugin.php install <plugin-name>

PhantomJS

Install phantomjs
sudo apt-get install phantomjs

Update config.yaml line for Phantomjs
phantomjs:
    path: /usr/bin/phantomjs

Publishing to S3

For publish-s3 to work properly, install curl for PHP
sudo apt-get install php5-curl

Add these settings, making sure indentation is correct (e.g. two spaces preceding the first line); also, view endpoint options
publish-s3:
  accesskey: <access_key>
  secretkey: <secret_key>
  bucket: <bucket name>
  endpoint: s3-<region>.amazonaws.com

SSL / HTTPS config

The Electronic Frontier Foundations's Certbot is a quick and easy way to get your server configured with SSL.
View Apache server errors

Enter this commend to view errors in your terminal as they occur
tail -f /var/log/apache2/error.log

You can also set up a shortcut for this
sudo vim ~/.bashrc

Edit add this at the bottom
alias errorapache="tail -f /var/log/apache2/error.log"

Similarly, you can also add a shortcut for view the database
alias dwdb="mysql -u <username> -p datawrapper"

Save and close
Execute your updated bashrc file for the changes to take effect
source ~/.bashrc

Help! Remaining issues

I still have several problems with various plugins and bits of functionality. Any advice would be greatly appreciated!

If you set up config.yaml with an asset domain on S3 when using publish-s3, you need to manually publish the static directory there. But there's a bug in the application code so that, when republishing the chart, the new paths are not matching the expected locations of the static files. You could try to remedy this by hand, but I'd like to avoid that.
embed-static-png: I get the following error when I run ./home/ubuntu/datawrapper/scripts/execute_jobs restart in an attempt to fixed the queued jobs that are piling up:

PHP Notice: Undefined index: path in /home/ubuntu/datawrapper/plugins/phantomjs/plugin.php on line 17
/home/ubuntu/datawrapper/plugins/export-static-png/gen_static_fallback.js: 1: /home/ubuntu/datawrapper/plugins/export-static-png/gen_static_fallback.js: Syntax error: "(" unexpected
PHP Fatal error: Call to undefined function dw_send_error_mail() in /home/ubuntu/datawrapper/plugins/export-static-png/plugin.php on line 65

Fatal error: Call to undefined function dw_send_error_mail() in /home/ubuntu/datawrapper/plugins/export-static-png/plugin.php on line 65


admin-organizations: This plugin isn't publicly available. I'd love if it would be published or if anyone is intersted in collaborating to make an open-source version.


export-zip: It's looking in the wrong directory for at least one file:


Message: file_get_contents(/home/ubuntu/datawrapper/charts/static/vendor/json-js/json2.min.js): failed to open stream: No such file or directory
File: /home/ubuntu/datawrapper/plugins/export-zip/Zipper.php
Line: 62
Trace

#0 [internal function]: Slim::handleErrors(2, 'file_get_conten...', '/home/ubuntu/da...', 62, Array)
#1 /home/ubuntu/datawrapper/plugins/export-zip/Zipper.php(62): file_get_contents('/home/ubuntu/da...')
#2 /home/ubuntu/datawrapper/plugins/export-zip/plugin.php(37): DatawrapperPlugin_ExportZip_Zipper->createZip(Object(User), Object(Chart))
#3 [internal function]: DatawrapperPlugin_ExportZip->{closure}(Object(User), Object(Chart))
#4 /home/ubuntu/datawrapper/lib/utils/check_chart.php(8): call_user_func(Object(Closure), Object(User), Object(Chart))
#5 /home/ubuntu/datawrapper/plugins/export-zip/plugin.php(47): check_chart_readable('iRrRs', Object(Closure))
#6 [internal function]: DatawrapperPlugin_ExportZip->{closure}('iRrRs')
#7 /home/ubuntu/datawrapper/vendor/slim/slim/Slim/Router.php(187): call_user_func_array(Object(Closure), Array)
#8 /home/ubuntu/datawrapper/vendor/slim/slim/Slim/Slim.php(1146): Slim_Router->dispatch(Object(Slim_Route))
#9 /home/ubuntu/datawrapper/vendor/slim/slim/Slim/Middleware/Flash.php(84): Slim->call()
#10 /home/ubuntu/datawrapper/vendor/slim/slim/Slim/Middleware/MethodOverride.php(91): Slim_Middleware_Flash->call()
#11 /home/ubuntu/datawrapper/vendor/slim/slim/Slim/Middleware/PrettyExceptions.php(65): Slim_Middleware_MethodOverride->call()
#12 /home/ubuntu/datawrapper/vendor/slim/slim/Slim/Slim.php(1094): Slim_Middleware_PrettyExceptions->call()
#13 /home/ubuntu/datawrapper/www/index.php(95): Slim->run()
#14 {main}