Tatoeba update 2017-02-12

tatoeba_update_2017-02-12.md

• Add announcement on prod (~19:00 UTC = 20:00 my time).

$ cd /var/www-prod
$ git pull

• Switch to maintenance mode (~20:00 UTC = 21:00 my time).

$ touch /var/www-prod/maintenance.mode

• Comment out Sphinx cronjob.

$ sudo crontab -e

• Create branch prod_2017-02-12 and switch prod to new branch.

$ cd /var/www-prod
$ git checkout prod_2017-02-12

• Install dependencies.

$ composer install

• Copy the CakePHP config

$ cd app/Config
$ cp ../config/core.php .
$ cp ../config/database.php .

• Add stuff in app/config/core.php

/**
 * Configure the Error handler used to handle errors for your application. By default
 * ErrorHandler::handleError() is used. It will display errors using Debugger, when debug > 0
 * and log errors with CakeLog when debug = 0.
 *
 * Options:
 *
 * - `handler` - callback - The callback to handle errors. You can set this to any callable type,
 *   including anonymous functions.
 *   Make sure you add App::uses('MyHandler', 'Error'); when using a custom handler class
 * - `level` - integer - The level of errors you are interested in capturing.
 * - `trace` - boolean - Include stack traces for errors in log files.
 *
 * @see ErrorHandler for more information on error handling and configuration.
 */
  Configure::write('Error', array(
          'handler' => 'ErrorHandler::handleError',
          'level' => E_ALL & ~E_DEPRECATED,
          'trace' => true
  ));

/**
 * Configure the Exception handler used for uncaught exceptions. By default,
 * ErrorHandler::handleException() is used. It will display a HTML page for the exception, and
 * while debug > 0, framework errors like Missing Controller will be displayed. When debug = 0,
 * framework errors will be coerced into generic HTTP errors.
 *
 * Options:
 *
 * - `handler` - callback - The callback to handle exceptions. You can set this to any callback type,
 *   including anonymous functions.
 *   Make sure you add App::uses('MyHandler', 'Error'); when using a custom handler class
 * - `renderer` - string - The class responsible for rendering uncaught exceptions. If you choose a custom class you
 *   should place the file for that class in app/Lib/Error. This class needs to implement a render method.
 * - `log` - boolean - Should Exceptions be logged?
 * - `extraFatalErrorMemory` - integer - Increases memory limit at shutdown so fatal errors are logged. Specify
 *   amount in megabytes or use 0 to disable (default: 4 MB)
 * - `skipLog` - array - list of exceptions to skip for logging. Exceptions that
 *   extend one of the listed exceptions will also be skipped for logging.
 *   Example: `'skipLog' => array('NotFoundException', 'UnauthorizedException')`
 *
 * @see ErrorHandler for more information on exception handling and configuration.
 */
  Configure::write('Exception', array(
          'handler' => 'ErrorHandler::handleException',
          'renderer' => 'ExceptionRenderer',
          'log' => true
  ));

• Change app/config/database.php:

'driver' => 'mysql'

to

'datasource' => 'Database/Mysql'

• Run ACL commands.

$ cd /var/www-prod/app

Console/cake acl delete aco Recordings
Console/cake acl create aco controllers Audio
Console/cake acl create aco Audio import
Console/cake acl create aco Audio save_settings

Console/cake acl grant Group.2 Audio all
Console/cake acl grant Group.3 Audio all
Console/cake acl grant Group.4 Audio all
Console/cake acl deny  Group.2 Audio/import all
Console/cake acl deny  Group.3 Audio/import all
Console/cake acl deny  Group.4 Audio/import all

• Run database update scripts.

$ cd /var/www-prod/docs/database/updates
$ mysql -u tatouser -p < 2016-12-22_1-schema.sql
$ mysql -u tatouser -p < 2016-12-22_2-data.sql

• Regerenate sphinx.conf and reindex sentences.

$ cd /var/www-prod/app
$ Console/cake sphinx_conf > /etc/sphinxsearch/sphinx.conf.new
$ cd /etc/sphinxsearch/
$ mv sphinx.conf sphinx.conf.20170212; mv sphinx.conf.new sphinx.conf
$ screen -R sphinx
$ cd /var/www-prod/app
$ Console/cake sphinx_indexes update main
// Detach screen: ctrl+a, d

• Make backup of current nginx config.

$ cd /etc/nginx/sites-enabled
$ cp tatoeba ../tatoeba.20170212

• Change nginx config in the tatoeba.org server block

(1)

        location / {
            limit_req zone=one burst=10;
            if (-f $document_root/../../maintenance.mode) {
                return 503;
            }
            try_files $uri $uri/ /index.php?url=$uri&$args;
        }

to

        location / {
            limit_req zone=one burst=10;
            if (-f $document_root/../../maintenance.mode) {
                return 503;
            }
            try_files $uri $uri/ /index.php?$args;
        }

(2)

        location ~ \.php {
                try_files $uri =404;
                include /etc/nginx/fastcgi_params;
                fastcgi_split_path_info ^(.+\.php)(/.+)$;
                fastcgi_pass 127.0.0.1:9000;
                fastcgi_intercept_errors on;
                fastcgi_index index.php;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        # wait a bit more, to have a higher chance to get a page under heavy load
        fastcgi_read_timeout 180;
        }

to

        location ~ \.php$ {
            try_files $uri =404;
            include /etc/nginx/fastcgi_params;
            fastcgi_pass    127.0.0.1:9000;
            fastcgi_index   index.php;
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            # wait a bit more, to have a higher chance to get a page under heavy load
            fastcgi_read_timeout 180;
        }

• Restart nginx

$ /etc/init.d/nginx restart

• Update dependencies for dedup script.

$ cd /var/www-prod/docs/tatoeba2-django/
$ pip install -r requirements.txt

• Wait until indexation is done then remove comments for Sphinx cronjob.

$ sudo crontab -e

• Turn off maintenance mode.

$ rm /var/www-prod/maintenance.mode