Skip to content

Instantly share code, notes, and snippets.

@balsama

balsama/ama-manual-cloud-deploy.md

Last active June 11, 2020 03:59
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save balsama/d5c02f14a5536a18f2c92960132671b9 to your computer and use it in GitHub Desktop.
Save balsama/d5c02f14a5536a18f2c92960132671b9 to your computer and use it in GitHub Desktop.
Download ZIP
Instructions to manually deploy Migrate Assistant to an Acquia Cloud environment
Raw
ama-manual-cloud-deploy.md

Manually deploy a newly created migrate project to Acquia Cloud

Prerequisites

  1. Admin access to an Acquia Cloud hosting environment.
  2. A Drupal 7 site running locally
  3. SSH keys on Cloud for your local machine

Instructions

  1. Generate a new migrate project. Use the migrate-quickstart command to generate a new Drupal 9 project per the instructions in the Quick Start section of AMA’s README. You should also run the commands that are echoed at the end of following those instructions.

    Note: the last output command, composer run server, is optional and only needed if you intend to log into the site locally and you don’t have a local web server already running.

  2. Add a .gitignore. We'll be initializing a git repo later. We don't want to add the files directory to that repo, so let's create a .gitignore file now and exclude that directory. Make sure your at the root of the new project created in Step 1. (You should be already if you followed the instructions in Step 1 properly).

    $ echo "/docroot/sites/default/files" >> .gitignore

  3. Create a directory to hold our exported config. We're going to install this site on Cloud from existing config exported from this initial local install. Best practices include putting that config in a git-managed directory outside of docroot. We'll tell Drupal to use this directory later in the settings.php file. For now, just create it.

    $ mkdir config_sync
$ touch config_sync/.gitkeep

  4. Make Standard profile compatible with installing from config. The Standard profile that ships with Drupal 9 cannot be installed from config because of its install hook and its use of the Shortcut module. An issue exists to fix this so we can use tha patch from that issue. Since composer config doesn't allow you to edit arrays more than three levels deep, we'll have to open up composer.json and manually edit it. Search for the drupal/core section within extra => patches. Add the following line to the end of the list of patches:

    "Issue #2357215: Clean up hook_install() in Standard Install Profile": "https://www.drupal.org/files/issues/2019-12-14/drupal-standard_clean_up_hook_install-2357215-47.patch"

    Now we need to update `drupal/core so that it applies the new patch:

    $ composer update drupal/core

  5. Export the site's config. At this point, we can take a snapshot of the destination site so we can recreate it elsewhere (Cloud). Run the following command:

    drush config:export --yes

  6. Create Migrate source database on cloud if it doesn't exist already.

    1. Determine if you have a Migrate database already defined
      1. log into your Cloud subscription
      2. Select the destination environment (Dev, Stage, or Prod)
      3. Select "Databases" from the menu on the left
      4. If you see a "Migrate" database already, open its card and take note of the credentials listed under the details tab and move on to the next step.
    2. If you don't see a database named "Migrate" below the first database listed, Press the "Add Database" button in the upper right corner of the, and:
      1. Give it a name (migrate) and press "Add Database.
      2. Take note of the credentials listed under the "Details" tab.

  7. Tell Drupal where databases and files are. We need several tweaks to our settings file so that our app will be able to find all of the databases and config/file directories on Cloud. Open the docroot/sites/default/settings.php file and make the following changes. Note that you might need to grant write access to the file first before editing. To do so, you can run chmod -R +w docroot/sites/default before opening the file for editing.

    1. Add the default database connection snippet to the bottom of the file. To find this snippet:

      1. log into your Cloud subscription
      2. Select the destination environment (Dev, Stage, or Prod)
      3. Select "Databases" from the menu on the left
      4. You should see a database listed there that has the same name as your subscription. From within that tile, click on "PHP". You can copy the snippet from there. It should look something like this: 
        if (file_exists('/var/www/site-php')) {
  require '/var/www/site-php/<NAMEOFSUB>/<NAMEOFSUB>-settings.inc';
}

    2. Add the D7 source database settings to the settings.php file. Add the following snippet, substituting the connection details for the Migrate database from the previous step for <DBNAME>, <DBUSERNAME>, and <DBPASS>.

      $databases['migrate']['default'] = [
  'database' => '<DBNAME>',
  'username' => '<DBUSERNAME>',
  'password' => '<DBPASS>',
  'prefix' => '',
  'host' => '127.0.0.1',
  'port' => '3306',
  'namespace' => 'Drupal\\Driver\\Database\\mysql',
  'driver' => 'mysql',
];

    3. Replace the autogenerated config directory with the one created outside of docroot.

      1. Delete the line that starts with $settings['config_sync_directory'] = 'SOMELONEPATH'
      2. At the end of the settings.php (or below the section that starts with if (file_exists('/var/www/site-php')) {), re-add that setting with the directory we created earlier: 
        $settings['config_sync_directory'] = '../config_sync';

    4. Add the path to the migrate source site's files directory (which we will upload later). Add the following line to the settings.php file:

      $settings['migrate_source_base_path'] = 'sites/files/migrate'

  8. Add Drush aliases. We'll need these to connect to and interact with the Cloud server.

    1. Download your Drush aliases from Acquia Cloud.
      1. From the Clou UI, select your avatar in the upper right corner and select "Account Settings" from the drop-down menu.
      2. Select "Credentials" from the secondary menu that appears.
      3. Download the Drush 9 credentials from the link provided. Note: Don't use the Drush 8 aliases link which comes first and is more prominent. The link you need is in the line that starts with the words "If you are using Drush 9, you can download aliases here".
      4. Save the aliases somewhere on your local machine and unzip the folder they came in.
    2. Add them to the app's repo.
      1. Create a directory for the aliases (from the root of your new app): 
        $ mkdir -p drush/sites
      2. Copy the alias for your migrate environment subscription into the newly created directory.

  9. Add the migrate source database to Cloud. You should already have a migrate database defined in Cloud from an earlier step. Navigate back to that UI and open the "Details" tab so that you have the credentials handy for copying and pasting into the steps in this section.

    You'll also need to know the ssh username and host for the target Cloud environment. You can find find these in the Cloud UI. Select the environment and then "Users and Keys". The SSH users and URLs are under the "SSH public keys" section hidden under a "Show examples" link. The general pattern is:

    <SUBSCRIPTION>.<ENVIRONMENT>@<SERVERNAME>.devcloud.hosting.acquia.com

    1. Create a database archive of your local D7 site. You can use Drush to do this:

      $ drush sql-dump --gzip --result-file=~/<NAME_OF_EXPORTED_DATABASE>.sql

    2. Upload the migrate source database to cloud. You can use scp for this. Substitue your SSH user and servername, along with the name you chose when you exported the database. (Note that Drush will append a datetime string to end of the name you supplied for the exported database):

      $ scp ~/<NAME_OF_EXPORTED_DATABASE>.sql ssh <SUBSCRIPTION>.<ENVIRONMENT>@<SERVERNAME>.devcloud.hosting.acquia.com:/mnt/files/<SUBSCRIPTION>.<ENVIRONMENT>/backups

      Note: At the end of the upload, you might see a message similar to ssh: No such file or directory. This does not necessarily mean the upload was unsuccessful. You can use drush <SUBSCRIPTION>.<ENVIRONMENT> ssh to SSH into the server and verify the file is there.

    3. SSH into the Cloud server so that we can interact with mysql directly:

      $ drush @<SUBSCRIPTION>.<ENVIRONMENT ssh

    4. Drop existing tables (if necessary). If this is an existing database that has been used for other demos, you'll need to drop all of its tables first. Substitute the placeholders in the following command with the credentials you found in the "Details" tab of the Migrate database in the Cloud UI. This step is done from the Cloud server after SSH-ing in in the previous step.

      Note: This is obviously a destructive step. Take care that you are interacting with the Migrate database in the correct environment before proceeding:

      $ mysqldump -u<DBUSER> -p<DBPASS> \
    --add-drop-table --no-data <DBNAME> | \
    grep -e '^DROP \| FOREIGN_KEY_CHECKS' | \
    mysql -u<DBUSER> -p<DBPASS> <DBNAME>

    5. Import the database:

      mysql -u <DBUSER> -p <DBNAME> < /mnt/files/<SUBSCRIPTION>.<ENVIRONMENT>/backups/<NAME_OF_EXPORTED_DATABASE>.sql

  10. Add the migrate source files to cloud. Similar to the database, the migration target site will need the files directory from the source site available which means we need to upload it to cloud.

    1. Create a directory for the files.
      1. Connect to the Cloud server: 
        $ drush @<subscription>.<environment ssh
      2. Create the directory: 
        $ mkdir /mnt/files/<SUBSCRIPTION>.<ENVIRONMENT>/files/migrate
      3. Disconnect from the Cloud server: 
        $ exit
    2. Archive the files. In order to efficiently move the files, they should be archived and compressed. From the files files directory in the source D7 site run the following command: 
      tar -zcvf d7_files.tar.gz *
    3. Move the files to the Cloud server. Use SCP to move the files from your local computer to the Cloud server: 
      $ scp <PATH_TO_ARCHIVED_FILED>/<FILENAME>.tar.gz ssh <SUBSCRIPTION>.<ENVIRONMENT>@<SERVERNAME>.devcloud.hosting.acquia.com:/mnt/files/<SUBSCRIPTION>.<ENVIRONMENT>/files/migrate
    4. Uncompress and extract the uploaded files.
      1. Connect to the Cloud server: 
        $ drush @<SUBSCRIPTION>.<ENVIRONMENT> ssh
      2. Switch to the source files directory. (For some reason, I got permissions erros if I wasn't in the actualy directory): 
        $ cd /mnt/files/<SUBSCRIPTION>.<ENVIRONMENT>/files/migrate
      3. Extract the files: 
        $ tar -xf /mnt/files/<SUBSCRIPTION>.<ENVIRONMENT>/files/migrate/<FILENAME>.tar.gz

  11. Add everything to git and push to cloud.

    1. Exit the cloud server. You can exit the Cloud environment and return to your local. Any interaction with the Cloud server from here on out can be executed through the Drush Aliases. 
      $ exit
    2. Initialize a new git repo 
      $ git init
    3. Add the Cloud git remote. You can find the URL for the Cloud git repo in the Cloud UI. It's in the upper right corner behind a Git logo. I named the remote ah in this command but you can use origin or whatever you please. Subsequent commands will assume you have named yours ah as well. 
      $ git remote add ah <URL>
    4. Remove submodules. Since Acquia Migrate was cloned into this repo, it's will be considered a Git Submodule - which have a whole set of pain points of their own - unless we delete the .git directory from it. 
      $ rm -rf acquia_migrate/.git
    5. Checkout a unique branch. You might want to checkout a unique branch in case this repo has other things going on already so as to not cause conflicts with an existing master branch. For example: 
      $ git checkout -b <UNIQUE_BRANCHNAME>
    6. Add everything: 
      $ git add -A
    7. Commit it and push it to Cloud: 
      $ git commit -m "Initial commit"
$ git push --set-upstream ah <UNIQUE_BRANCHNAME>

  12. Deploy the new branch to a Cloud environment.

    1. Login to the Cloud UI.
    2. Select <UNIQUE_BRANCHNAME> from the "Swicth Code" dialog in the environmennt of your choice.

  13. Install the site on Cloud.

    1. INSTALL! From you local machine, use your Drush alias to install the site on Cloud from the config we exported earlier. 
      $ drush @datdemos.prod site:install --existing-config --yes

That's it! 🎉🚀

@wimleers
Copy link

🤯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment