miq_appliance_upgrade.md

Upgrading the ManageIQ appliance

I've been working to upgrade ManageIQ, and there are a few paths you can take with this.

1. Deploy a new appliance, load configs and database on
2. Deploy from scratch, load configs and database on
3. Upgrade the app upon the existing appliance

For this article, I'm examining the final option: upgrading the application upon the existing appliance. Guide will be written against my existing manageiq-dev server. If other options prove more viable I'll write about them as well and re-arrange this!

Glossary

• PostgreSQL ManageIQ database, /var/opt/rh/rh-postgresql95/lib/pgsql/data
• Application files, MIQ project files within /var/www/miq/vmdb
• User customizations, /root/manageiq
• Untracked files, similar to User customizations, but modifications/additions within the application files that are unique to this instance and likely hand-rolled
• vmdb, aliased to "cd /var/www/miq/vmdb"

Current State:

• Current version: Fine-4
• Desired version: Gapridashvili-7

How-to / Walkthrough:

Take a snapshot of the machine if possible. VMware, Hyper-V, etc

Stop all manageiq-related services. You'll leave postgres up as there will be a migration script run against it.

systemctl stop evmserverd

Take a copy of the database in case you destroy it but don't want to revert everything

backup_target=/mnt/backups/$(date "+%Y_%m_%d")
backup_file=$backup_target/vmdb_$(date -u --iso-8601=seconds)
mkdir $backup_target
pg_dump --format custom --file $backup_file vmdb_production

Back up all customizations in application files

vmdb
git status
# On branch fine
# Changes not staged for commit:
#   (use "git add/rm <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#    deleted:    log/.gitkeep
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#certs.prod/
#certs/server.cer
#certs/server.cer.OLD
#certs/server.cer.key
#certs/server.cer.key.OLD
#certs/server_original.cer
#certs/server_original.cer.key
#certs/v2_key_prod
#lib/tasks/rhconsulting_alerts.rake
#lib/tasks/rhconsulting_buttons.rake
#lib/tasks/rhconsulting_customization_templates.rake
#lib/tasks/rhconsulting_illegal_chars.rb
#lib/tasks/rhconsulting_miq_ae_datastore.rake
#lib/tasks/rhconsulting_options.rb
#lib/tasks/rhconsulting_policies.rake
#lib/tasks/rhconsulting_provision_dialogs.rake
#lib/tasks/rhconsulting_reports.rake
#lib/tasks/rhconsulting_roles.rake
#lib/tasks/rhconsulting_service_catalogs.rake
#lib/tasks/rhconsulting_service_dialogs.rake
#lib/tasks/rhconsulting_tags.rake
#lib/tasks/rhconsulting_widgets.rake
#log

With the above information, we'll back up the untracked files that matter as well as the application files.

backup_target=/mnt/backups/$(date "+%Y_%m_%d")
cp -r certs.prod/ $backup_target/
cp -r certs/ $backup_target/
cp lib/tasks/rhconsult* $backup_target/

Now for the Git work. Somewhere in the (super complicated) ManageIQ appliance build process they do either a git configuration or a shallow clone that limits what branches are visible from within that working local repository. We need to semi-undo that by adding the branch

# explicitly add gaprindashvili branch to branches list
git remote set-branches --add origin gaprindashvili

# update everything
git fetch
 
# create local branch
git checkout -b gaprindashvili
 
# set upstream for local branch 1:1
git --set-upstream-to origin/gaprindashvili
 
# update again to ensure we have latest of gaprindashvili vs origin
git fetch
 
# reset hard onto what origin's state is, becoming up to date and preserving unstaged changes
git reset --hard origin/gaprindashvili
 
# check up-to-date-ness
git status
git log

Now that you're on the desired version and commit, run the MIQ project update script, this does a bundle update, db migration, and bower install all in one. Maybe a little more too?

vmdb
bin/update

In this Fine -> gapri specifically, I encountered an error during dependency resolution where installing the sqllite3 gem failed to install. This is because the sqlite3 gem is not actually a gem, but is a C-thing that needs to be compiled, and we are missing the libraries to do so with.

yum install sqlite-devel -y

Update your bower too, because it's gonna break the update script otherwise

vmdb
npm i -g bower

Retry the update script, it now gets beyond that sqlite depend failure and goes on to migrate the database, fetch new bower depends, and more.

vmdb
bin/update

Take a short walk/get a coffee, this takes a few minutes at least.

Upon a successful execution of the update script, reboot!

sync;reboot

When the machine comes back up, ManageIQ will appear to start as normal, but httpd will not. Set a generic httpd SELinux policy, allowing access to config files (and other normal webserver-y things)

setsebool -P httpd_unified 1
systemctl start httpd