Notes

notes.md

clone Manageiq

git clone git@github.com:romanblanco/manageiq.git 
git remote add upstream git@github.com:ManageIQ/manageiq.git

Gemfile.dev.rb

require 'rubygems'
gem 'pry'
gem 'pry-byebug'

gem 'pry-rails'
gem 'pry-doc'
gem 'pry-loudmouth'
gem 'pry-inline'
gem 'awesome_print'
gem 'rack-mini-profiler'
gem 'table_print'

Compressed IDs (not anymore: ManageIQ/manageiq-ui-classic#2482)

[8] pry(#<OpsController>)> include CompressedIds
=> Object3
[9] pry(#<OpsController>)> from_cid("10r1")
=> 10000000000001
[10] pry(#<OpsController>)> to_cid(from_cid("10r1"))
=> "10r1"

Debugging snippets

object = Object.new
puts object.method(:blank?).source_location
=> ["/gems/activesupport-5.0.0.beta1/lib/active_support/core_ext/object/blank.rb", 14]

Vm.all.collect { |vm| 
  {:name => vm.name,
   :vendor => vm.vendor,
   :version => ExtManagementSystem.find(vm.ems_id).api_version.to_f
  } if (vm.ems_id.present? && 
        vm.vendor == "vmware" && 
        ExtManagementSystem.find(vm.ems_id).api_version.to_f > 6.5) 
}.compact

Add provider from Rails console

# to find the providers model for example:
rg 'ManageIQ::Providers::\w+::CloudManager'

ManageIQ::Providers::Azure::CloudManager.create!(:name => "new provider xyz, :zone => Zone.first)

Add embeded ansible (not a real one)

server = MiqServer.my_server(true)
server.role = "embedded_ansible"
server.activate_roles('embedded_ansible')
server.save! provider = ManageIQ::Providers::EmbeddedAnsible::Provider.first_or_initialize
provider.name = "Embedded Ansible"
provider.zone = server.zone
provider.verify_ssl = 0
provider.url = "https://dev-ansible-tower3.cloudforms.lab.eng.rdu2.redhat.com/api/v1/"
provider.save!
provider.update_authentication(:default => {:userid => "admin", :password => "smartvm"}) provider.authentication_check
EmsRefresh.refresh(provider.managers.first)

Refresh provider from Rails console

a = EmsCloud.find_by(:id => 2)
EmsRefresh.refresh(a)

also see: https://github.com/martinpovolny/manageiq-utils

VMs

Vm.archived? == true if Vm.ems_id.nil? && Vm.storage.present? 
Vm.orphaned? == true if Vm.ems_id.nil? && Vm.storage.nil?
# ems -- ext management system

Openstack testing credentials

admin & dog8code
ssh: root & redhat --> source ./keystonerc_admin

Angular, inspect last selected element

angular.element($0).scope()
console.log(JSON.stringify(...))

Debugging components

rm -fr node_modules/ yarn.lock
yarn install
yarn run build
yarn start # does not minify the code

if waiting_for_task

bundle exec rails c

=> enable_console_sql_logging ; simulate_queue_worker

if rescue in decrypt: can not decrypt v2_key encrypted string

bundle exec ./tools/fix_auth.rb -P smartvm -p miq_password [database]

Gettext

bundle exec rake gettext:find
bundle exec rails c

require 'gettext_i18n_rails/haml_parser'
file = 'file.html.haml'
Gettext::Tools::x 
Gettext.run(file)

Linters

haml_lint -i Rubocop rubocop.yml <file.haml>

Tests

bundle exec rake test:javascript  
bin/rake environment jasmine  
bundle exec rspec SPEC=spec/helpers  
bundle exec rspec -e "string"  
bundle exec rspec <spec_path> 
yarn run jest <spec_path> # in manageiq-ui-classic ; if jest test 

in config/database.yml:

test:&test:  
  database: vmdb_test<%=ENV['TEST_ENV_NUMBER']%>  

PARALLEL=true bundle exec rake test:vmdb:setup
PARALLEL=true bundle exec rake test:vmdb

Database

PSQL

• \l -- list databases
• \c -- connect to database
• \dt -- list tables
• \dtS+ -- (options: S = show system objects, + = additional detail)
• \d __table__ -- list tables, views, and sequences
• \x -- toggle expanded output (looks better)
• \du+ -- list users
• \q -- quit

Setup

postgres $ psql

DROP DATABASE vmdb_development;
CREATE DATABASE vmdb_development OWNER root;

# set region
bundle exec rake db:seed
bundle exec rake db:migrate

Backup

pg_dump -Fc dbname > filename # A custom-format dump is not a script for psql, but instead must be restored with pg_restore

pg_dump -C  vmdb_colin > vmdb_colin.sql

Restore

pg_restore -Fc db.backup -d vmdb_development

psql vmdb_development root -f backup backup.sql` 

psql vmdb_development < backup.sql

Update database records -f

[12] pry(main)> OrchestrationStack.any? { |os| os.retired? }
  OrchestrationStack Load (2.3ms)  SELECT "orchestration_stacks".* FROM "orchestration_stacks"
  OrchestrationStack Inst Including Associations (13.4ms - 248rows)
=> false
[13] pry(main)> OrchestrationStack.first.update(:retired => true)
  OrchestrationStack Load (0.6ms)  SELECT  "orchestration_stacks".* FROM "orchestration_stacks" ORDER BY "orchestration_stacks"."id" ASC LIMIT $1  [["LIMIT", 1]]
  OrchestrationStack Inst Including Associations (0.2ms - 1rows)
   (0.4ms)  BEGIN
  SQL (2.9ms)  UPDATE "orchestration_stacks" SET "retired" = $1, "updated_at" = $2 WHERE "orchestration_stacks"."id" = $3  [["retired", true], ["updated_at", 2017-03-02 19:20:13 UTC], ["id", 10000000000001]]
   (1.1ms)  COMMIT
=> true
[14] pry(main)> OrchestrationStack.any? { |os| os.retired? }
  OrchestrationStack Load (1.2ms)  SELECT "orchestration_stacks".* FROM "orchestration_stacks"
  OrchestrationStack Inst Including Associations (14.0ms - 248rows)
=> true

Log

tail -f log/evm.log | egrep 'ERROR|FATAL|...|' --color

in Configuration → Advanced tab:

:level_rails: info

Disable assets

see also ManageIQ/manageiq#9577 (comment)

in config/initializers/no_assets.rb:

Rails.application.config.assets.debug = false # merge all assets to one
Rails.application.config.assets.quiet = true # do not print assets

in .git/info/exclude:

config/initializers/no_assets.rb

in User → Configuration → Advanced:

:log:
:level_rails: info

*:level_rails: info does not print to the output, but hides DB queries

bin/update without test & automate reset

SKIP_TEST_RESET=true SKIP_AUTOMATE_RESET=true bin/update

Github

• use [skip ci] in commit message to skip CI

• Fixes: #[0-9]+ to close issue automatically, when PR is merged

• comment @miq_bot add-label / remove-label / assign

Git

push specific commit:

git push <remote> <commit hash>:<branch>

Patch

https://stackoverflow.com/a/6658352 https://stackoverflow.com/questions/4770177/git-patch-does-not-apply#15375869

git format-patch -1 <sha>
git am < file.patch

git diff > 1234.diff
git apply --ignore-space-change --ignore-whitespace --reject 1234.diff

or

git diff --no-prefix > fix.patch
patch -p0 < fix.patch 
# or
git apply --check fix.patch

virt-manager setup

sudo dnf install @virtualization
sudo vim /etc/polkit-1/rules.d/80-libvirt.rules

in /etc/polkit-1/rules.d/80-libvirt.rules, write:

polkit.addRule(function(action, subject) {
  if (action.id == "org.libvirt.unix.manage" && subject.local && subject.active && subject.isInGroup("wheel")) {
      return polkit.Result.YES;
  }
});

systemctl status libvirtd

to setup NAT, see: http://wiki.libvirt.org/page/TaskNATSetupVirtManager

DDNS (kria) setup

sudo systemctl enable sshd
sudo systemctl start sshd
# wget redhat-ddns-client-1.3-4.noarch.rpm # link is on MOJO attachement
dnf -y install redhat-ddns-client-1.3-4.noarch.rpm

in /etc/rc.d/rc.local, write:

#!/bin/sh
/usr/bin/redhat-ddns-client update &

chmod +x /etc/rc.d/rc.local
systemctl enable rc-local.service
systemctl start rc-local.service
#systemctl status rc-local.service
redhat-ddns-client enable

TODO: Kiosk (kria) setup

17:42 <skateman> exec sudo -u kiosk XAUTHORITY=/home/kiosk/.Xauthority DISPLAY=:0 google-chrome-stable \ 
17:42 <skateman>   --kiosk \
17:42 <skateman>   --window-size=1280,1024 \
17:42 <skateman>   --disable-session-crashed-bubble \
17:42 <skateman>   --no-default-browser-check \
17:42 <skateman>   --no-first-run \
17:42 <skateman>   --disable-translate \
17:42 <skateman>   https://github.com/ManageIQ/manageiq/pulls?utf8=%E2%9C%93&q=is%3Aopen%20is%3Apr%20label%3Aui%20-label%3Awip

____

cat << EOF > /etc/yum.repos.d/google-chrome.repo 
[google-chrome] 
name=google-chrome - \$basearch 
baseurl=http://dl.google.com/linux/chrome/rpm/stable/\$basearch
enabled=1
gpgcheck=1
gpgkey=https://dl-ssl.google.com/linux/linux_signing_key.pub
EOF

dnf install google-chrome-stable curl cabextract xorg-x11-font-utils fontconfig xterm
wget ftp://fr2.rpmfind.net/linux/sourceforge/p/po/postinstaller/fedora/releases/24/i386/msttcorefonts-2.5-3.fc24.noarch.rpm
dnf install msttcorefonts-2.5-3.fc24.noarch.rpm
dnf groupinstall "base-x"
useradd -m kiosk

cat << EOF > /root/.xinitrc
#!/bin/sh
cp \$XAUTHORITY /home/kiosk/.Xauthority
chown kiosk:kiosk /home/kiosk/.Xauthority
rm -rf /home/kiosk/.config/google-chrome

unclutter -idle 0 &

exec sudo -u kiosk XAUTHORITY=/home/kiosk/.Xauthority DISPLAY=:0 google-chrome-stable \
  --kiosk  \
  --window-size=1280,1024 \
  --disable-session-crashed-bubble \
  --no-default-browser-check \
  --no-first-run \
  --disable-translate \
  http://bit.ly/2cTYduB
EOF

____

cat /etc/systemd/system/xinit-login.service 
[Unit]
After=systemd-user-sessions.service

[Service]
Environment=XINITRC=/root/.xinitrc
ExecStart=/usr/bin/xinit

[Install]
WantedBy=multi-user.target



____

systemctl daemon-reload
systemctl enable xinit-login

Performance / Capacity & Utilization (C&U)

in Configuration → CFME Region → C&U → Switch everything on → Save

in Server:EVM → Switch C&U items on → Save

bundle exec rake evm:start

Toolbars

#miq-toolbar-menu - DOM ID contains data of toolbars

Setup separate ManageIQ server with different memcached server for master branch only

git clone https://github.com/ManageIQ/manageiq.git manageiq-master
cd manageiq-master ; mkdir plugins
git clone https://github.com/ManageIQ/manageiq-providers-amazon.git plugins/manageiq-providers-amazon
git clone https://github.com/ManageIQ/manageiq-content.git plugins/manageiq-content
git clone https://github.com/ManageIQ/manageiq-ui-classic.git plugins/manageiq-ui-classic
echo "override_gem 'manageiq-providers-amazon', :path => File.expand_path('plugins/manageiq-providers-amazon', __dir__)" >> Gemfile.dev.rb
echo "override_gem 'manageiq-content', :path => File.expand_path('plugins/manageiq-content', __dir__)" >> Gemfile.dev.rb
echo "override_gem 'manageiq-ui-classic', :path => File.expand_path('plugins/manageiq-ui-classic', __dir__)" >> Gemfile.dev.rb
cp config/database.pg.yml config/database.yml
sed -i 's/127.0.0.1\:11211/127.0.0.1\:11212/' config/settings.yml
bin/setup
memcached -p 11212 -l 127.0.0.1 -d
bundle exec rails s -b [::] -p 4000

To keep it updated I have simple run.sh script:

#!/bin/bash

# save changes
git stash

# update manageiq repo
git pull

# update ui-classic plugin
cd plugins/manageiq-ui-classic/
git pull
cd -

# update manageiq-provider-amazon plugin
cd plugins/manageiq-providers-amazon/
git pull
cd -

# update manageiq-content plugin
cd plugins/manageiq-content/
git pull
cd -

# get back changes
git stash pop

# run server
memcached -p 11212 -l 127.0.0.1 -d && bundle exec rails s -b [::] -p 4000

In browser open localhost4:4000 (or any other than localhost found in /etc/hosts)

Bugzilla workflow

1. NEW - nobody has looked at the BZ

2. ASSIGNED - manager/lead assigned the BZ to an engineer

3. ON_DEV - engineer actively looking at BZ - PR forthcoming

4. POST - fixed, merged and backported (if needed) -- Pending BUILD

5. MODIFIED - Included in BUILD

6. ON_QA - QE needs to verify fix in BUILD

7. VERIFIED - QE has verified fix in BUILD

Other

@sb -- sandbox

caller.select { |path| path if path !~/\.rbenv/ }

bundle exec rails s -b [::] -p 3000

sudo firewall-cmd --add-port 3000/tcp -zone=public

X_TREE_NODE_PREFIXES -- tree_builder.rb ( "xx" => "Hash", etc.) (views/layouts/gtl/_list.html.haml)

CLASSES_THAT_PARTICIPATE_IN_RBAC -- https://github.com/ManageIQ/manageiq/blob/master/lib/rbac/filterer.rb#L7

https://jetruby.com/expertise/common-ruby-on-rails-mistakes-controllers/ https://jetruby.com/expertise/common-rails-mistakes-model-database/ https://jetruby.com/expertise/common-rails-mistakes-ruby-way/

https://github.com/benoittgt/understand_ruby_memory/blob/master/README.md https://github.com/brunofacca/zen-rails-security-checklist/blob/master/README.md

http://betterspecs.org/

CI=1 bundle exec rake

... generates report in coverage/index.html

SSH

ssh-keygen -o -a 100 -t ed25519

CFME

origin: gitlab/romanblanco/cfme.git
downstream: gitlab/cfme/cfme.git
upstream: https://github.com/ManageIQ/manageIQ

git cherry-pick HASH

CFME Appliance

https://access.redhat.com/webassets/avalon/d/Mastering_CloudForms_Automation_Red_Hat/Mastering_CloudForms_Automation_Red_Hat.pdf

https://github.com/ramrexx/CloudForms_Essentials

Rails.logger.error

root@ip / smartvm

/var/www/miq/vmdb
/var/www/miq/vmdb/log/production.log
/var/www/miq/vmdb/log/evm.log
/opt/rh/cfme-gemset/bundler/gems/manageiq-ui-classic-35beb5e894ab/app

ui-components in manageiq-ui-classic

ui-components/ $ yarn link
manageiq-ui-classic/ $ yarn link @manageiq/ui-components
ui-components/ $ yarn
ui-components/ $ yarn start
manageiq-ui-classic/ $ yarn cache clean && rake yarn:clobber && bin/update
manageiq-ui-classic/ $ bin/webpack --watch

(~/devel/manageiq-ui-classic/ master yarn unlink @manageiq/react-ui-components)

or

• ~/devel/manageiq-ui-classic/ gaprindashvili rm -fr vendor/assets/bower_components/manageiq-ui-components
• ~/devel/manageiq-ui-classic/ gaprindashvili ln -s $HOME/devel/ui-components/ vendor/assets/bower_components/manageiq-ui-components
• ~/devel/manageiq-ui-classic/ gaprindashvili ls vendor/assets/bower_components/

*in case of emergency, manageiq-ui-classic/ $ rm -fr vendor/assets/bower/bower_components && bower install

ui-components in manageiq-ui-service

manageiq-ui-service/ $ yarn
manageiq-ui-service/ $ cd node_modules/@manageiq
manageiq-ui-service/node_modules/@manageiq/ $ ln -s $HOME/devel/ui-components/ ui-components
ui-components/ $ yarn build # this needs to be ran for every change in ui-components
manageiq-ui-service/ $ yarn start

To update ui-components:

ui-components/ $ yarn build

No data in UI, error in JS console

• ui-components is not compiled, -> yarn -> yarn start
• in manageiq-ui-classic: bin/webpack

CURL API GET

curl -i -X GET http://localhost:3000/api/service_dialogs/10000000000076?expand=resources -u admin:smartvm

Export / Import

bin/rake evm:export:customization_templates -- --directory /tmp/customization_templates
bin/rake evm:import:customization_templates -- --source /tmp/customization_templates
bin/rake evm:import:customization_template -- --source /tmp/customization_templates/Test_customization_template.yaml

bin/rake evm:export:reports -- --directory /tmp/reports
bin/rake evm:import:reports -- --source /tmp/reports
bin/rake evm:import:report -- --source /tmp/reports/Test_Report.yaml

... rails --help | grep -E "evm:(export|import)"

https://forums.fedoraforum.org/showthread.php?316527-Cannot-install-both-compat-openssl10-devel-and-openssl-devel-at-the-same-time

openssl issues while compiling older ruby versions on new fedora