Skip to content

Instantly share code, notes, and snippets.

@gwenshap
Created October 18, 2013 17:01
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 6 You must be signed in to fork a gist
  • Save gwenshap/7044525 to your computer and use it in GitHub Desktop.
Save gwenshap/7044525 to your computer and use it in GitHub Desktop.
Move CM Database to MySQL
These are the steps that you need to follow in order to migrate scm database from PostgreSQL to MySQL - let me know if you need assistance or have any further questions.
1. Stop all cluster services including the Cloudera Manager monitoring services and Agents on all hosts, using the Cloudera Manager UI.
2. Invoke the API call to /api/v4/cm/deployment. Example that directs the resulting output to a file for later use:
$ curl -v -u admin:password http://your_cloudera_manager_host:7180/api/v4/cm/deployment > path/to/file
Note: It is critical this file be retained and protected as it will be needed to re-import the functional configuration into the new database.
3. Take a fail-safe database backup from the embedded postgres database or make a tarball (or other copy) of the /var/lib/cloudera-scm-server-db/ directory to a safe location. Either will accomplish the needed outcome, just pick the most convenient for your environment. And backup contents of the directory /etc/cloudera-scm-server/*
4. On the server where the Cloudera Manager embedded database is running:
$ service cloudera-scm-server stop
$ service cloudera-scm-server-db stop
$ chkconfig cloudera-scm-server-db off
5. If not already done, ensure the MYSQL JDBC connector is installed and in the correct location [1] see "Installing the MySQL JDBC Connector".
6. Install, configure, and start a new database following the published instructions for MySQL (or Postgres ).
7. Follow the best-practice configuration inclusions to the my.cnf file.
8. Create "temp" db and grants for "temp" db as documented in this "Example 1: Running the script when MySQL is installed on another host" [2]
Example:
mysql> create temp;
mysql> grant all on *.* to 'temp'@'%' identified by 'temp' with grant option;
9. Execute $ scm_prepare_database.sh mysql -h mysqlhost -u temp -ptemp --scm-host [cloudera-manager-host] [database-name] [username] [password]
Note: NB: run /usr/share/cmf/schema/scm_prepare_database.sh with no arguments to get syntax assistance and for help customizing this preparation for your environment (see more in [3]).
10. When complete, start the Cloudera Manager server service:
$ service cloudera-scm-server start
11. Log in to the Cloudera Manager web user interface and upload the enterprise license key.
12. After license upload do the following:
$ service cloudera-scm-server restart
13. From the command line of the node where the configuration export (from step 2) was performed, issue:
$ curl --upload-file /path/to/file -u youruser:yourpassword http://your_cloudera_manager_host:7180/api/v4/cm/deployment?deleteCurrentDeployment=true
14. Log in to Cloudera Manager and verify configuration options are now all present.
15. To avoid later contention: yum erase cloudera-manager-server-db
Note: Ensure the file /etc/cloudera-scm-server/db.properties reflects the detail for the newly-established database and not the previous embedded postgres database.
Links:
[1] http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Installation-Guide/cmig_install_mysql.html
[2] http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Installation-Guide/cmig_install_path_C.html
[3] http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Installation-Guide/cmig_install_path_B.htm
@tgraysonco
Copy link

This is no longer a safe procedure to use for migrating cloudera databases for CM and its Management Services.

@Raghu-Gurrala
Copy link

I have migrated from Postgres(embedded) to MySQL successfully two days back for CDH version 5.9

There are couple of steps need to be taken care of in the new Version of CDH 5.x

  1. Check you CDH api version and update it.
  2. At line 27:
    $ curl --upload-file /path/to/file -H "Content-Type:application/json" -u youruser:yourpassword http://your_cloudera_manager_host:7180/api/v14/cm/deployment?deleteCurrentDeployment=true
  3. In the end you need to delete the Guid for cloudera manager agent on the server's

For CDH version 5.X --> SSH to all the servers including the cloudera-manager-host and then delete the giud file in /var/lib/cloudera-scm-agent/cm_guid. [3][4]
Restart all the cloudera-manager-agents.
$ service cloudera-scm-agent restart.

[3] https://www.cloudera.com/documentation/enterprise/release-notes/topics/cm_rn_known_issues.html
[4] http://community.cloudera.com/t5/Cloudera-Manager-Installation/Error-CM-Server-guid-updated-CDH-5-9-0/m-p/47221

@itvikaskumar
Copy link

I tried to migrate Cloudera Manager to mysql but while uploading the backed up deployment. I retrieve below error.
does any one know how to fix this?

curl --upload-file /root/mysql/deployment.json -H "Content-Type:application/json" -u admin:admin http://Clouderamanagerhost:7180/api/v4/cm/deployment?deleteCurrentDeployment=true
{
"message" : "Unknown configuration attribute 'role_jceks_password'."

@widewing
Copy link

@itvikaskumar You need to change the API version. v15 should be the case

@mughilanand
Copy link

I execute all steps but no configuration and service is shown in cloudera manager

@shaileshagarwal17
Copy link

Any help with migrating cm databases from mysql to oracle?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment