how-to-migrate-icon1-db-to-icon2.md

How to migrate ICON1 database to ICON2

Requirements

• H/W Spec

  • 2TB SSD (300GB for ICON1 data, 1.2TB+ for ICON2 data)
  • CPU with 4 Cores (the more the better)
  • 16GB RAM (the more the better)

• Download and install Docker and Docker Compose

  • Get Docker
  • Install Docker Compose

• goloop-icon:v1.0.7 docker image

  • Build

    $ git clone git@github.com:icon-project/goloop.git
    $ cd goloop
    $ git checkout v1.0.7
    $ make goloop-icon-image
    $ docker tag goloop-icon goloop-icon:v1.0.7
    

  • Verify the generated image

    $ docker images goloop-icon
    REPOSITORY    TAG       IMAGE ID       CREATED          SIZE
    goloop-icon   latest    0868fa391ca9   15 minutes ago   733MB
    goloop-icon   v1.0.7    0868fa391ca9   15 minutes ago   733MB
    

• ICON1 mainnet database

  • Download and store the database under ./mainnet folder

    # curl <URL for ICON1 data> --output mainnet_data.tar.gz
    # mkdir mainnet
    # tar zxf mainnet_data.tar.gz -C mainnet
    

Usage

Configure Containers

Prepare docker-compose.yml like the following example.

version: "3.7"
services:
    web:
        image: nginx:latest
        volumes:
            - ./web:/usr/share/nginx/html
    node:
        image: goloop-icon:v1.0.7
        volumes:
            - ./data:/goloop/data
            - ./config:/goloop/config
            - ./mainnet:/mainnet
        ports:
            - "9080:9080"
            - "8080:8080"

Start containers

$ docker-compose up -d

Check the system info

$ docker-compose exec node goloop system info

The result will be something like the following.

{
  "buildVersion": "v1.0.7",
  "buildTags": "linux/amd64 tags(rocksdb)-2021-11-09-08:14:01",
  "setting": {
    "address": "hxb7176940240a69ab0ef785059ba680bde00ebbf1",
    "p2p": "127.0.0.1:8080",
    "p2pListen": "0.0.0.0:8080",
    "rpcAddr": ":9080",
    "rpcDump": false
  },
  "config": {
    "eeInstances": 1,
    "rpcDefaultChannel": "",
    "rpcIncludeDebug": false
  }
}

Check the address value which will be used later for validator setting.

Make channel to ICON

$ docker-compose exec node goloop chain join \
    --platform icon \
    --channel icon_dex \
    --genesis icon_genesis.zip \
    --tx_timeout 60000 \
    --node_cache small \
    --normal_tx_pool 10000 \
    --db_type rocksdb \
    --seed 127.0.0.1

You can check the status of the chain.

$ docker-compose exec node goloop chain ls
[
  {
    "cid": "0x1",
    "nid": "0x1",
    "channel": "icon_dex",
    "state": "stopped",
    "height": 0,
    "lastError": ""
  }
]

Create a migration config file

Prepare a ./web/migration_config.json file that contains the following information.

$ cat ./web/migration_config.json 
{
    "validators": [
        "hxb7176940240a69ab0ef785059ba680bde00ebbf1"
    ]
}

Note that the validator address should be same as the address of goloop system info result above.

Prepare import configuration

Create a ./config/import_config.json file like the following.

$ cat ./config/import_config.json 
{
    "store_uri": "/mainnet/.storage/db_icon_dex",
    "config_url": "http://web/migration_config.json"
}

Start import task

$ docker-compose exec node goloop chain import_icon 0x1 @config/import_config.json

You can check the current status of migration with the following command.

$ docker-compose exec node goloop chain ls
[
  {
    "cid": "0x1",
    "nid": "0x1",
    "channel": "icon_dex",
    "state": "import_icon 20735 running",
    "height": 70,
    "lastError": ""
  }
]

Note that the number above, 20735, indicates the current block height of ICON1 data.

Wait until the import task finished

You must wait for the import operation to be completed. If you can see the status as the following, the import task has finished.

$ docker-compose exec node goloop chain ls
[
  {
    "cid": "0x1",
    "nid": "0x1",
    "channel": "icon_dex",
    "state": "import_icon 41730698 finished",
    "height": 1038635,
    "lastError": ""
  }
]

Stop import task

Once you confirm the import task has finished, you need to stop the import task explicitly by issuing the following stop command.

$ docker-compose exec node goloop chain stop 0x1
OK
$ docker-compose exec node goloop chain ls
[
  {
    "cid": "0x1",
    "nid": "0x1",
    "channel": "icon_dex",
    "state": "import_icon stopping",
    "height": 0,
    "lastError": ""
  }
]

Wait a moment until the chain status is changed to finished. It will erase all temporal data generated during the import task.

$ docker-compose exec node goloop chain ls
[
  {
    "cid": "0x1",
    "nid": "0x1",
    "channel": "icon_dex",
    "state": "import_icon finished",
    "height": 0,
    "lastError": ""
  }
]

To see the last block height, you need to issue stop command again.

$ docker-compose exec node goloop chain stop 0x1
OK
$ docker-compose exec node goloop chain ls
[
  {
    "cid": "0x1",
    "nid": "0x1",
    "channel": "icon_dex",
    "state": "stopped",
    "height": 41730696,
    "lastError": ""
  }
]

Now you can see the last block height, 41730696, which is 3 blocks less than the last number of blocks in the finished phase.