Start provider and consumer chains, create IBC channel, fork consumer, start light client proxy

ics_light_client_attack.md

Setting things up

Run the ./local-testnet-consumer-fork.sh The script does the following:

• starts the provider and consumer chains
• creates the ~/.hermes/config.toml file with the provider and consumer chains
• creates a channel between provider and consumer
• stops the consumer chain, forks the consumer, restarts the consumer chain, starts the forked consumer chain
• creates the ~/.hermes/config_fork.toml file with the provider and forked consumer

Running misbehaviour

• in one terminal: hermes --config ~/.hermes/config.toml start
• in a different terminal: hermes --config ~/.hermes/config_fork.toml update client --client 07-tendermint-0 --host-chain provider
• observe the output in first terminal
  • you can get the Tx hash from the output of start and retrieve it with curl http://localhost:26658/tx?hash=0x<hash>
• check the client state on provider and verify it is frozen, it should include:

$ hermes query client state --chain provider --client 07-tendermint-0
...
        frozen_height: Some(
            Height {
                revision: 0,
                height: 1,
            },
        ),
	...

Note: with hermes v1.5.x there is a new check added for trusted validator set that causes the misbehavior to not be submitted. This is the check: https://github.com/informalsystems/hermes/blob/5d934f79e192d76ee68a0fb29e1e08df10102e12/crates/relayer/src/light_client/tendermint.rs#L181-L190

This error may be seen on the update client issued as last step above.

2023-06-21T20:38:35.499625Z  WARN ThreadId(40) spawn:chain{chain=provider}:client{client=07-tendermint-0}:connection{connection=connection-0}:channel{channel=channel-0}:worker.client.misbehaviour{client=07-tendermint-0 src_chain=consumer dst_chain=provider}:foreign_client.detect_misbehaviour_and_submit_evidence{update_event=None client=consumer->provider:07-tendermint-0}: misbehaviour checking result: error raised while checking for misbehaviour evidence: failed to check misbehaviour for 07-tendermint-0 at consensus height 0-31: error raised while submitting the misbehaviour evidence: mismatch between the trusted validator set of the update header (FACE4A2CEEF272F3895FD20FF57BEA1D6DE87997289CB101A37D2674DB95D25F) and that of the trusted block that was fetched (0815A5A081D9449707D43AAB16F7DB7E5BCD3C0FC3CCF9E3D4295BBD243EA89F), aborting misbehaviour detection.

It can happen with following sequence of events:

• ICS protocol triggers a client update with header from consumer chain after the fork (relayed by the hermes start instance), let's say is for height t. The consensus state validator set is set to the validator set of height t+1 from the consumer branch
• hermes update client is done with a header form the forked chain at some height u and latest trusted height of t. The update in this case will retrieve the validator set of height t+1 from the fork chain.

The hashes may be different so the misbehaviour cannot be submitted. We need to investigate this, understand why they are different. Also what is the impact on correctness for current hermes misbehaviour submission (as done by client worker).

To go around this specify the --trused-height of a consensus state before fork:

hermes --config ~/.hermes/config_fork.toml update client --client 07-tendermint-0 --host-chain provider --trusted-height 5

Consensus states can be listed with:

$ hermes query client consensus --chain provider --client 07-tendermint-0

Pick one of the first 3 listed in the output of the command. Once we have the evidence CLI ready the update client command should not be needed in the steps described above.

local-testnet-consumer-fork-1.4.sh

#!/bin/bash
set -eux 


diag() {
  echo ">>
>> $@
>>" 1>&2
}

# User balance of stake tokens 
USER_COINS="100000000000stake"
# Amount of stake tokens staked
STAKE="100000000stake"
# Node IP address
NODE_IP="127.0.0.1"

# Home directory
HOME_DIR=$HOME

# Validator moniker
MONIKER="coordinator"

# Validator directory
PROV_NODE_DIR=${HOME_DIR}/provider-${MONIKER}
CONS_NODE_DIR=${HOME_DIR}/consumer-${MONIKER}
CONS_FORK_NODE_DIR=${HOME_DIR}/consumer-fork-${MONIKER}

# Coordinator key
PROV_KEY=${MONIKER}-key


# Clean start
pkill -f interchain-security-pd &> /dev/null || true
rm -rf ${PROV_NODE_DIR}

# Build genesis file and node directory structure
interchain-security-pd init $MONIKER --chain-id provider --home ${PROV_NODE_DIR}
jq ".app_state.gov.voting_params.voting_period = \"3s\" | .app_state.staking.params.unbonding_time = \"86400s\"" \
   ${PROV_NODE_DIR}/config/genesis.json > \
   ${PROV_NODE_DIR}/edited_genesis.json && mv ${PROV_NODE_DIR}/edited_genesis.json ${PROV_NODE_DIR}/config/genesis.json

sleep 1

# Create account keypair
interchain-security-pd keys add $PROV_KEY --home ${PROV_NODE_DIR} --keyring-backend test --output json > ${PROV_NODE_DIR}/${PROV_KEY}.json 2>&1
sleep 1

# Add stake to user
PROV_ACCOUNT_ADDR=$(jq -r '.address' ${PROV_NODE_DIR}/${PROV_KEY}.json)
interchain-security-pd add-genesis-account $PROV_ACCOUNT_ADDR $USER_COINS --home ${PROV_NODE_DIR} --keyring-backend test
sleep 1


# Stake 1/1000 user's coins
interchain-security-pd gentx $PROV_KEY $STAKE --chain-id provider --home ${PROV_NODE_DIR} --keyring-backend test --moniker $MONIKER
sleep 1

interchain-security-pd collect-gentxs --home ${PROV_NODE_DIR} --gentx-dir ${PROV_NODE_DIR}/config/gentx/
sleep 1

sed -i -r "/node =/ s/= .*/= \"tcp:\/\/${NODE_IP}:26658\"/" ${PROV_NODE_DIR}/config/client.toml
sed -i -r 's/timeout_commit = "5s"/timeout_commit = "3s"/g' ${PROV_NODE_DIR}/config/config.toml
sed -i -r 's/timeout_propose = "3s"/timeout_propose = "1s"/g' ${PROV_NODE_DIR}/config/config.toml


# Start gaia
interchain-security-pd start \
    --home ${PROV_NODE_DIR} \
    --rpc.laddr tcp://${NODE_IP}:26658 \
    --grpc.address ${NODE_IP}:9091 \
    --address tcp://${NODE_IP}:26655 \
    --p2p.laddr tcp://${NODE_IP}:26656 \
    --grpc-web.enable=false &> ${PROV_NODE_DIR}/logs &

sleep 5

# Build consumer chain proposal file
tee ${PROV_NODE_DIR}/consumer-proposal.json<<EOF
{
    "title": "Create a chain",
    "description": "Gonna be a great chain",
    "chain_id": "consumer",
    "initial_height": {
        "revision_height": 1
    },
    "genesis_hash": "Z2VuX2hhc2g=",
    "binary_hash": "YmluX2hhc2g=",
    "spawn_time": "2022-03-11T09:02:14.718477-08:00",
    "deposit": "10000001stake",
    "consumer_redistribution_fraction": "0.75",
    "historical_entries": 1000,
    "blocks_per_distribution_transmission": 1000,
    "unbonding_period":  1728000000000000,
    "ccv_timeout_period": 2419200000000000,
    "transfer_timeout_period": 3600000000000
}
EOF

interchain-security-pd keys show $PROV_KEY --keyring-backend test --home ${PROV_NODE_DIR}

# Submit consumer chain proposal
interchain-security-pd tx gov submit-proposal consumer-addition ${PROV_NODE_DIR}/consumer-proposal.json --chain-id provider --from $PROV_KEY --home ${PROV_NODE_DIR} --node tcp://${NODE_IP}:26658  --keyring-backend test -b block -y

sleep 1

# Vote yes to proposal
interchain-security-pd tx gov vote 1 yes --from $PROV_KEY --chain-id provider --home ${PROV_NODE_DIR} -b block -y --keyring-backend test

sleep 3

## CONSUMER CHAIN ##

# Clean start
pkill -f interchain-security-cd &> /dev/null || true
rm -rf ${CONS_NODE_DIR}
rm -rf ${CONS_FORK_NODE_DIR}

# Build genesis file and node directory structure
interchain-security-cd init $MONIKER --chain-id consumer  --home  ${CONS_NODE_DIR}
sleep 1

# Create user account keypair
interchain-security-cd keys add $PROV_KEY --home  ${CONS_NODE_DIR} --keyring-backend test --output json > ${CONS_NODE_DIR}/${PROV_KEY}.json 2>&1

# Add stake to user account
CONS_ACCOUNT_ADDR=$(jq -r '.address' ${CONS_NODE_DIR}/${PROV_KEY}.json)
interchain-security-cd add-genesis-account $CONS_ACCOUNT_ADDR 1000000000stake --home ${CONS_NODE_DIR}

# Add consumer genesis states to genesis file
interchain-security-pd query provider consumer-genesis consumer --home ${PROV_NODE_DIR} -o json > consumer_gen.json
jq -s '.[0].app_state.ccvconsumer = .[1] | .[0]' ${CONS_NODE_DIR}/config/genesis.json consumer_gen.json > ${CONS_NODE_DIR}/edited_genesis.json \
&& mv ${CONS_NODE_DIR}/edited_genesis.json ${CONS_NODE_DIR}/config/genesis.json
rm consumer_gen.json

# Create validator states
echo '{"height": "0","round": 0,"step": 0}' > ${CONS_NODE_DIR}/data/priv_validator_state.json

# Copy validator key files
cp ${PROV_NODE_DIR}/config/priv_validator_key.json ${CONS_NODE_DIR}/config/priv_validator_key.json
cp ${PROV_NODE_DIR}/config/node_key.json ${CONS_NODE_DIR}/config/node_key.json

# Set default client port
sed -i -r "/node =/ s/= .*/= \"tcp:\/\/${NODE_IP}:26648\"/" ${CONS_NODE_DIR}/config/client.toml

# Start gaia
interchain-security-cd start --home  ${CONS_NODE_DIR} \
        --rpc.laddr tcp://${NODE_IP}:26648 \
        --grpc.address ${NODE_IP}:9081 \
        --address tcp://${NODE_IP}:26645 \
        --p2p.laddr tcp://${NODE_IP}:26646 \
        --grpc-web.enable=false \
        &> ${CONS_NODE_DIR}/logs &

sleep 3

# Setup Hermes in packet relayer mode
pkill -f hermes 2> /dev/null || true

tee ~/.hermes/config.toml<<EOF
[global]
log_level = "debug"

[mode]

[mode.clients]
enabled = true
refresh = true
misbehaviour = true

[mode.connections]
enabled = false

[mode.channels]
enabled = false

[mode.packets]
enabled = true

[[chains]]
id = "consumer"
ccv_consumer_chain = true
account_prefix = "cosmos"
clock_drift = "5s"
gas_multiplier = 1.1
grpc_addr = "tcp://${NODE_IP}:9081"
key_name = "relayer"
max_gas = 2000000
rpc_addr = "http://${NODE_IP}:26648"
rpc_timeout = "10s"
store_prefix = "ibc"
trusting_period = "2days"
websocket_addr = "ws://${NODE_IP}:26648/websocket"

[chains.gas_price]
       denom = "stake"
       price = 0.00

[chains.trust_threshold]
       denominator = "3"
       numerator = "1"

[[chains]]
id = "provider"
account_prefix = "cosmos"
clock_drift = "5s"
gas_multiplier = 1.1
grpc_addr = "tcp://${NODE_IP}:9091"
key_name = "relayer"
max_gas = 2000000
rpc_addr = "http://${NODE_IP}:26658"
rpc_timeout = "10s"
store_prefix = "ibc"
trusting_period = "2days"
websocket_addr = "ws://${NODE_IP}:26658/websocket"

[chains.gas_price]
       denom = "stake"
       price = 0.00

[chains.trust_threshold]
       denominator = "3"
       numerator = "1"
EOF

# Delete all previous keys in relayer
hermes keys delete --chain consumer --all
hermes keys delete --chain provider --all

# Restore keys to hermes relayer
hermes keys add --key-file  ${CONS_NODE_DIR}/${PROV_KEY}.json --chain consumer
hermes keys add --key-file  ${PROV_NODE_DIR}/${PROV_KEY}.json --chain provider


sleep 5

hermes create connection \
     --a-chain consumer \
    --a-client 07-tendermint-0 \
    --b-client 07-tendermint-0

hermes create channel \
    --a-chain consumer \
    --a-port consumer \
    --b-port provider \
    --order ordered \
    --channel-version 1 \
    --a-connection connection-0

sleep 5

hermes --json start &> ~/.hermes/logs &

interchain-security-pd q tendermint-validator-set --home ${PROV_NODE_DIR}
interchain-security-cd q tendermint-validator-set --home ${CONS_NODE_DIR}

DELEGATIONS=$(interchain-security-pd q staking delegations $PROV_ACCOUNT_ADDR --home ${PROV_NODE_DIR} -o json)

OPERATOR_ADDR=$(echo $DELEGATIONS | jq -r '.delegation_responses[0].delegation.validator_address')

interchain-security-pd tx staking delegate $OPERATOR_ADDR 1000000stake \
       	--from $PROV_KEY \
       	--keyring-backend test \
       	--home ${PROV_NODE_DIR} \
       	--chain-id provider \
	-y -b block

sleep 13

interchain-security-pd q tendermint-validator-set --home ${PROV_NODE_DIR}
interchain-security-cd q tendermint-validator-set --home ${CONS_NODE_DIR}

##### Fork consumer

tee ~/.hermes/config_fork.toml<<EOF
[global]
log_level = "debug"

[mode]

[mode.clients]
enabled = true
refresh = true
misbehaviour = true

[mode.connections]
enabled = false

[mode.channels]
enabled = false

[mode.packets]
enabled = true

[[chains]]
id = "consumer"
ccv_consumer_chain = true
account_prefix = "cosmos"
clock_drift = "5s"
gas_multiplier = 1.1
grpc_addr = "tcp://${NODE_IP}:9071"
key_name = "relayer"
max_gas = 2000000
rpc_addr = "http://${NODE_IP}:26638"
rpc_timeout = "10s"
store_prefix = "ibc"
trusting_period = "2days"
websocket_addr = "ws://${NODE_IP}:26638/websocket"

[chains.gas_price]
       denom = "stake"
       price = 0.00

[chains.trust_threshold]
       denominator = "3"
       numerator = "1"

[[chains]]
id = "provider"
account_prefix = "cosmos"
clock_drift = "5s"
gas_multiplier = 1.1
grpc_addr = "tcp://${NODE_IP}:9091"
key_name = "relayer"
max_gas = 2000000
rpc_addr = "http://${NODE_IP}:26658"
rpc_timeout = "10s"
store_prefix = "ibc"
trusting_period = "2days"
websocket_addr = "ws://${NODE_IP}:26658/websocket"

[chains.gas_price]
       denom = "stake"
       price = 0.00

[chains.trust_threshold]
       denominator = "3"
       numerator = "1"
EOF

rm -rf ~/.cometbft-light/
read -r height hash < <(
	curl -s "localhost:26648"/commit \
		| jq -r '(.result//.).signed_header.header.height + " " + (.result//.).signed_header.commit.block_id.hash')
diag "Height: ${height}, Hash: ${hash}"

sleep 10
cp -r ${CONS_NODE_DIR} ${CONS_FORK_NODE_DIR}
# Set default client port
sed -i -r "/node =/ s/= .*/= \"tcp:\/\/${NODE_IP}:26638\"/" ${CONS_FORK_NODE_DIR}/config/client.toml

# Start gaia
interchain-security-cd start --home  ${CONS_FORK_NODE_DIR} \
        --rpc.laddr tcp://${NODE_IP}:26638 \
        --grpc.address ${NODE_IP}:9071 \
        --address tcp://${NODE_IP}:26635 \
        --p2p.laddr tcp://${NODE_IP}:26636 \
        --grpc-web.enable=false \
        &> ${CONS_FORK_NODE_DIR}/logs &

sleep 3

local-testnet-consumer-fork-1.5.sh

#!/bin/bash
set -eux 


diag() {
  echo ">>
>> $@
>>" 1>&2
}

# User balance of stake tokens 
USER_COINS="100000000000stake"
# Amount of stake tokens staked
STAKE="100000000stake"
# Node IP address
NODE_IP="127.0.0.1"

# Home directory
HOME_DIR=$HOME

# Validator moniker
MONIKER="coordinator"

# Validator directory
PROV_NODE_DIR=${HOME_DIR}/provider-${MONIKER}
CONS_NODE_DIR=${HOME_DIR}/consumer-${MONIKER}
CONS_FORK_NODE_DIR=${HOME_DIR}/consumer-fork-${MONIKER}

# Coordinator key
PROV_KEY=${MONIKER}-key


# Clean start
pkill -f interchain-security-pd &> /dev/null || true
rm -rf ${PROV_NODE_DIR}

# Build genesis file and node directory structure
interchain-security-pd init $MONIKER --chain-id provider --home ${PROV_NODE_DIR}
jq ".app_state.gov.voting_params.voting_period = \"3s\" | .app_state.staking.params.unbonding_time = \"86400s\"" \
   ${PROV_NODE_DIR}/config/genesis.json > \
   ${PROV_NODE_DIR}/edited_genesis.json && mv ${PROV_NODE_DIR}/edited_genesis.json ${PROV_NODE_DIR}/config/genesis.json

sleep 1

# Create account keypair
interchain-security-pd keys add $PROV_KEY --home ${PROV_NODE_DIR} --keyring-backend test --output json > ${PROV_NODE_DIR}/${PROV_KEY}.json 2>&1
sleep 1

# Add stake to user
PROV_ACCOUNT_ADDR=$(jq -r '.address' ${PROV_NODE_DIR}/${PROV_KEY}.json)
interchain-security-pd add-genesis-account $PROV_ACCOUNT_ADDR $USER_COINS --home ${PROV_NODE_DIR} --keyring-backend test
sleep 1


# Stake 1/1000 user's coins
interchain-security-pd gentx $PROV_KEY $STAKE --chain-id provider --home ${PROV_NODE_DIR} --keyring-backend test --moniker $MONIKER
sleep 1

interchain-security-pd collect-gentxs --home ${PROV_NODE_DIR} --gentx-dir ${PROV_NODE_DIR}/config/gentx/
sleep 1

sed -i -r "/node =/ s/= .*/= \"tcp:\/\/${NODE_IP}:26658\"/" ${PROV_NODE_DIR}/config/client.toml
sed -i -r 's/timeout_commit = "5s"/timeout_commit = "3s"/g' ${PROV_NODE_DIR}/config/config.toml
sed -i -r 's/timeout_propose = "3s"/timeout_propose = "1s"/g' ${PROV_NODE_DIR}/config/config.toml


# Start gaia
interchain-security-pd start \
    --home ${PROV_NODE_DIR} \
    --rpc.laddr tcp://${NODE_IP}:26658 \
    --grpc.address ${NODE_IP}:9091 \
    --address tcp://${NODE_IP}:26655 \
    --p2p.laddr tcp://${NODE_IP}:26656 \
    --grpc-web.enable=false &> ${PROV_NODE_DIR}/logs &

sleep 5

# Build consumer chain proposal file
tee ${PROV_NODE_DIR}/consumer-proposal.json<<EOF
{
    "title": "Create a chain",
    "description": "Gonna be a great chain",
    "chain_id": "consumer",
    "initial_height": {
        "revision_height": 1
    },
    "genesis_hash": "Z2VuX2hhc2g=",
    "binary_hash": "YmluX2hhc2g=",
    "spawn_time": "2022-03-11T09:02:14.718477-08:00",
    "deposit": "10000001stake",
    "consumer_redistribution_fraction": "0.75",
    "historical_entries": 1000,
    "blocks_per_distribution_transmission": 1000,
    "unbonding_period":  1728000000000000,
    "ccv_timeout_period": 2419200000000000,
    "transfer_timeout_period": 3600000000000
}
EOF

interchain-security-pd keys show $PROV_KEY --keyring-backend test --home ${PROV_NODE_DIR}

# Submit consumer chain proposal
interchain-security-pd tx gov submit-proposal consumer-addition ${PROV_NODE_DIR}/consumer-proposal.json --chain-id provider --from $PROV_KEY --home ${PROV_NODE_DIR} --node tcp://${NODE_IP}:26658  --keyring-backend test -b block -y

sleep 1

# Vote yes to proposal
interchain-security-pd tx gov vote 1 yes --from $PROV_KEY --chain-id provider --home ${PROV_NODE_DIR} -b block -y --keyring-backend test

sleep 3

## CONSUMER CHAIN ##

# Clean start
pkill -f interchain-security-cd &> /dev/null || true
rm -rf ${CONS_NODE_DIR}
rm -rf ${CONS_FORK_NODE_DIR}

# Build genesis file and node directory structure
interchain-security-cd init $MONIKER --chain-id consumer  --home  ${CONS_NODE_DIR}
sleep 1

# Create user account keypair
interchain-security-cd keys add $PROV_KEY --home  ${CONS_NODE_DIR} --keyring-backend test --output json > ${CONS_NODE_DIR}/${PROV_KEY}.json 2>&1

# Add stake to user account
CONS_ACCOUNT_ADDR=$(jq -r '.address' ${CONS_NODE_DIR}/${PROV_KEY}.json)
interchain-security-cd add-genesis-account $CONS_ACCOUNT_ADDR 1000000000stake --home ${CONS_NODE_DIR}

# Add consumer genesis states to genesis file
interchain-security-pd query provider consumer-genesis consumer --home ${PROV_NODE_DIR} -o json > consumer_gen.json
jq -s '.[0].app_state.ccvconsumer = .[1] | .[0]' ${CONS_NODE_DIR}/config/genesis.json consumer_gen.json > ${CONS_NODE_DIR}/edited_genesis.json \
&& mv ${CONS_NODE_DIR}/edited_genesis.json ${CONS_NODE_DIR}/config/genesis.json
rm consumer_gen.json

# Create validator states
echo '{"height": "0","round": 0,"step": 0}' > ${CONS_NODE_DIR}/data/priv_validator_state.json

# Copy validator key files
cp ${PROV_NODE_DIR}/config/priv_validator_key.json ${CONS_NODE_DIR}/config/priv_validator_key.json
cp ${PROV_NODE_DIR}/config/node_key.json ${CONS_NODE_DIR}/config/node_key.json

# Set default client port
sed -i -r "/node =/ s/= .*/= \"tcp:\/\/${NODE_IP}:26648\"/" ${CONS_NODE_DIR}/config/client.toml

# Start gaia
interchain-security-cd start --home  ${CONS_NODE_DIR} \
        --rpc.laddr tcp://${NODE_IP}:26648 \
        --grpc.address ${NODE_IP}:9081 \
        --address tcp://${NODE_IP}:26645 \
        --p2p.laddr tcp://${NODE_IP}:26646 \
        --grpc-web.enable=false \
        &> ${CONS_NODE_DIR}/logs &

sleep 3

# Setup Hermes in packet relayer mode
pkill -f hermes 2> /dev/null || true

tee ~/.hermes/config.toml<<EOF
[global]
log_level = "debug"
[mode]
[mode.clients]
enabled = true
refresh = true
misbehaviour = true
[mode.connections]
enabled = false
[mode.channels]
enabled = false
[mode.packets]
enabled = true
[[chains]]
id = "consumer"
ccv_consumer_chain = true
account_prefix = "cosmos"
clock_drift = "5s"
gas_multiplier = 1.1
grpc_addr = "tcp://${NODE_IP}:9081"
key_name = "relayer"
max_gas = 2000000
rpc_addr = "http://${NODE_IP}:26648"
rpc_timeout = "10s"
store_prefix = "ibc"
trusting_period = "2days"
event_source = { mode = 'push', url = 'ws://${NODE_IP}:26648/websocket' , batch_delay = '50ms' }
[chains.gas_price]
       denom = "stake"
       price = 0.00
[chains.trust_threshold]
       denominator = "3"
       numerator = "1"
[[chains]]
id = "provider"
account_prefix = "cosmos"
clock_drift = "5s"
gas_multiplier = 1.1
grpc_addr = "tcp://${NODE_IP}:9091"
key_name = "relayer"
max_gas = 2000000
rpc_addr = "http://${NODE_IP}:26658"
rpc_timeout = "10s"
store_prefix = "ibc"
trusting_period = "2days"
event_source = { mode = 'push', url = 'ws://${NODE_IP}:26658/websocket' , batch_delay = '50ms' }
[chains.gas_price]
       denom = "stake"
       price = 0.00
[chains.trust_threshold]
       denominator = "3"
       numerator = "1"
EOF

# Delete all previous keys in relayer
hermes keys delete --chain consumer --all
hermes keys delete --chain provider --all

# Restore keys to hermes relayer
hermes keys add --key-file  ${CONS_NODE_DIR}/${PROV_KEY}.json --chain consumer
hermes keys add --key-file  ${PROV_NODE_DIR}/${PROV_KEY}.json --chain provider


sleep 5

hermes create connection \
     --a-chain consumer \
    --a-client 07-tendermint-0 \
    --b-client 07-tendermint-0

hermes create channel \
    --a-chain consumer \
    --a-port consumer \
    --b-port provider \
    --order ordered \
    --channel-version 1 \
    --a-connection connection-0

sleep 5

hermes --json start &> ~/.hermes/logs &

interchain-security-pd q tendermint-validator-set --home ${PROV_NODE_DIR}
interchain-security-cd q tendermint-validator-set --home ${CONS_NODE_DIR}

DELEGATIONS=$(interchain-security-pd q staking delegations $PROV_ACCOUNT_ADDR --home ${PROV_NODE_DIR} -o json)

OPERATOR_ADDR=$(echo $DELEGATIONS | jq -r '.delegation_responses[0].delegation.validator_address')

interchain-security-pd tx staking delegate $OPERATOR_ADDR 1000000stake \
       	--from $PROV_KEY \
       	--keyring-backend test \
       	--home ${PROV_NODE_DIR} \
       	--chain-id provider \
	-y -b block

sleep 13

interchain-security-pd q tendermint-validator-set --home ${PROV_NODE_DIR}
interchain-security-cd q tendermint-validator-set --home ${CONS_NODE_DIR}


# sleep 5

# tee ${PROV_NODE_DIR}/stop-consumer-proposal.json<<EOF
# {
#     "title": "Stop the consumer",
#     "description": "It was a great chain",
#     "chain_id": "consumer",
#     "stop_time": "2022-01-27T15:59:50.121607-08:00",
#     "deposit": "100000001stake"
# }
# EOF

# # sleep 1

# interchain-security-pd tx gov submit-proposal stop-consumer-chain \
# ${PROV_NODE_DIR}/stop-consumer-proposal.json \
#                             --chain-id provider \
#                             --from $PROV_KEY \
#                             --home ${PROV_NODE_DIR} \
#                             --keyring-backend test \
#                             -b block -y

# # sleep 1

# interchain-security-pd tx gov vote 2 yes \
#                          --from $PROV_KEY \
#                          --keyring-backend test \
#                          --chain-id provider \
#                          --home $PROV_NODE_DIR \
#                          -b block -y



##### Fork consumer

tee ~/.hermes/config_fork.toml<<EOF
[global]
log_level = "info"
[mode]
[mode.clients]
enabled = true
refresh = true
misbehaviour = true
[mode.connections]
enabled = false
[mode.channels]
enabled = false
[mode.packets]
enabled = true
[[chains]]
id = "consumer"
ccv_consumer_chain = true
account_prefix = "cosmos"
clock_drift = "5s"
gas_multiplier = 1.1
grpc_addr = "tcp://${NODE_IP}:9071"
key_name = "relayer"
max_gas = 2000000
rpc_addr = "http://${NODE_IP}:26638"
rpc_timeout = "10s"
store_prefix = "ibc"
trusting_period = "2days"
event_source = { mode = 'push', url = 'ws://${NODE_IP}:26638/websocket' , batch_delay = '50ms' }
[chains.gas_price]
       denom = "stake"
       price = 0.00
[chains.trust_threshold]
       denominator = "3"
       numerator = "1"
[[chains]]
id = "provider"
account_prefix = "cosmos"
clock_drift = "5s"
gas_multiplier = 1.1
grpc_addr = "tcp://${NODE_IP}:9091"
key_name = "relayer"
max_gas = 2000000
rpc_addr = "http://${NODE_IP}:26658"
rpc_timeout = "10s"
store_prefix = "ibc"
trusting_period = "2days"
event_source = { mode = 'push', url = 'ws://${NODE_IP}:26658/websocket' , batch_delay = '50ms' }
[chains.gas_price]
       denom = "stake"
       price = 0.00
[chains.trust_threshold]
       denominator = "3"
       numerator = "1"
EOF

rm -rf ~/.cometbft-light/
read -r height hash < <(
	curl -s "localhost:26648"/commit \
		| jq -r '(.result//.).signed_header.header.height + " " + (.result//.).signed_header.commit.block_id.hash')
diag "Height: ${height}, Hash: ${hash}"

sleep 10
cp -r ${CONS_NODE_DIR} ${CONS_FORK_NODE_DIR}
# Set default client port
sed -i -r "/node =/ s/= .*/= \"tcp:\/\/${NODE_IP}:26638\"/" ${CONS_FORK_NODE_DIR}/config/client.toml

# Start gaia
interchain-security-cd start --home  ${CONS_FORK_NODE_DIR} \
        --rpc.laddr tcp://${NODE_IP}:26638 \
        --grpc.address ${NODE_IP}:9071 \
        --address tcp://${NODE_IP}:26635 \
        --p2p.laddr tcp://${NODE_IP}:26636 \
        --grpc-web.enable=false \
        &> ${CONS_FORK_NODE_DIR}/logs &

sleep 3