Setup instructions for Hyperledger FireFly and the Rinkeby testnet

_instructions.md

Update June 3rd 2022

These instructions now use the alpha channel of FireFly, so you no longer need to compile from source to use the new features demonstrated in this Gist.

Use Hard Hat to install the FireFly batch-pin contract to Rinkeby

Hard hat configuration/deploy scripts are provided, you just need to run them with a couple of environment variables set to get your contract deployed.

You will need an account on Rinkeby that has some Ether to deploy the contract.

BE EXTREMELY CAREFUL THAT YOU ARE USING THE RINKEBY TESTNET IN METAMASK

I recommend you generate a new private account, and then follow these instructions.

• You can use Metamask to generate your account
• Faucet to get Ether: https://rinkebyfaucet.com/
• Export your account private key as follows, to use with Hard Hat

• Run the following commands to do the deploy

# Contract is in the FireFly Core repo
cd firefly/smart_contracts/ethereum/solidity_firefly
# Install the dependencies - you will need Node.js 16 or later installed
npm install
# Set your INFURA_PROJECT_ID
export INFURA_PROJECT_ID=XXXXXXXXXX
# Set a private key you exported above as a hex string
export RINKEBY_PRIVATE_KEY=XXXXXXXXX
# Run the deploy
npx hardhat --network rinkeby run ./scripts/deploy.ts

• Success looks like this:

FireFly contract deployed to: 0x273891509b2cfC5078daF2C8fA5f317bA0B6b96b

• Go to https://rinkeby.etherscan.io to find what Block Number your contract was deployed at

Run the firefly-rinkeby-init.sh script attached to this Gist

For convenience all the parameters needed for ff init are provided in the attached script.

It does the following:

• Sets the fromBlock in your FireFly Core, so it doesn't try and index the events on Rinkeby from block zero
• Adds a number of useful tweaks to the EthConnect configuration for public networks
  • Note only 3 confirmations are requested in this configuration for events
• Configures ff init with:
  • --enable-fftm to enabled the FireFly Transaction Manager component (FFTM)
  • --channel alpha to use the v1.1 pre-release branch of functionality that contains features needed by FFTM
  • --blockchain-node remote-rpc to specify to connect to a remote JSON/RPC endpoint (Infura)

Must be run from the firefly directory unless you edit -m manifest.json to point at your firefly directory

Note you needed to build FireFly Core with make docker in the earlier step

Example to run:

# Go to the firefly directory
cd firefly
# Set the name of the FireFly stack you want to create
export STACK_NAME=rinkeby
# Set your full Infura URL
export INFURA_URL=https://rinkeby.infura.io/v3/XXXXXXXX
# Set the CONTRACT_ADDRESS your installed via Hard Hat
export CONTRACT_ADDRESS=0xXXXXXXX
# Set your From Block based your inpsection of https://rinkeby.etherscan.io/
export FROM_BLOCK=10598108
# Run the script from where you downloaded it
chmod 755 ~/Downloads/firefly-rinkeby-init.sh
~/Downloads/firefly-rinkeby-init.sh

If your delete and re-create your environment with new keys, you should re-deploy your FireFly contract to Rinkeby

Success looks like:

ff start rb

Your docker compose file for this stack can be found at: /Users/pbroadhurst/.firefly/stacks/rb/init/docker-compose.yml

+ jq -r '.members[].account.address'
+ cat /Users/pbroadhurst/.firefly/stacks/rb/stack.json
0x0b5e48d8c1609ee3f505df00748da42d51c75062

The last line is the address of a newly generated account.

Fund your newly generated FireFly node account

The CLI generates an account for the new organization to register itself with, within the FireFly network.

Note the FireFly community is exploring making this registration optional, when running FireFly is a Gateway mode - where your application is not exploiting the messaging/identity features of FireFly within a multi-party business network. You are just using FireFly to integrate with the digitial assets and smart contrats on existing blockchain(s).

You need to fund this accountm, by transferring Rinkeby ether to it, before you start your node. Otherwise node registration will fail, and you won't be able to initiate transactions/transfers from your FireFly node.

An amount of 0.01 Rinkeby Ether should be more than sufficient to start (you can easily get 0.1 Rinkeby ether per day from the faucet, so only transferring 1/10th of this gives you opportunities to re-do if you get things wrong).

AGAIN BE REALLY SURE YOU ARE USING RINKEBY WHEN YOU USE METAMASK HERE

Start up your FireFly node

You're ready to go. Make sure you have the version of the FireFly CLI that contains the FireFly Transaction Manager support (you did this in the first step if you built with the fftm branch).

I suggest using -b mode to avoid it tearing down the environment if things go wrong, so you can investigate the logs. Assuming you set STACK_NAME to rinkeby:

ff start -v -b rinkeby

Start up will take some time, and you will see it starting and stopping containers as it goes.

The step where things really get interesting is where it says: registering org and node for member 0

This is the point it's submitted transactions to the blockchain, and is waiting for those to confirm, both the submission of the transaction and the listening for receipts.

Explore the logs

While the stack is starting, you can use docker ps to see the containers being started.

Here are the interesting logs to look at:

FireFly Signer

docker logs -f rinkeby_ethsigner

This will be one of the first containers to start, it is where signing happens and all JSON/RPC calls pass through here.

If you can successfully talk to Rinkeby through Infura, you will see successful net_version calls as so - because a docker healtcheck is configured by the CLI that makes these calls every 15s:

[2022-05-01T04:09:57.274Z]  INFO ffsigner: server listening on HTTP [::]:8545
[2022-05-01T04:10:12.050Z]  INFO ffsigner: RPC:"000000001":"1" --> net_version req=45_5J_Ta
[2022-05-01T04:10:12.787Z]  INFO ffsigner: RPC:"000000001":"1" <-- [200] OK req=45_5J_Ta

You will see a host of other logs as the other components start.

FireFly Transaction Manager

Note this container starts and then restarts during the startup sequence, and the first time it starts its config is not ready. So if you see an error trying to access the config, don't worry it's just waiting for the CLI to write the config and restart it.

docker logs -f rinkeby_fftm_0

Some interesting snippets to look out for:

Nonce management

[2022-05-01T04:11:45.363Z] DEBUG fftm_0: Next nonce 0 for signer 0xaf9010a86acbbae01e7be76dfe060402c8e089ff spent req=_gAoe8qt requestId=e8e6826b-856c-442e-929b-a2474f2c4b91
[2022-05-01T04:11:45.363Z] DEBUG fftm_0: Nonce 0 in-flight. Next nonce: 1

Submitting the registration transaction

[2022-05-01T04:11:46.211Z]  INFO fftm_0: Sending transaction: &{RequestBase:{FFCAPI:{RequestID: Version: Variant: RequestType:}} GasPrice:"1052683185" TransactionHeaders:{From:0xaf9010a86acbbae01e7be76dfe060402c8e089ff To:0xb320a75C9A0a9de35Fa8Af60a7C2083bde7dC810 Nonce:0xc0000683c0 Gas:0xc000068360 Value:<nil>} TransactionData:0x3674e15c00000000000000000000000000000000000000000000000000000000000000a0561088b3ce43480a8dcd1398a45865a1ada4bdec1813445b8003a20748b73acc8b098d1d4273c00d5056b2e80ded4aec59855f8e473c421af08b2d03fa49313800000000000000000000000000000000000000000000000000000000000000e00000000000000000000000000000000000000000000000000000000000000140000000000000000000000000000000000000000000000000000000000000000966665f73797374656d0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002e516d555235366b464b31694165544b34756f4431624157374e725670683676587856665565474334654b3151575200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000018319d70666de18ed7d1c17e1bbd8ac0b16716fbe2059a8ae356a09d3bd067e14}

Getting the transaction hash, which you can view in Etherscan

[2022-05-01T04:11:52.973Z]  INFO fftm_0: Transaction hash=0x64997c82ff8d149c61d3f11081a5e7b8480c63ce72dfc3d512812eb10bc46679

Updating the operation in FireFly Core - which you can view in the FireFly Explorer:

[2022-05-01T04:11:52.973Z] DEBUG fftm_0: Updating operation e8e6826b-856c-442e-929b-a2474f2c4b91 status=Pending
[2022-05-01T04:11:52.974Z] DEBUG fftm_0: ==> PUT http://firefly_core_0:5101/admin/api/v1/operations/e8e6826b-856c-442e-929b-a2474f2c4b91 breq=aDy_KZuI
[2022-05-01T04:11:52.993Z] DEBUG fftm_0: <== PUT http://firefly_core_0:5101/admin/api/v1/operations/e8e6826b-856c-442e-929b-a2474f2c4b91 [200] (18.93ms) breq=aDy_KZuI

Monitoring block headers, and finding the transaction made it into a block:

[2022-05-01T04:12:00.449Z] DEBUG fftm_0: New block notifications [0x3890eafe9b63dd374fbae69f698796f90da985fc53ac73fc117d83c7619e336f]
[2022-05-01T04:12:00.564Z] DEBUG fftm_0: Downloaded block header by hash: 10598246 / 0x3890eafe9b63dd374fbae69f698796f90da985fc53ac73fc117d83c7619e336f parent=0x6f96edfde7630ad16c14cb810b264a23a738c9f5515a168c20e15a4053004481
[2022-05-01T04:12:00.566Z]  INFO fftm_0: Detected transaction 0x64997c82ff8d149c61d3f11081a5e7b8480c63ce72dfc3d512812eb10bc46679 added to block 10598246 / 0x3890eafe9b63dd374fbae69f698796f90da985fc53ac73fc117d83c7619e336f - receipt check scheduled
[2022-05-01T04:12:00.699Z]  INFO fftm_0: Receipt for transaction 0x64997c82ff8d149c61d3f11081a5e7b8480c63ce72dfc3d512812eb10bc46679 downloaded. BlockNumber=10598246 BlockHash=0x3890eafe9b63dd374fbae69f698796f90da985fc53ac73fc117d83c7619e336f
[2022-05-01T04:12:00.700Z] DEBUG fftm_0: Waiting for confirmation after block 10598246 event=TX:th=0x64997c82ff8d149c61d3f11081a5e7b8480c63ce72dfc3d512812eb10bc46679

Getting confirmations up to the point it is confirmed as a transaction, and the operation in FireFly Core can be marked complete:

[2022-05-01T04:12:15.358Z]  INFO fftm_0: Confirmation 1 at block 10598247 / 0x9e4f5886af1135de6bdd825a6d5b4698510f2f9b283950a900b62d4a62786b22 item=TX:th=0x64997c82ff8d149c61d3f11081a5e7b8480c63ce72dfc3d512812eb10bc46679
...
[2022-05-01T04:12:30.368Z]  INFO fftm_0: Confirmation 2 at block 10598248 / 0xa8bd900cb065b374163824e1c3199a529bfa7219b149336f904dad9a8a840a85 item=TX:th=0x64997c82ff8d149c61d3f11081a5e7b8480c63ce72dfc3d512812eb10bc46679
...
[2022-05-01T04:12:45.375Z]  INFO fftm_0: Confirmation 3 at block 10598249 / 0x1a39f4a762b7293e02871141434bc2f69c168d3932803ca2057a6a6aca1425e5 item=TX:th=0x64997c82ff8d149c61d3f11081a5e7b8480c63ce72dfc3d512812eb10bc46679
[2022-05-01T04:12:45.375Z]  INFO fftm_0: Confirmed with 3 confirmations event=TX:th=0x64997c82ff8d149c61d3f11081a5e7b8480c63ce72dfc3d512812eb10bc46679
[2022-05-01T04:12:46.137Z] DEBUG fftm_0: Updating operation e8e6826b-856c-442e-929b-a2474f2c4b91 status=Succeeded

Index some tokens

Now you have a running FireFly node, you can ask it to index ERC-20 tokens on the Rinkeby network.

This guide doesn't go into details of deploying ERC-20 tokens, but here is an example of the API call to make to tell FireFly to index a new token:

POST http://localhost:5000/api/v1/namespaces/default/tokens/pools

{
  "connector": "erc20_erc721",
  "name": "rinkeby-dem",
  "type": "fungible",
  "config": {
      "address": "0x5bb4dffbc8dc43dd9d73ff685257ac054d63e171",
      "blockNumber": "10371113"
  }
}

Note the token pool will not show up straight away - this is because FireFly will "broadcast" the decision to index this token pool to its network - using a blockchain backed transaction. Now in this case the network is only our local Laptop, and no information is leaked to the chain (just a hash of data in our private IPFS). But we do need to wait for those Rinkeby transactions to go through before the pool is visible.

This broadcast step is something that the community is looking at making optional.

firefly-rinkeby-init.sh

#!/bin/bash -x

if [[ -z $INFURA_URL ]]; then
   echo "Must set INFURA_URL"
   exit 1
fi
if [[ -z $STACK_NAME ]]; then
   echo "Must set STACK_NAME"
   exit 1
fi
if [[ -z $CONTRACT_ADDRESS ]]; then
   echo "Must set CONTRACT_ADDRESS to previously deployed FireFly contract"
   exit 1
fi
if [[ -z $FROM_BLOCK ]]; then
   echo "Must set FROM_BLOCK to the block where the FireFly contract was deployed"
   exit 1
fi

cat <<EOF > /tmp/core.extraconf.yaml
blockchain:
  ethereum:
    ethconnect:
      fromBlock: $FROM_BLOCK
EOF

cat <<EOF > /tmp/ethc.extraconf.yaml
rest:
  rest-gateway:
    openapi:
      catchupModeBlockGap: 5000
      catchupModePageSize: 5000
      eventPollingIntervalSec: 3
      confirmations:
        enabled: true
        requiredConfirmations: 3
        includeInPayload: true
        pollingIntervalSec: 3
    maxTXWaitTime: 300
    maxInFlight: 100

EOF

set -e

ff init $STACK_NAME \
    --channel alpha \
    -t erc20_erc721 \
    --ethconnect-config /tmp/ethc.extraconf.yaml \
    --core-config /tmp/core.extraconf.yaml \
    --contract-address $CONTRACT_ADDRESS \
    --fftm-enabled \
    --blockchain-provider ethereum \
    --blockchain-node remote-rpc \
    --request-timeout 300 \
    --remote-node-url $INFURA_URL \
    --chain-id 4 \
    1

cat ~/.firefly/stacks/$STACK_NAME/stack.json | jq -r '.members[].account.address'