Update on lotus-backed block explorer

lotus-explorer.md

We have a working version of the block explorer that uses lotus instead of go-filecoin to collect blocks.

It will be fully functional after we answer the following questions:

1. How do we use the GetTipSetByHeight jsonrpc method? The method takes a height, and a TipSet - which to us is a bit confusing. Why would we pass a tipset to the method that returns a tipset at the height we're looking for? We tried to manually construct this request with a known (hardcoded) tipset, with no luck.

We would use this functionality in our lotus block explorer to make sure that we see all blocks. Right now, comparing with the lotus dashboard shows we're missing a couple blocks. More on the polling strategy here.

This would also enable us to provide web3.js esque to fields for the explorer. Right now we have to start from the ChainHead because it is the only tipset we are able to fetch. This makes it more difficult to build database-less pagination (which would be quite simple if we can fetch tipsets by height).

Lastly, this will help us more easily calculate the weight of each tipset, and

2. How should we handle preflight CORS requests from the browser? Is that on the devops team to hide Lotus behind a proxy and explicitly handle preflight requests? Or should lotus' jsonrpc explicitly handle them?

When sending rpc (post) requests directly to Lotus from the browser, the browser automatically sends a preflight options request (see MDN for more info). This happens because we set the application/json header, which constitutes a "complex" browser request. When an options request is fired to Lotus, Lotus expects a jsonrpc method (which you can't include in an options request) and returns a 500 error. No subsequent request is able to be made after the preflight requests comes back with an error status code.

This keeps us from fetching live information from Lotus (without running our own proxy server).

Smaller less important questions below:

3. How close do we want the go-filecoin block explorer UI to fit the lotus block explorer UI? There are a couple pieces of information that are harder to grab:

a. The signature of each message

When we request ChainGetBlockMessages we do not get back any signatures. We could dig through the lotus code to find out how to fetch the signatures if this is important.

b. The message receipts table

Constructing the message receipts table should be easy, but we can't seem to get the ChainGetMessageReceipts jsonrpc method working right. It takes in a cid just like all the other methods we're using, but when we construct and send the request, we get back the error:

{"jsonrpc":"2.0","result":null,"id":1,"error":{"code":1,"message":"cbor input had wrong number of fields"}}

c. Handling "tickets"

There was a section in the go-filecoin block explorer called "ticket", which wasn't working. I fixed this by passing in the VRFProof supplied in the block headers. Not sure if the VRFProof represents the right piece of information we want to show there?

d. Handling "child" blocks

There's a small button in the go-filecoin explorer that allows the user to visit the block's children. This is a feature we could definitely build, but it might take some time (and it's very computationally expensive), so wanted to check and make sure it was important. This will be way easier to build with a lotus block postgres database.

e. Calculating block "power" (as seen in UI)

What's the most efficient way? The block itself only comes back with a parentWeight field. Could we use the TipSetWeight? Possibly in conjunction with the parentWeight? We might be misunderstanding how power is calculated in practice too.

Block polling strategy

https://github.com/openworklabs/lotus-explorer

Currently we have no way to fetch tipsets directly, which we think causes us to miss blocks. This is the implemented strategy:

1. Get ChainHead
2. Iterate through ChainHead blocks (do not cache these blocks, since the tipset height we're looking at is n and we need to start at n-1) (maybe even n-5 to be safe?)
3. For each parent, get the parentBlock
4. get the messages of each parentBlock
5. Add parentBlock to cache
6. Mark parentBlock hash as "seen", so we don't visit it again (avoids any unexpected circular dependencies?)
7. Repeat step 3 for each of parentBlocks parents until we reach the desired endpoint

Not 100% confident yet, but it makes sense that this strategy misses blocks - blocks could get added to a tipset that were unreachable from the ChainHead's direct descendents and ancestors. The more complete strategy will be to:

1. Get ChainHead
2. Iterate through ChainHead blocks (do not cache these blocks, since the tipset height we're looking at is n and we need to start at n-1) (maybe even n-5 to be safe?)
3. For each parent, get the parentBlock
4. get the messages of each parentBlock
5. Add parentBlock to cache
6. Mark parentBlock hash as "seen", so we don't visit it again (avoids any unexpected circular dependencies?)
7. Repeat step 3 for each of parentBlocks parents and sibling tipset blocks until we reach the desired endpoint

Note - the deployed Lotus dashboard seems to also be missing some blocks in its tipsets (unless we're misunderstanding, miscalculating). We can provide some examples and steps to replication.