---
title: w3 indexing pipeline
---
flowchart TD
s3-dotstorage[(S3 dotstorage\nbucket)] -->|ObjectCreated fa:fa-car| bucket-to-indexer(bucket-to-indexer ƛ)
bucket-to-indexer -->|region/bucket/cid/cid.car| indexer-queue[/indexer queue/]
indexer-queue --> indexer(Indexer ƛ)
indexer --> |zQmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn| multihash-queue[/multihash queue/]
multihash-queue --> ipni-advertisment-content(ipni advertisment content ƛ)
ipni-advertisment-content --> |PUT /advertCid|s3-ipni-advert
ipni-advertisment-content --> |advert CID| ad-queue[/advertisment queue/]
ad-queue --> ipni-publish(ipni publish ƛ)
ipni-publish --> |PUT /head|s3-ipni-advert[(S3 IPNI advert \n bucket)]
ipni-publish --> |POST head|IPNI[["`**IPNI**`"]]
carpark[(S3 carpark\nbucket)] --> |ObjectCreated fa:fa-car|carpark-consumer(carpark-consumer ƛ)
carpark-consumer --> |EventBridge newCar|event-bridge[Event Bridge]
event-bridge --> carpark-eipfs-indexer
carpark-eipfs-indexer --> |region/bucket/cid/cid.car| indexer-queue[/indexer queue/]
indexer ------> dynamo[Dynamo\nblocks index]
event-bridge --> |EventBridge newCar|satnav-write-queue
satnav-write-queue --> satnav-write-handler
satnav-write-handler --> |PUT carCid/carCid.idx|s3-satnav[(S3 satnav\nbucket)]
s3-satnav --> satnav-replicator-queue[/satnav-replicator-queue/]
satnav-replicator-queue --> satnav-replicator-handler
satnav-replicator-handler ---> |PUT carCid/carCid.idx|r2-satnav[(R2 satnav \n bucket)]
api.web3.storage ---------->|PUT carCid/carCid.idx|r2-satnav[(R2 satnav \n bucket)]
Last active
December 4, 2023 15:42
-
-
Save olizilla/9bd936ebc36523184fce8910c00c8133 to your computer and use it in GitHub Desktop.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment