w3s-to-ipni-pipeline.md

---
title: w3 indexing pipeline
---
flowchart TD
    s3-dotstorage[(S3 dotstorage\nbucket)] -->|ObjectCreated fa:fa-car| bucket-to-indexer(bucket-to-indexer ƛ)
    bucket-to-indexer -->|region/bucket/cid/cid.car| indexer-queue[/indexer queue/]
    indexer-queue --> indexer(Indexer ƛ)
    indexer --> |zQmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn| multihash-queue[/multihash queue/]
    multihash-queue --> ipni-advertisment-content(ipni advertisment content ƛ)
    ipni-advertisment-content --> |PUT /advertCid|s3-ipni-advert
    ipni-advertisment-content --> |advert CID| ad-queue[/advertisment queue/]
    ad-queue --> ipni-publish(ipni publish ƛ)
    ipni-publish --> |PUT /head|s3-ipni-advert[(S3 IPNI advert \n bucket)]
    ipni-publish --> |POST head|IPNI[["`**IPNI**`"]]
    
    carpark[(S3 carpark\nbucket)] --> |ObjectCreated fa:fa-car|carpark-consumer(carpark-consumer ƛ)
    carpark-consumer --> |EventBridge newCar|event-bridge[Event Bridge]
    event-bridge --> carpark-eipfs-indexer
    carpark-eipfs-indexer --> |region/bucket/cid/cid.car| indexer-queue[/indexer queue/]

    indexer ------> dynamo[Dynamo\nblocks index]

    event-bridge --> |EventBridge newCar|satnav-write-queue
    satnav-write-queue --> satnav-write-handler
    satnav-write-handler --> |PUT carCid/carCid.idx|s3-satnav[(S3 satnav\nbucket)]
    s3-satnav --> satnav-replicator-queue[/satnav-replicator-queue/]
    satnav-replicator-queue --> satnav-replicator-handler
    satnav-replicator-handler ---> |PUT carCid/carCid.idx|r2-satnav[(R2 satnav \n bucket)]
    
    api.web3.storage ---------->|PUT carCid/carCid.idx|r2-satnav[(R2 satnav \n bucket)]