Skip to content

Instantly share code, notes, and snippets.

Avatar
👀
https://web3.storage

Alan Shaw alanshaw

👀
https://web3.storage
View GitHub Profile
@alanshaw
alanshaw / upload-large-cars.md
Last active January 11, 2023 18:29
Upload large CAR files to w3up
View upload-large-cars.md

Install the new CLI:

npm install -g @web3-storage/w3cli

Create and register new space:

w3 space create SPACENAME
@alanshaw
alanshaw / w3-up-server.md
Last active January 11, 2023 15:59
How do you use w3up on the server/in CI where there's no persistent filesystem?
View w3-up-server.md

On your local computer:

  1. Create a keypair for the server

    npx ucan-key ed --json

    Note down did and key values!

@alanshaw
alanshaw / agent.js
Last active December 1, 2022 11:26
w3access agent refactor
View agent.js
// Goals:
// 1. Decouple store from agent
// 2. Simplify agent creation
// 3. Agent governs data format not store
// 4. Initialization of agent, not store
// 5. DRY initialization in agent, not repeated in each store impl
/**
* @param {AgentData} [data] Agent data
* @param {object} [options]
@alanshaw
alanshaw / extract-gateway-peers.mjs
Created November 30, 2022 12:38
Extract the gateway peer IDs from the bifrost infra repo
View extract-gateway-peers.mjs
import fs from 'fs'
import path from 'path'
const dir = path.join('ansible', 'inventories', 'bifrost', 'host_vars')
const regex = /ipfs_config_identity_peerid: "(.*)"/ig
async function main () {
const peers = []
for (const f of fs.readdirSync(dir)) {
if (!(f.startsWith('ipfs-bank') && f.endsWith('ipfs.gateway.dwebops.net.yml'))) continue
@alanshaw
alanshaw / react-uploader.jsx
Last active September 28, 2022 20:36
Simplified examples for the W3UI website
View react-uploader.jsx
import { useState } from 'react'
import { useUploader } from '@w3ui/react-uploader'
export default function Component () {
const { uploader } = useUploader()
const [file, setFile] = useState(null)
const [cid, setCid] = useState('')
const handleUploadSubmit = async e => {
e.preventDefault()
@alanshaw
alanshaw / linkdex.md
Created August 24, 2022 07:50
Linkdex idea
View linkdex.md

Linkdex

An index mapping block CID to linked block CID.

Characteristics

  • Small - just the link blocks - no data for raw leaves. The majority of files are small files, so the DAGs are not super deep.
  • Root CID independent - it doesn’t matter what the root CID is, we can just iteratively decode each block in a CAR and index the links it has

What is it useful for?

@alanshaw
alanshaw / sns2s3.js
Created June 10, 2022 13:44
SNSEvent to S3Event
View sns2s3.js
/**
* Extract an S3Event from the passed SNSEvent.
* @param {import('aws-lambda').SNSEvent} snsEvent
* @returns {import('aws-lambda').S3Event}
*/
function toS3Event (snsEvent) {
const s3Event = { Records: [] }
for (const snsRec of snsEvent.Records) {
try {
for (const s3Rec of JSON.parse(snsRec.Sns.Message).Records || []) {
@alanshaw
alanshaw / filter.txt
Created June 7, 2022 14:16
Filter github PRs you need to review
View filter.txt
review-requested:alanshaw
View dagChecker.js
// Lambda index CAR ////////////////////////////////////////////////////////////
const blocks = Set<CID>()
const links = Set<CID>()
const car = readCar(path)
for (const block in car) {
blocks.add(block.cid)
getLinks(block).forEach(cid => links.add(cid))
}
@alanshaw
alanshaw / custom-bs-path.mjs
Last active April 19, 2022 15:03
Store to NFT.Storage using a custom blockstore path
View custom-bs-path.mjs
import { NFTStorage } from 'nft.storage'
import { getFilesFromPath } from 'files-from-path'
import { FsBlockStore as Blockstore } from 'ipfs-car/blockstore/fs'
import Path from 'path'
const token = 'YOUR_API_TOKEN'
// ///////////////////////////////////////////////////////////////////////////
// Save DAG to a custom location by changing this path...
const bsPath = './blockstore'