Arweave Related Assets (Preview / Thumbnail) Spec

arweave-related-asset-spec.md

Arweave Related Assets (Preview / Thumbnail) Spec

Abstract

As the permaweb grows in popularity, complexity, and use there has been a growing need to associate assets as related to a logical asset, work, or publication. One major pain point for many permaweb builders is the large size of data assets. For example, a photographer may wish to publish a high-definition photograph ranging in size from hundreds of megabytes to one or more gigabytes. Another example would be video or 3D model assets that would tax the end user's browser when consuming these assets through aggregation dapps. A standard specification is thus proposed as a way to enhance composability and consumption of assets across the permaweb.

Use Cases

1. Thumbnail/preview asset association to represent primary assets
   • Example: A photographer wants to associate smaller, web-friendly versions of large photographs
   • Example: A 3D artist wants to associate an animated GIF to act as a thumbnail of their model
   • Example: A musician wants to publish in a lossless format and associate a streamable transcoding
   • Example: A musician wants to associate album cover art with their audio publications
2. Association of related assets
   • Example: A musician wants to include audio stems and samples with their song publication
   • Example: A digital artist wants to include project files with their publication (e.g. PSDs)
3. Summarization/aggregates, manifest extensions
   • Example: A data scientist wants to associate an aggregation summary of raw data
4. Allow permaweb dapps and users to consume alternate or lighter versions of assets
   • Example: A permaweb aggregation dapp developer wants to offer a lighter, more data-usage-friendly feed

Previous Work

Art By City Examples

1. Image Previews

69vPDs_Zse04iiwYst0JucbOWTZj_suFT5NFE0Ydmwk

DeSoGhost #249 by Becopro

{
  "images": [
    {
      "image": "ywDgsmy_M4iwMXsHwfw-pfS3VRiCbYeZHMLNhEpUS54",
      "preview": "JSrP-65mEfjpYRkxA8NCCclwO9L6m6AjzWVXhL7V-7w",
      "preview4k": "12TptImDca8AoJsox6ZbfmoCc6a2K0bVEQHcZCkJ3BE"
    }
  ]
}

Note: "images" is an array because art by city image publications can include any number of images as part of the work (e.g. for a series or collection)

2. 3D Model Preview

rtGz60g-6q4P5j8XJbDcHRiNMS5xOvdLD9JVV02Mxbg

Modified Dodecahedron 3D by PARATECT

{
    "image": {
        "image": "MJg1RnioOB2HxOl7nC7bI-ahhRhhifkhg7pgtgBvGSk",
        "preview": "L7L3_Pg2xgUGarWxSqul-bxIMTvWkNGiURyuEv2zDBc",
        "preview4k": "yVQfB5FU8WmoLYgPMiZITW1ujPbPYXnB72pnGDFAKj8"
    },
    "model": {
        "model": "IQZcin-3bWo0cYaeiq1FWSKgr3tDmw7z0pQBZwg9eOw"
    }
}

3. Audio

q0GSg9bSntJQIj-FiHYApMat1e20EGM2JkdgrkhtZjI

Air by Slava

{
    "image": {
        "image": "yUfT20pPzp4WByFSMz2aFUXZs1liGI4yM2OppNSTJpw",
        "preview": "Zzw2Cbq2ze6kVdGvkg_KkPMbqz-y-u7U8qaGdNWOLYQ",
        "preview4k": "7v_paxh8pd4IsEKFZcA9KdELlbauEBHvmfysMsAssdg"
    },
    "audio": {
        "audio": "knDvn3n4U6FiDTvKSvy-WBOltBxSojAj5yNF7a_rBOo"
    }
}

Criticism

• Developed as needed for the Art By City dapp
• Not meant to be as generic/open as this specification
• Doesn't fully describe the assets
• Clunky, repetitive
• Please replace with something better 😄

Proposal Starting Point

Option 1: Manifest Approach

Starting with the above Art By City examples, provide more information about related assets. Assuming these entries are in an associated JSON manifest (e.g. ArFS)

Note: "resolution" below refers to the maximum possible resolution of the described asset. This is a guarantee that the described asset is no bigger than that resolution.

1. Image Example

This example includes an image asset with web-friendly alternate resolutions.

{
    "asset": "ywDgsmy_M4iwMXsHwfw-pfS3VRiCbYeZHMLNhEpUS54",
    "related": [
        { "type": "image/jpeg", "resolution": "hd", "asset": "JSrP-65mEfjpYRkxA8NCCclwO9L6m6AjzWVXhL7V-7w" },
        { "type": "image/jpeg", "resolution": "4k", "asset": "12TptImDca8AoJsox6ZbfmoCc6a2K0bVEQHcZCkJ3BE" }
    ]
}

Alternatively, rather than choosing identifiers for common resolutions, the dimensions could be included explicitly as a string identifier (e.g. resolution: "1920x1080")

2. 3D Model Example

This example includes a 3D model asset with animated and static preview/thumbnail assets.

{
    "asset": "IQZcin-3bWo0cYaeiq1FWSKgr3tDmw7z0pQBZwg9eOw",
    "related": [
        { "type": "image/gif", "asset": "tx-id-of-a-preview-gif" },
        { "type": "image/png", "asset": "MJg1RnioOB2HxOl7nC7bI-ahhRhhifkhg7pgtgBvGSk" },
        { "type": "image/jpeg", "resolution": "hd", "asset": "L7L3_Pg2xgUGarWxSqul-bxIMTvWkNGiURyuEv2zDBc" },
        { "type": "image/jpeg", "resolution": "4k", "asset": "yVQfB5FU8WmoLYgPMiZITW1ujPbPYXnB72pnGDFAKj8" }
    ]
}

3. Audio Example

This example includes an audio asset with image preview/thumbnail assets and streamable audio asset

{
    "asset": "knDvn3n4U6FiDTvKSvy-WBOltBxSojAj5yNF7a_rBOo",
    "related": [
        { "type": "audio/mpeg", "asset": "tx-id-of-streamable-audio" },
        { "type": "image/png", "asset": "yUfT20pPzp4WByFSMz2aFUXZs1liGI4yM2OppNSTJpw" },
        { "type": "image/jpeg", "resolution": "hd", "asset": "Zzw2Cbq2ze6kVdGvkg_KkPMbqz-y-u7U8qaGdNWOLYQ" },
        { "type": "image/jpeg", "resolution": "4k", "asset": "7v_paxh8pd4IsEKFZcA9KdELlbauEBHvmfysMsAssdg" },
    ]
}

Option 2: Tags Approach

Alternatively, in the interest of dapps being able to quickly find and display related preview assets for a given asset, minimal but powerful hints could be included as tags. This way, when the primary asset is discovered, developers already have enough information to begin figuring out which preview assets they're interested in, minimizing additional queries or parsing of custom manifests.

Related assets would be tagged with a Related-To tag with a value of the transaction ID original asset it is related to.

If relevant, in the case of media such as images or video, resolution hints can be provided via Width and Height tags.

Tag Name	Optional?	Tag Value
Related-To	False	Transaction ID of related asset
Content-Type	True	Encouraged 😄
Width	True	Pixel width of content, e.g. 1920
Height	True	Pixel height of content, e.g. 1080

Instead of asset could dataTxId be used instead? This would marry this spec along with ArFS since dataTxId is used to reference the data that the metadata JSON points to, which is the same concept as above. Using asset still works too, as it doesnt conflict with any of the other tags used by ArFS.

Regarding the resolution field I think it would be better to explicit have width and height values instead of a string that needs to be parsed.

Having these values easy accessible will make it easier for apps to filter the resolutions they care.

Example:

{
  "type": "image/jpeg",
  "resolution": {
    "label": "hd",
    "width": 1280,
    "height": 720
  },
  "asset": "JSrP-65mEfjpYRkxA8NCCclwO9L6m6AjzWVXhL7V-7w"
}

or even use null if you only care about the label:

{
  "type": "image/jpeg",
  "resolution": {
    "label": "hd",
    "width": null,
    "height": null
  },
  "asset": "JSrP-65mEfjpYRkxA8NCCclwO9L6m6AjzWVXhL7V-7w"
}

Instead of asset could dataTxId be used instead? This would marry this spec along with ArFS since dataTxId is used to reference the data that the metadata JSON points to, which is the same concept as above. Using asset still works too, as it doesnt conflict with any of the other tags used by ArFS.

Definitely!

Although, after presenting this at the specs meetup and thinking about it a bit more, it might be a better impl to do this via tags. For example, related assets could have a Relates-To tag with value being the primary asset tx id. Content-Type is already there in tags. Could add Resolution and Width and Height for images.

This way, there's no manifest to fetch and parse - it's a hard sell to get most of the permaweb to start thinking about manifests as the primary entry point to an asset rather than the asset itself. Instead, this array of related assets could be fetched by querying Relates-To and parsing the tags.

Additionally, this would allow third parties to publish and associate their own assets they believe should be associated, or have been remixed from

@vilenarios @karlprieb Updated the gist with an alternative tags based approach, lmk thoughts!

One remaining question: would it make sense to signal the type of relation something has to the original asset?

For example, if it is meant to be an "alternative render" as is the case with thumbnails, this could explicitly be signaled with another tag like Relation-Type with a value like thumbnail.

I like this idea.
I would like to have support for multi channel video. So the entire art piece could be 10 video files that are played in parallel on different devices.
Could this concept be standardized as an ERC for ethereum?

@vilenarios @karlprieb
Draft spec here: https://gist.github.com/jim-toth/2de5f602c790e77c92970007c0df373b

I like this idea. I would like to have support for multi channel video. So the entire art piece could be 10 video files that are played in parallel on different devices. Could this concept be standardized as an ERC for ethereum?

This is mostly about how to tag assets when publishing them to Arweave. These tags can be queried from Arweave Gateways. I suppose with an erc you could have a primary asset be the URI, and then know to search Related-To of the arweave tx id. Not sure how to or if it's appropriate to make these kind of associations in an EVM contract