💡 Smart Description: This code defines a DedupeAssetQueue class that takes an array of strings and updates the batch size to 100. It also includes methods for adding, updating, fetching, rendering, creating, writing, or deleting assets in large scale applications

Dedupe Asset Queue with Cache and Format.md

Dedupe Asset Queue with Cache and Format

Preview:

import PiecesCacheSingleton from 'cache/pieces_cache';
import { Asset, Format } from '../../PiecesSDK/core';
import ConnectorSingleton from './connector_singleton';
import { mergeAssetsWithTransferables } from 'transferables';
import { renderFetched } from './stream_assets';

export default class DedupeAssetQueue {
  queue: string[];
  // Specified type for idMap to be more explicit
  idMap: Record<string, boolean>;
  inFetch: boolean;
  private readonly batchSize: number;

  constructor() {
    this.queue = [];
    this.idMap = {};
    this.inFetch = false;
    this.batchSize = 100;
  }

  push(id: string) {
    // Checking for the absence of id in the map rather than presence.
    // If it's not present, then we add it, thus reducing a level of indentation.
    if (!this.idMap[id]) {
      this.idMap[id] = true;
      this.queue.push(id);
      if (!this.inFetch) this.doFetch();
    }
  }


  /**
  
  The original code was set to iterate up to this.batchSize (which is 100), regardless of the actual length of this.queue array. This led to unnecessary iterations when the queue has fewer than 100 elements.

  The unnecessary iterations performed checks on non-existent elements, leading to wasted computational resources.

  The optimized code uses Math.min(this.batchSize, this.queue.length), ensuring the loop iterates only over the actual elements present in the queue, up to a maximum of this.batchSize.

   If the queue has fewer elements than the batch size, the loop stops earlier, thus saving unnecessary iterations and checks.

  */
  async doFetch() {
    const cache = PiecesCacheSingleton.getInstance();
    const config = ConnectorSingleton.getInstance();
    this.inFetch = true;

    while (this.queue.length) {
      try {
        const assetReqList: Promise<Asset>[] = [];

        // Loop iterates only over the actual elements in the queue up to the batchSize,
        // whichever is less. This avoids unnecessary iterations and checks for 'undefined'.
        for (let i = 0; i < Math.min(this.batchSize, this.queue.length); i++) {
          this.idMap[this.queue[i]] = false;
          assetReqList.push(
            config.assetApi.assetSnapshot({
              asset: this.queue[i],
              transferables: false,
            })
          );
        }

        // Using slice to create a new queue after removing the first 'batchSize' elements.
        // Slice creates a new array, and in large scale applications, the garbage collector would clear out old unused memory,
        // this could potentially save memory in cases where the queue size grows significantly.
        this.queue = this.queue.slice(this.batchSize);

        const fetchedAssets = await Promise.all(assetReqList);

        const formatReqList: Promise<Format>[] = [];

        // Using 'for of' loop, which is simpler and cleaner when we don't need to use the index of elements.
        for (const asset of fetchedAssets) {
          for (const format of asset.formats.iterable || []) {
            formatReqList.push(
              config.formatApi.formatSnapshot({
                format: format.id,
                transferable: true,
              })
            );
          }
        }

        const fetchedFormats = await Promise.all(formatReqList);

        fetchedFormats.forEach((format) => {
          cache.fetchedFormats[format.id] = new Date();
          cache.formatTransferables[format.id] = {
            file: format.file,
            fragment: format.fragment,
          };
        });

        const mergedAssets = mergeAssetsWithTransferables({
          assets: { iterable: fetchedAssets },
          references: cache.formatTransferables,
        });

        renderFetched({ assets: mergedAssets });
      } catch (error: any) {
        console.error(`Error fetching: ${error}`);
        // Comment to show the possibility of more robust error handling.
        // throw error;
      }
    }

    this.inFetch = false;
  }
}

Associated Context
Type	Code Snippet ( .ts )
Associated Tags	renderFetched reactjs ethereum subscription smartcontracts assetApi stream_assets mergeAssetsWithTransferables Connector Singleton autodesk-viewer PiecesCacheSingleton Format Asset DedupeAssetQueue solana batchSize
💡 Smart Description	This code defines a DedupeAssetQueue class that takes an array of strings and updates the batch size to 100. It also includes methods for adding, updating, fetching, rendering, creating, writing, or deleting assets in large scale applications
🔎 Suggested Searches	PiecesCacheSingleton instance ConnectorSingleton mergeAssetsWithTransferables renderFetched DedupeAssetQueue queue length and batchSize PiecesCacheSingleton create a new array after removing first 'batchSize' elements Iterating over all items in the queue with transferables
Related Links	No Related Links
Related People	Tsavo Knott
Sensitive Information	No Sensitive Information Detected
Shareable Link	https://tsavo.pieces.cloud/?p=ec75438eb6