Skip to content

Instantly share code, notes, and snippets.

@julian-q
Created August 15, 2024 04:04
Show Gist options
  • Save julian-q/eb19e4c034b731adeee5d7aa6ececf49 to your computer and use it in GitHub Desktop.
Save julian-q/eb19e4c034b731adeee5d7aa6ececf49 to your computer and use it in GitHub Desktop.
Create `metadata.json` for existing Minecraft data
# note: specifically for the minecraft data, based on "minecraft_video_dataset.py"
import json [1/110]
from pathlib import Path
splits = ["training", "validation"]
metadata = {}
for split in splits:
data_dir = Path("data/minecraft") / split
paths = sorted(list(data_dir.glob("**/*.npz")), key=lambda x: x.name)
lengths = [300] * len(paths)
metadata[split] = lengths
json.dump(metadata, open("data/minecraft/metadata.json", "w"))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment