Skip to content

Instantly share code, notes, and snippets.

@wesbos
Created November 11, 2020 02:07
Show Gist options
  • Save wesbos/e7efda4fa9f8221349e0bcf8c2b945e7 to your computer and use it in GitHub Desktop.
Save wesbos/e7efda4fa9f8221349e0bcf8c2b945e7 to your computer and use it in GitHub Desktop.
deno syntax episode downloader
// deno run --allow-net --allow-write download-shows.ts
import { download } from "https://deno.land/x/download/mod.ts";
const showList = 'https://syntax.fm/api/shows';
async function getShowList(): Promise<Show[]> {
const list = await (await fetch(showList)).json();
return list;
}
function generateShowName(show: Show) {
return `Syntax ${show.number} - ${show.title.replace('/', '')} - ${show.displayDate}.mp3`;
}
async function downloadShow(show: Show) {
const fileObj = await download(show.url, {
file: generateShowName(show),
dir: './shows'
});
}
async function go() {
const showList = await getShowList();
for (const show of showList) {
console.log(`Downloading show ${generateShowName(show)}`);
await downloadShow(show);
}
console.log('Done!');
}
go();
interface Show {
number: number;
title: string;
date: number;
url: string;
slug: string;
html: string;
notesFile: string;
displayDate: string;
displayNumber: string;
}
@lagden
Copy link

lagden commented Nov 11, 2020

add mkdir

// deno run --allow-net --allow-write --allow-read download-shows.ts

// ...

async function downloadShow(show: Show, dir: string) {
  const fileObj = await download(show.url, {
    file: generateShowName(show),
    dir
  })
}

async function mkdir(dir: string) {
  try {
    const fileInfo = await Deno.stat(dir)
  } catch {
    await Deno.mkdir(dir, {mode: 0o755})
  }
}

async function go(dir: string) {
  await mkdir(dir)
  const showList = await getShowList()
  for (const show of showList) {
    console.log(`Downloading show ${generateShowName(show)}`)
    await downloadShow(show, dir)
  }
  console.log('Done!')
}

go('shows')

//...

@Maximilianos
Copy link

Maximilianos commented Nov 11, 2020

great stuff :)

My only suggestion would be to optimise the downloads slightly by firing them off all together instead of sequentially:

async function go() {
  const showList = await getShowList();

  const pendingDownloads = showList.map(show => {
    console.log(`Downloading show ${generateShowName(show)}`);
    return downloadShow(show);
  });

  await Promise.all(pendingDownloads);

  console.log('Done!');
}

go();

@wesbos
Copy link
Author

wesbos commented Nov 11, 2020

there are 300 shows, would likely get blocked by the server or download all of them slowly. Not a huge benefit to doing that.

allSettled would be better too, because if 1 show failed, the whole jig is up

@Maximilianos
Copy link

good point on the allSettled - haven't had to use it yet to be honest, so doesn't spring to mind!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment