Skip to content

Instantly share code, notes, and snippets.

@curran
Last active March 28, 2022 18:40
Show Gist options
  • Save curran/5a01cbdcbb1122e6326d8b9618db8883 to your computer and use it in GitHub Desktop.
Save curran/5a01cbdcbb1122e6326d8b9618db8883 to your computer and use it in GitHub Desktop.
Observable Scrape

A DIY method of authoring ES modules in Observable and exporting into downstream workflows such as React apps.

import fetch from 'node-fetch';
import fs from 'fs';
// Observable API key.
// See https://observablehq.com/@observablehq/api-keys
const apiKey = 'insert-your-api-key-here';
const notebookId = 'insert your notebook-id-here';
const url = `https://api.observablehq.com/d/${notebookId}.js?v=3&api_key=${apiKey}`;
const scrapeNextFile = (lines, index) => {};
fetch(url)
.then((response) => response.text())
.then((str) => {
// Allow writing code that runs in ES module using "///"
let lines = str
.split('\n')
.map((line) => line.replace('///', ''));
let begin = lines.findIndex((line) => line.indexOf('BEGIN_EXTRACT') !== -1);
while (begin !== -1) {
const end = lines.findIndex((line) => line.indexOf('END_EXTRACT') !== -1);
const extractLines = lines.slice(begin + 1, end);
const extract = extractLines.join('\n');
const name = lines[begin].split(' ').pop();
const fileName = `src/observable/${name}.js`;
const fileContent = `import d3 from 'd3';\n${extract}\nexport {${name}};`;
fs.writeFileSync(fileName, fileContent);
console.log('Scraped file ' + fileName);
// Continue searching for the next occurrence of BEGIN_EXTRACT
lines = lines.slice(end + 1);
begin = lines.findIndex((line) => line.indexOf('BEGIN_EXTRACT') !== -1);
}
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment