Skip to content

Instantly share code, notes, and snippets.

@JayGoldberg
Last active December 19, 2023 18:48
Show Gist options
  • Save JayGoldberg/289a257bc885c77bf30d8dff265782ea to your computer and use it in GitHub Desktop.
Save JayGoldberg/289a257bc885c77bf30d8dff265782ea to your computer and use it in GitHub Desktop.
BigQuery requires new-line delimited JSON files where the geometry column is single string.
// https://medium.com/google-cloud/how-to-load-geojson-files-into-bigquery-gis-9dc009802fb4
// Javascript (NodeJS) vs Python
function convertGeoJsonBq() {
const fs = require('fs');
fs.readFile(process.argv[1], 'utf8', (err, data) => {
if (err) throw err;
const geojson = JSON.parse(data);
const features = geojson.features;
const output = features.map((feature) => {
const props = feature.properties;
props.geometry = JSON.stringify(feature.geometry);
if (!schema) {
schema = Object.keys(props).map((key) => {
if (key === 'geometry') {
return 'geometry:GEOGRAPHY';
} else if (typeof props[key] === 'string') {
return key;
} else {
return `${key}:${typeof props[key] === 'number' ? 'float64' : 'int64'}`;
}
}).join(',');
}
return props;
});
const schema = Object.keys(output[0]).join(',');
fs.writeFile('to_load.json', JSON.stringify(output, null, 2), (err) => {
if (err) throw err;
console.log('Schema: ', schema);
});
});
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment