Skip to content

Instantly share code, notes, and snippets.

@rjurney
Created July 1, 2024 21:12
Show Gist options
  • Save rjurney/66cd9c7b90136817189cd62e01b579cc to your computer and use it in GitHub Desktop.
Save rjurney/66cd9c7b90136817189cd62e01b579cc to your computer and use it in GitHub Desktop.
Download and unzip the International Consortium of Investigative Journalists (ICIJ) knowledge graph dataset
#!/usr/bin/env bash
: '
@echo off
powershell -ExecutionPolicy Bypass -Command "$ErrorActionPreference='Stop'; $ProgressPreference='SilentlyContinue';
$output_file = 'data/full-oldb.LATEST.zip'
$extract_dir = 'data'
Write-Host "`nDownloading the ICIJ Offshore Leaks Database to $output_file`n"
Invoke-WebRequest -Uri 'https://offshoreleaks-data.icij.org/offshoreleaks/csv/full-oldb.LATEST.zip' -OutFile $output_file
Write-Host "`nUnzipping $output_file to $extract_dir`n"
Expand-Archive -Path $output_file -DestinationPath $extract_dir
exit
'
# Bash script starts here
output_file="data/full-oldb.LATEST.zip"
extract_dir="data"
echo -e "\nDownloading the ICIJ Offshore Leaks Database to $output_file\n"
curl "https://offshoreleaks-data.icij.org/offshoreleaks/csv/full-oldb.LATEST.zip" -o "$output_file"
echo -e "\nUnzipping $output_file to $extract_dir\n"
unzip "$output_file" -d "$extract_dir"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment