Skip to content

Instantly share code, notes, and snippets.

@chrisdlangton
Last active June 5, 2021 14:45
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save chrisdlangton/57e9e8e09566324d182a065458d17737 to your computer and use it in GitHub Desktop.
Save chrisdlangton/57e9e8e09566324d182a065458d17737 to your computer and use it in GitHub Desktop.
Node.js 14 download to file via proxy
require('dotenv').config()
const { URL } = require('url');
const http = require('http')
const https = require('https')
const yaml = require('js-yaml')
const fs = require('fs')
const config = yaml.load(fs.readFileSync(process.env.CONFIG_FILE, 'utf8'))
const download_to_file = (url, dest_path, callback) => {
const file = fs.createWriteStream(dest_path)
let options = {}
options.minVersion = "TLSv1.2"
options.maxVersion = "TLSv1.3"
options.host = (new URL(url)).hostname
options.path = (new URL(url)).pathname
const handler = res => {
res.pipe(file)
file.on('finish', () => file.close(callback))
}
const err_handler = err => {
console.warn(err)
}
if ('proxy' in config.app) {
options.host = config.app.proxy.host
options.port = config.app.proxy.port
options.path = url
options.headers = {
Host: new URL(url).hostname,
// 'Proxy-Authorization': 'Basic ' + new Buffer(`${username}:${password}`).toString('base64')
}
http.get(options, handler).on('error', err_handler).end()
} else if (url.startsWith('https')) {
https.get(options, handler).on('error', err_handler).end()
} else if (url.startsWith('http')) {
http.get(options, handler).on('error', err_handler).end()
}
}
@chrisdlangton
Copy link
Author

The .env example

CONFIG_FILE=config.yaml

The package.json example

{
    "name": "",
    "version": "0.0.1",
    "dependencies": {
        "js-yaml": "^3.13.1",
        "dotenv": "^8.2.0"
    },
    "author": "",
    "license": "ISC"
}

The config.yaml example

---
app:
  proxy:
    host: proxy.trivialsec.local
    port: 3128
    username: squid
    password: "should be stored in a vault and not here in plain text"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment