Skip to content

Instantly share code, notes, and snippets.

@mariohercules
Last active May 30, 2019 13:18
Show Gist options
  • Save mariohercules/56a373ad6a313acd2070189a54af34f5 to your computer and use it in GitHub Desktop.
Save mariohercules/56a373ad6a313acd2070189a54af34f5 to your computer and use it in GitHub Desktop.
Download Files in Directory with Curl

USAGE

$ ./download_files.sh http://url_to_files/downloads/

CREATE SCRIPT

$ touch download_files.sh

#!/bin/bash

set -e

url=

# Get the first parameters
echo Starting in URL: "$1"

url="$1"

# Loop in files

for file in $(curl -s $url |
                  grep href |
                  sed 's/.*href="//' |
                  sed 's/".*//' |
                  grep '^[a-zA-Z].*'); do
    curl -s -O $url/$file
done

#!/bin/bash
set -e
url=
# Get the first parameters
echo Starting in URL: "$1"
url="$1"
# Loop in files
for file in $(curl -s $url |
grep href |
sed 's/.*href="//' |
sed 's/".*//' |
grep '^[a-zA-Z].*'); do
curl -s -O $url/$file
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment