Skip to content

Instantly share code, notes, and snippets.

@nemani
Last active March 27, 2024 12:10
Show Gist options
  • Star 26 You must be signed in to star a gist
  • Fork 11 You must be signed in to fork a gist
  • Save nemani/defdde356b6678352bcd4af69b7fe529 to your computer and use it in GitHub Desktop.
Save nemani/defdde356b6678352bcd4af69b7fe529 to your computer and use it in GitHub Desktop.
Download All Lambda Functions
# Parallelly download all aws-lambda functions
# Assumes you have ran `aws configure` and have output-mode as "text"
# Works with "aws-cli/1.16.72 Python/3.6.7 Linux/4.15.0-42-generic botocore/1.12.62"
download_code () {
local OUTPUT=$1
aws lambda get-function --function-name $OUTPUT --query 'Code.Location' | xargs wget -O ./lambda_functions/$OUTPUT.zip
}
mkdir -p lambda_functions
for run in $(aws lambda list-functions | cut -f 6 | xargs);
do
download_code "$run" &
done
echo "Completed Downloading all the Lamdba Functions!"
# If you want to download only ones with specific prefix
# https://github.com/sambhajis-gdb/download_all_lambda_function/blob/master/get_lambda_with_prefix.sh
# Credits to https://stackoverflow.com/users/6908140/sambhaji-sawant for the
@natanloterio
Copy link

natanloterio commented Dec 25, 2021

This is another version:


download_code () {
    local OUTPUT=$1
    aws lambda get-function --function-name $OUTPUT --query 'Code.Location' | xargs wget -O ./lambda_functions/$OUTPUT.zip  
}

mkdir -p lambda_functions

for run in $(aws lambda list-functions | grep FunctionName | awk -F'"' '{print $4}');
      do
           download_code "$run" &
      done
  
      echo "Completed Downloading all the Lamdba Functions!"


After that, you could unzip them in separated folders using:


find . -name '*.zip' -exec sh -c 'unzip -d "${1%.*}" "$1"' _ {} \;

@abhay330
Copy link

If someone is looking for a Pythonic way to do this : https://github.com/abhay330/download_all_lambda_function

@avicoder
Copy link

It crashed my 64GB RAM mac, Twice :/

@behrens-cloud
Copy link

behrens-cloud commented Dec 21, 2022

It crashed my 64GB RAM mac, Twice :/

I can verify that the original version does that on Macs. @handeglc removed the & to not run in background. I did the same and also added a loop to capture all regions. It could be improved even more of course. Thanks @nemani for posting this gist!

# Downloads all aws-lambda functions to a subdirectory
# Assumes you have a role that has at least read access to lambda.
# Credits to https://gist.github.com/nemani/defdde356b6678352bcd4af69b7fe529

# trap ctrl-c and call ctrl_c()
trap ctrl_c INT
function ctrl_c() {
        echo "** CTRL-C Detected - aborting..."
        exit -1
}

download_region(){
    REGION=$1
    echo Searching region: $REGION
    for code in $(aws lambda list-functions --region $REGION --query 'Functions[].FunctionName' --output text);
    do
        download_path=./lambda_functions/$REGION/$code
        mkdir -p $download_path
        aws lambda get-function --function-name $code --region $REGION --query 'Code.Location' | xargs wget --quiet -O $download_path/$code.zip
        unzip -o -f -d $download_path $download_path/$code.zip
    done
}

# search all regions
for region in $(aws ec2 describe-regions --query "Regions[].{Name:RegionName}" --output text | sort -r); do
    download_region $region
done

echo "Completed Downloading all the Lamdba Functions!"

@MossabDiae
Copy link

Thanks for sharing, the code didn't work as stated in some previous replies I believe because of the filtering criteria to get the lambda function name.
AWS CLI is returning a JSON format which can be better explored using jq

If the above code didn't work for you too, try my implementation of the fix here.

@hrpitta
Copy link

hrpitta commented Feb 17, 2023

@abhay330 thanks man! Worked very well

@bf4
Copy link

bf4 commented Feb 27, 2023

very nice collaboration. I extended it in a fork https://gist.github.com/bf4/825009c3e49bf53259e8c18c4c38f190

#!/bin/bash
#
# Usage:
#  ./download_all.sh
#  ./download_all.sh download us-east-1 my-function
#  ./download_all.sh help
#
# Downloads all aws-lambda functions to a subdirectory
# Assumes you have a role that has at least read access to lambda.
# Credits to https://gist.github.com/nemani/defdde356b6678352bcd4af69b7fe529

set -euo pipefail
IFS=$'\n\t'
# Influenced by https://dev.to/thiht/shell-scripts-matter

abort=0
show_help() {
 abort=${abort:0}
 sed -ne '/^#/!q;s/.\{1,2\}//;1d;p' < "$0" >&$((abort+1))
 exit $abort
}

# shellcheck disable=SC2034
export AWS_PROFILE="${AWS_PROFILE:-your-profile-here}"
readonly DOWNLOAD_PATH_BASE="./lambda_functions"

# trap ctrl-c and call ctrl_c()
trap ctrl_c INT
ctrl_c() {
  echo "** CTRL-C Detected - aborting..."
  exit 1
}

get_function_names() {
  local region; region="$1"
  aws lambda list-functions --region "$region" --query 'Functions[].FunctionName' --output text
}

get_function() {
  local region; region="$1"
  local function_name; function_name="$2"
  local download_path
  download_path="${DOWNLOAD_PATH_BASE}/$region/$function_name"
  mkdir -p "$download_path"
  aws lambda get-function --function-name "$function_name" --region "$region" --query 'Code.Location' \
    | xargs wget --quiet -O "$download_path/$function_name.zip"

  unzip -o -f -d "$download_path" "$download_path/$function_name.zip"
}

download_region(){
  local region; region="$1"
  echo "searching region: $region"
  for function_name in $(get_function_names "$region"); do
    get_function "$region" "$function_name"
  done
}

get_regions(){
  aws ec2 describe-regions --query "Regions[].{Name:RegionName}" --output text | sort -r
}

main() {
  # search all regions
  for region in $(get_regions); do
    download_region "$region"
  done

  echo "Completed Downloading all the Lamdba Functions!"
}

case "${1:-}" in
  help|-h|--help)
    show_help
  ;;

  download)
    shift
    get_function "$1" "$2"
  ;;

  *)
    shift
    main
  ;;
esac

@psamson
Copy link

psamson commented Jul 4, 2023

The original script worked for me on a Mac M1, ...BUT it deleted all the Lambda functions on AWS after downloading them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment