Script that uses the aws cli to download cloudwatch logs to a file

download-aws-logs.sh

#!/usr/bin/env bash

set -euo pipefail
IFS=$'\n\t'

LOG_GROUP_NAME=""
LOG_STREAM_NAME=""
REGION=""
OUTPUT_FILE="$(date +"%Y%m%d").log"

result=$(aws logs get-log-events \
    --start-from-head \
    --log-group-name=${LOG_GROUP_NAME} \
    --log-stream-name=${LOG_STREAM_NAME} \
    --region=${REGION})
echo ${result} | jq -r .events[].message >> ${OUTPUT_FILE}

nextToken=$(echo $result | jq -r .nextForwardToken)
while [ -n "$nextToken" ]; do
    echo ${nextToken}
    result=$(aws logs get-log-events \
        --start-from-head \
        --log-group-name=${LOG_GROUP_NAME} \
        --log-stream-name=${LOG_STREAM_NAME} \
        --region=${REGION} \
        --next-token="${nextToken}")
    
    if [[ $(echo ${result} | jq -e '.events == []') == "true" ]]; then
        echo "response with empty events found -> exiting."
        exit
    fi

    echo ${result} | jq -r .events[].message >> ${OUTPUT_FILE}

    nextToken=$(echo ${result} | jq -r .nextForwardToken)
done

@dylanjsa Thank you for your response. Let me explain a few more details. I am deploying Database using ECS Task Definition (both RDS and MSSQL). There is a single task. However, during the task execution, I can see that the logs are not getting populated in real time or at regular intervals. Rather, there is no log message till some time and then there is a surge and then again nothing for a few seconds and then again a surge and so on. When this happens, it becomes a challenge, since the next token fetches nothing. Also, I am trying it out from a Shell Script step in a GitHub Action.