Skip to content

Instantly share code, notes, and snippets.

@garystafford

garystafford/kinesis-redshift-streaming-data.sh

Last active October 14, 2020 13:57
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save garystafford/661a81549610ea1d892cc97b75f7001a to your computer and use it in GitHub Desktop.
Save garystafford/661a81549610ea1d892cc97b75f7001a to your computer and use it in GitHub Desktop.
Download ZIP
Raw
kinesis-redshift-streaming-data.sh
# Install required Python packages
python3 -m pip install --user -r scripts/requirements.txt
# Set default AWS Region for script
export AWS_DEFAULT_REGION=us-east-1
# Option #1: Execute script in foreground
python3 ./scripts/kinesis_put_streaming_data.py
# Option #2: execute script in background
nohup python3 -u ./scripts/kinesis_put_streaming_data.py > output.log 2>&1 </dev/null &
# Check that the process is running
ps -aux | grep 'python3 -u ./scripts/kinesis_put_streaming_data.py'
# Wait 1-2 minutes, then check output to confirm script is working
cat output.log
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment