Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Copies an rds parameter group with support for cross-region copying. Quick bashed out script because AWS cli doesn't support.
"""
Copies a parameter group in RDS, particularly useful for doing cross-region copies (which AWS documents,
but doesn't actually support at the time of this writing.
Usage: python cp-parameter-group.py us-west-1:lookerdb-56 us-west-2:lookerdb-56
"""
import boto3
import sys
def chunks(sequence, chunk_size):
"""
Yields the sequence as a sequence of sequences size chunk_size (or fewer,
in the case of the last chunk). Guarantees delivery of everything (as
opposed to strategies that leave elements off of the end when
len(sequence) % chunk_size != 0
"""
start = 0
while start < len(sequence):
end = start + chunk_size
yield sequence[start : start + chunk_size]
start = end
# region:parameter_name
source_region, source_name = sys.argv[1].split(":")
source_client = boto3.client("rds", region_name=source_region)
source_summary = source_client.describe_db_parameter_groups(
DBParameterGroupName=source_name
)
source_family = source_summary["DBParameterGroups"][0]["DBParameterGroupFamily"]
source_description = source_summary["DBParameterGroups"][0]["Description"]
source_parameters = source_client.describe_db_parameters(
DBParameterGroupName=source_name
)["Parameters"]
source_parameters = [
p for p in source_parameters if p["IsModifiable"] and "ParameterValue" in p
]
target_region, target_name = sys.argv[2].split(":")
target_client = boto3.client("rds", region_name=target_region)
groups_in_target_region = set(
[
g["DBParameterGroupName"]
for g in target_client.describe_db_parameter_groups()["DBParameterGroups"]
]
)
if target_name in groups_in_target_region:
raise ValueError(
"This group (%s) already exists in region %s" % (target_name, target_region)
)
print "Created %s in %s" % (target_name, target_region)
target_client.create_db_parameter_group(
DBParameterGroupName=target_name,
DBParameterGroupFamily=source_family,
Description=source_description,
)
# AWS limits parameter modifications to 20 at a time
for parameters in chunks(source_parameters, 20):
target_client.modify_db_parameter_group(
DBParameterGroupName=target_name, Parameters=parameters
)
for parameter in parameters:
print "%s = %s" % (parameter["ParameterName"], parameter["ParameterValue"])
print "Complete."
@phill-tornroth

This comment has been minimized.

Copy link
Owner Author

@phill-tornroth phill-tornroth commented Aug 23, 2018

Tell me if you make improvements!

@EpiJunkie

This comment has been minimized.

Copy link

@EpiJunkie EpiJunkie commented Dec 20, 2018

Hello @phill-tornroth, I made some improvements and put them here: https://github.com/EpiJunkie/aws_rds_parameter_groups_utility

@williamsdb

This comment has been minimized.

Copy link

@williamsdb williamsdb commented Feb 25, 2019

Hi

When I run I get the following

botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the DescribeDBParameterGroups operation: User: arn:aws:iam::0000000000:user/amazon-live is not authorized to perform: rds:DescribeDBParameterGroups on resource: arn:aws:rds:eu-west-1:000000000:pg:xxx-pg-eu-2

I sort of assumed that giving the AWS user AmazonRDSFullAccess policy in IAM would be sufficient but clearly not! What policies do I need to attach to the IAM user to get this to work do you know?

Thanks

@williamsdb

This comment has been minimized.

Copy link

@williamsdb williamsdb commented Feb 25, 2019

Update!

Turns out it wasn't really permissions at all. in aws configure the Default region name was set to eu-west-2. Changing it to eu-west-1 sort it.

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment