Skip to content

Instantly share code, notes, and snippets.

@natanlao
Last active August 11, 2020 05:12
Show Gist options
  • Save natanlao/ff455f1ad18aebe74bab738b827106e9 to your computer and use it in GitHub Desktop.
Save natanlao/ff455f1ad18aebe74bab738b827106e9 to your computer and use it in GitHub Desktop.
Tips for working with s3cmd on rsync.net

Working with s3cmd on rsync.net

This document lists some problems I encountered using s3cmd on rsync.net and how I solved them.

Uploading with the ONEZONE_IA class

s3cmd sync doesn't support the --storage-class argument, and even that doesn't support the ONEZONE_IA storage class. You can still upload objects to that class by specifying the x-amz-storage-class header, but s3cmd sync will ignore that. The best solution I have is to upload your files as you normally would, then use s3cmd modify to set the storage class after upload.

See here for prior art.

Certificate verification fails

Running a command with s3cmd on rsync.net might elicit this error:

ERROR: SSL certificate verification failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)

rsync.net support has advised me that this can be solved by passing the --no-check-certificate argument:

$ s3cmd sync --no-check-certificate ...

They did not reply to my request as to solutions to running s3cmd without skipping certificate verification.

Append-only access to S3 with IAM

You may have already seen this article. The advice in the post works great for me, with two adjustments:

  1. Access can be restricted to only the target bucket (as opposed to all of your buckets)
  2. The ListAllMyBuckets permission is not necessary.

Here's a revised IAM policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:ListBucket",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::YourBucketName",
                "arn:aws:s3:::YourBucketName/*"
            ]
        }
    ]
}
@natanlao
Copy link
Author

@abendy Unfortunately not. It might be worth raising the issue with support again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment