Skip to content

Instantly share code, notes, and snippets.

@shino
Last active November 17, 2016 01:03
Show Gist options
  • Save shino/447a57ee9a6c2af10823 to your computer and use it in GitHub Desktop.
Save shino/447a57ee9a6c2af10823 to your computer and use it in GitHub Desktop.

TODOs

  • Understand v4 auth
  • How to treat aws-region (probably hardcord it as us-east-1)
  • Implement authorization header / non-aws-chunked case
  • Implement query parameter style auth case
  • Calculate SHA256 of Payload (along with MD5? Necessary?)
    • What error happens if sha256 is wrong?
  • Test Authorization header case
    • boto
    • Does ceph test support v4?
  • Test query parameter style case
    • boto?
    • Does ceph test support v4?
  • Client side implementation in basho/erlcloud
  • Keep tests with v2 auth
  • Performance consideration
    • Must we calculate sha256 of payload? or just use x-amz-content-sha256?
    • Can we test s3 with fake x-amz-conent-sha256 and authorization headers? (both should be faked for consistency of headers)
    • Erlang/OTP performace difference between MD5 and sha256
    • Shoud calculate both MD5 and sha256 for payload ? Or sha256 only?
    • How to calculate ETag for MP uploaded objects?

TODOs related to aws-chunked upload

  • Investigate which client/SDK supports aws-chunked uploads
  • Authorization header / aws-chunked / no tranfer encoding case
    • Content-Length header does not represent object size, how to know/compute?
    • s3cmd and boto do not implement this.
  • Authorization header / aws-chunked / chunked tranfer encoding case
    • No Content-Length header, c-l will be determined after transfer?
  • Access stats log for chunked upload

TODOs lower priority

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment