shino/s3-tests-to-riakcs-failures.org Secret

## s3-tests-to-riakcs-failures.org

      
    Raw
  

              s3-tests-to-riakcs-failures.org
            
          
    Categorize ERRORs and FAILs of s3-tests against Riak CS

API Complatibility bug of Riak CS, Midium


  Copy and Metadata
    
      [x] test_object_copy_retaining_metadata
        
          CS bug, metadata of source object are not retained, just discarded
            AssertionError: {} != {'key2': 'value2', 'key1': 'value1'}
            
          
          Copy API doesn’t copy user meta data. [JIRA: RCS-153] · Issue #1097 · basho/riak_cs
            basho/riak_cs#1097
        
      
      [x] test_object_copy_replacing_metadata
        
          CS bug, metadata is not replaced, Content-Type in this test case
            AssertionError: 'audio/ogg' != 'audio/mpeg'
            
          
      Fix by “Add x-amz-metadata directive handling to COPY/REPLACE user metadata”
        basho/riak_cs@20c1fb9 : https://github.com/basho/riak_cs/commit/20c1fb9?w=1
    
  
  Presigned URL with response headers
    
      [ ] test_object_raw_response_headers
        
          Assertion failure, expected 200 but 403
            AssertionError: 403 != 200
            
          
          Test for presigned get request with response-* headers
          response headers is included in STS, but Riak CS does not support them yet,
            should fail now.
        
      
  MP parts validation
    
      [x] test_multipart_upload_resend_part
        
          400 Bad Request with <Code>InvalidPartOrder</Code>
          Boto includes multiple elements with PartNumber=1 which have identical ETag.
          Riak CS’s validation of PartNumber (strictry increasing) may be too restrictive.
        
      
  Bucket/Object ACL
    
      ACL headers do not have double quotes around values
        
          AWS example
            x-amz-grant-read: emailAddress="xyz@amazon.com"
            
          
          test_s3.py generates
            x-amz-grant-full-control: id=e4b5c8c87b266c1e0990074096f81c8090eb70322f7b4ab64c900348eb291483
            
          
      [ ] test_object_acl_full_control_verify_owner
        
          Owner (main) adds FULL_CONTROL grant to alt, alt adds READ_ACP grant to himself,
            test asserts owner is not changed.
          Error occurs at adding READ_ACP by alt, 403 Forbidden
        
      
      [ ] test_object_header_acl_grants
        
          400 Bad Requests at PUT Object with all ACL headers (full_control, read, write,
            read_acl, write_acl) by id of alt
          Perhaps caused by ACL validation at wm_object resource (?)
        
      
      [ ] test_s3.test_bucket_header_acl_grants
        
          400 Bad Requests at PUT Bucket with all ACL headers (full_control, read, write,
            read_acl, write_acl) by id of alt
          Seems like almost the same as above with s/object/bucket/
        
      
      [ ] test_s3.test_logging_toggle
        
          400 Bad Request at Bucket#set_as_logging_target()
          In HTTP layer, Request to <bucket-name>-log/?acl with ACL grants of FULL_CONTROL,
            WRITE and READ_ACP and grantee of group http://acs.amazonaws.com/groups/s3/LogDelivery
            results in 400.
          boto’s bucket.py, set_as_logging_target calls
          Riak CS does not implement Group Grantee except all_users/auth_users, I guess.
        
      
API Complatibility bug of Riak CS, Minor


  [x] test_put_object_ifmatch_failed
    
      `If-Match` does not fail and assertion fails
        AssertionError: None != 'PreconditionFailed'
        
      
      Test code reads as below, no double-quote chars around value.
        headers={'If-Match': 'ABCORZ'}
        
      
      cf: RFC 7232,  2.3. ETag and 3.1 If-Match
      Example XML of conditional GET failure
        <Error>
  <Code>PreconditionFailed</Code>
  <Message>At least one of the pre-conditions you specified did not hold</Message>
  <Condition>If-Match</Condition>
  <RequestId>5A5B6E223767278F</RequestId><HostId>9dg/jrO4GWGYTXjB+GWskcGhoaAOZDa5sN9q/Ftfs3TMTypv0BOYJp+sIH/+MThKbsPOt/jnVSw=</HostId></Error>
        
      
  [ ] test_put_object_ifmatch_nonexisted_failed
    
      Conditional PUT is not supported by AWS S3, so classified to Minor
        HTTP/1.1 501 Not Implemented
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NotImplemented</Code>
   <Message>A header you provided implies functionality that is not implemented</Message>
   <Header>If-Match</Header><RequestId>D9479B4450D8CE18</RequestId>
<HostId>KvoMIkZLQfCPfgXOuT59wH0wAPVxvnEIOmmAVWuYQmH0sMHXUB/T62BiLcFaQJW07PcoS6+nnN0=</HostId></Error>
        
      
      sidenote, Conditional GET is supported by AWS S3
      From RFC, PUT with If-Match: * can continue processing
        if server does not have resource at the time.
        
          cf. RFC 7232,  2.3. ETag and 3.1 If-Match
          Riak CS or WM bug
        
      
  [ ] test_multipart_upload_empty
    
      The test expects 400 for MP complete with no parts, but succeeds
      Hmm.. AWS S3 refuses complate with no parts?
    
  
  [ ] test_list_buckets_invalid_auth
    
      Wrong error_code
    
    eq(e.error_code, 'InvalidAccessKeyId')
AssertionError: u'AccessDenied' != 'InvalidAccessKeyId'
    
  
  [ ] test_bucket_create_naming_dns_underscore
    
      underscore in bucket names are not allowd in AWS S3
      cf. http://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html
      Test comments reads: “test_bucket_create_naming_dns_* are valid but not recommended”
        What does it mean??? Intentional? S3 bahaves so?
    
  
  [ ] test_list_buckets_bad_auth
    
      Wrong error_code
    
    eq(e.error_code, 'SignatureDoesNotMatch')
AssertionError: u'AccessDenied' != 'SignatureDoesNotMatch'
    
  
  [x] test_put_object_ifnonmatch_overwrite_existed_failed
    
      It seems this test should fails by RFC 7232, why ‘fails_on_aws’?
        Does AWS S3 not support conditional PUT?
        
          Seems like so. PUT Object doc does not mention If-Match / If-None-Match
            but GET Object doc does.
          http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html
          http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectGET.html
        
      
      412 is returned as expected but no xml body with error code
        eq(e.error_code, 'PreconditionFailed')
AssertionError: None != 'PreconditionFailed'
        
      
  [x] test_bucket_list_delimiter_prefix
    
      RiakCS include NextMarker in the response without truncated results
      cf Unnecessary NextMarker in Get Bucket’s response if
        CommonPrefixes contain the last key. [JIRA: RCS-155] · Issue #1099
        basho/riak_cs#1099
      Don’t include NextMarker element when not truncated · basho/riak_cs@ec6b730
        https://github.com/basho/riak_cs/commit/ec6b730?w=1
    
  
  [ ] test_object_copy_to_itself
    
      400 expected but succeeds
      AWS response
    
  
<Error>
  <Code>InvalidRequest</Code>
  <Message>This copy request is illegal because it is trying to copy an object to itself without changing the object's metadata, storage class, website redirect location or encryption attributes.</Message>
  <RequestId>99A7D6F7FAB8E56E</RequestId>
  <HostId>/ttIGpA0NhqtbwWJ+QQ+9z81KBPFV7MVX0wCzV6TJ7s7XIG1Hs7795+3/ijFbl8yUHsLa41yr2g=</HostId>
</Error>


  [ ] Non-UTF8 byte in metadata
    
      For example, test_object_set_get_non_utf8_metadata fails at
        eq(got, '=?UTF-8?Q?=04mymeta?=')
        
        as
        AssertionError: u'\x04mymeta' != '=?UTF-8?Q?=04mymeta?='
        
      
      Hmm… Complicated f(^^;)
      for tests
        
          test_object_set_get_non_utf8_metadata
          test_object_set_get_metadata_empty_to_unreadable_prefix
          test_object_set_get_metadata_empty_to_unreadable_suffix
          test_object_set_get_metadata_empty_to_unreadable_infix
          test_object_set_get_metadata_overwrite_to_unreadable_prefix
          test_object_set_get_metadata_overwrite_to_unreadable_suffix
          test_object_set_get_metadata_overwrite_to_unreadable_infix
        
      
      These are tagged as below, just skipping them is not bad.
        @attr('fails_strict_rfc2616')
        
      
  [x] Tight assertion for Not Found
    
      Failes at eq(e.reason, 'Not Found')
        AssertionError: 'Object Not Found' != 'Not Found'
        
      
      Who returns Object Not Found ?
      Assertion in s3-tests is too restrictive or Riak CS should mimic the message
      for tests
        
          test_bucket_notexists
          test_bucket_delete_notexist
          test_object_write_to_nonexist_bucket
          test_bucket_create_delete
          test_object_read_notexist
          test_object_raw_get_bucket_gone
          test_object_delete_key_bucket_gone
          test_object_raw_get_object_gone
          test_object_raw_authenticated_bucket_gone
          test_object_raw_authenticated_bucket_gone
          test_object_raw_authenticated_object_gone
          test_abort_multipart_upload_not_found
          test_atomic_write_bucket_gone
        
      
s3-tests bugs


  Inproper stripping of double-quotes in etag value
    
      [x] test_put_object_ifnonmatch_failed
        
          Any ETag does not match, no precondition failure
        
      
      [x] test_s3.test_put_object_ifmatch_good
        
          s3-tests wrongly string double-quote characters for If-Match header
          cf: RFC 7232,  2.3. ETag and 3.1 If-Match
          quick fix
        
      
@@ -2132,5 +2132,5 @@ def test_put_object_ifmatch_good():
     eq(got_data, 'bar')
 
-    key.set_contents_from_string('zar', headers={'If-Match': key.etag.replace('"', '').strip()})
+    key.set_contents_from_string('zar', headers={'If-Match': key.etag.strip()})

Test that fail for Riak CS and also marked as “fails for AWS S3”


  test_list_buckets_anonymous
    
      403 Forbidden
      it has @attr of fails_on_aws, should be removed by command line
    
  
  test_put_object_ifnonmatch_nonexisted_good
    
      riak cs does not accept PUT Object with If-None-Match: * header
        for non-existing resource.
      marked as @attr('fails_on_aws')
      If-None-Match: * is for avoiding concurrent resource creation
      cf: RFC 7232, 3.2.  If-None-Match
    
  
  test_atomic_dual_conditional_write_1mb
    
      Very tricky test case with precondition race. Probably depending on
        boto implemenetation, when it reads file content / sends request.
      This test has @attr fails_on_aws, should be skipped.
    
  
  Bucket name validation
    
      S3 bucket name must be =< 63
      cf http://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html
      “Bucket names must be at least 3 and no more than 63 characters long.”
      These tests have @attr fails_with_subdomain, should be excluded
        
          test_bucket_create_naming_good_long_250
          test_bucket_create_naming_good_long_25[0-5]
          test_bucket_list_long_name
          test_bucket_create_naming_dns_long
          test_bucket_create_naming_dns_dash_at_end
          test_bucket_create_naming_dns_dot_dot
          test_bucket_create_naming_dns_dot_dash
          test_bucket_create_naming_dns_dash_dot
        
      
Riak CS Unimplemented APIs


  POST Object
    
      These tests have both @attr(resource‘object’)= and @attr(method‘post’)=
        
          test_post_object_anonymous_request
          test_post_object_authenticated_request
          test_post_object_set_success_code
          test_post_object_set_invalid_success_code
          test_post_object_upload_larger_than_chunk
          test_post_object_set_key_from_filename
          test_post_object_ignored_header
          test_post_object_case_insensitive_condition_fields
          test_post_object_escaped_field_values
          test_post_object_success_redirect_action
          test_post_object_invalid_date_format
          test_post_object_user_specified_header
          test_post_object_condition_is_case_sensitive
          test_post_object_expires_is_case_sensitive
          test_post_object_missing_expires_condition
          test_post_object_missing_conditions_list
          test_post_object_upload_size_limit_exceeded
          test_post_object_missing_content_length_argument
          test_post_object_invalid_content_length_argument
          test_post_object_upload_size_below_minimum
        
      
  CORS
    
      CORS is NYI
      These tests have not specific attr, just include cors as substring as
        @attr(operation='set cors')
        
        
          test_s3.test_set_cors
          test_s3.test_cors_origin_response
        
      
  Bucket Versioning
    
      405 Method Not Allowed for request to <bucket>/?versioning endpoint
      These tests have @attr versioning, should be excluded
        
          test_versioning_bucket_create_suspend
          test_versioning_obj_create_read_remove
          test_versioning_obj_create_read_remove_head
          test_versioning_obj_suspend_versions
          test_versioning_obj_suspend_versions_simple
          test_versioning_obj_create_versions_remove_all
          test_versioning_obj_create_overwrite_multipart
          test_s3.test_versioning_obj_list_marker
          test_s3.test_versioning_copy_obj_version
          test_s3.test_versioning_multi_object_delete
          test_s3.test_versioning_multi_object_delete_with_marker
          test_s3.test_versioning_multi_object_delete_with_marker_create
          test_s3.test_versioned_object_acl
          test_s3.test_versioned_concurrent_object_create_concurrent_remove
          test_s3.test_versioned_concurrent_object_create_and_remove
        
      
Not actual errors/failures


  test_multipart_upload_size_too_small
    
      This does not fail, but does if enforce_multipart_part_size=true.