Both the schema below have the following properties
- Crash-consistent during a completeMultipartUpload/abortMultipartUpload
- Concurrency-safe in presence of multiple concurrent uploads
These properties allows us to avoid fcntl(3) based locking in shared mode with FS backend.
.minio.sys.tmp
├── <uploadId> -----------------> created on initMultipartUpload
├── <eTag>.0 ---------------> created on initMultipartUpload
├── <eTag>.<partNumber> ----> created on putObjectPart
where,
<uploadId>/<eTag>.0
contains the following json object
{
"Bucket": "bucketName",
"Object": "objectName"
}
The following example contains 3 parts each of 2 concurrent uploads with uploadId
6e463bb8-35bd-4408-809e-78f509f558b3
7bed54f8-ad71-48b1-a4f8-bd2e2a8efbda
to mybucket/myobject
.
.minio.sys.tmp
├── 6e463bb8-35bd-4408-809e-78f509f558b3
│ ├── 467886be95c8ecfd71a2900e3f461b4f.0
│ ├── 467886be95c8ecfd71a2900e3f461b4f.1
│ ├── 467886be95c8ecfd71a2900e3f461b4f.2
│ └── 467886be95c8ecfd71a2900e3f461b4f.3
└── 7bed54f8-ad71-48b1-a4f8-bd2e2a8efbda
├── 467886be95c8ecfd71a2900e3f461b4f.0
├── 467886be95c8ecfd71a2900e3f461b4f.1
├── 467886be95c8ecfd71a2900e3f461b4f.2
└── 467886be95c8ecfd71a2900e3f461b4f.3
6e463bb8-35bd-4408-809e-78f509f558b3/467886be95c8ecfd71a2900e3f461b4f.0
would contain
{
"Bucket": "mybucket",
"Object": "myobject"
}
- Easier to support/debug since uploadId directory will have at most 10000 entries
- uploadId modTime will reflect last updated part for a given upload; simplifies detection of 'stale' uploads
- Limited number of entries per uploadId directory; listing during cleanup of stale upload parts will be faster
.minio.sys.tmp
├── <uploadId>.<eTag>.0 ---------------> created on initMultipartUpload
├── <uploadId>.<eTag>.<partNumber> ----> created on putObjectPart
where,
<uploadId>.<eTag>.0
contains the following json object
{
"Bucket": "bucketName",
"Object": "objectName"
}
The following example contains 3 parts each of 2 concurrent uploads with uploadId
6e463bb8-35bd-4408-809e-78f509f558b3
7bed54f8-ad71-48b1-a4f8-bd2e2a8efbda
to mybucket/myobject
.
.minio.sys.tmp
├── 6e463bb8-35bd-4408-809e-78f509f558b3.467886be95c8ecfd71a2900e3f461b4f.0
├── 6e463bb8-35bd-4408-809e-78f509f558b3.467886be95c8ecfd71a2900e3f461b4f.1
├── 6e463bb8-35bd-4408-809e-78f509f558b3.467886be95c8ecfd71a2900e3f461b4f.2
├── 6e463bb8-35bd-4408-809e-78f509f558b3.467886be95c8ecfd71a2900e3f461b4f.3
├── 7bed54f8-ad71-48b1-a4f8-bd2e2a8efbda.467886be95c8ecfd71a2900e3f461b4f.0
├── 7bed54f8-ad71-48b1-a4f8-bd2e2a8efbda.467886be95c8ecfd71a2900e3f461b4f.1
├── 7bed54f8-ad71-48b1-a4f8-bd2e2a8efbda.467886be95c8ecfd71a2900e3f461b4f.2
└── 7bed54f8-ad71-48b1-a4f8-bd2e2a8efbda.467886be95c8ecfd71a2900e3f461b4f.3
6e463bb8-35bd-4408-809e-78f509f558b3.467886be95c8ecfd71a2900e3f461b4f.0
contains
{
"Bucket": "bucketName",
"Object": "objectName"
}
- Single directory to hold all ongoing uploads including single PUT object
- Maximum object name supportable is limited by platform-specific path segment length limits. In GNU/Linux it's 255.
Here are some cases of interest and what happens in the proposed multipart backend.
[N B Background append process need to be rewritten to not use fs.json to track parts uploaded so far.]
When a completeMultipartUpload request and abortMultipartUpload request arrives at different minio instances around the same time.
In this case it is possible for both these requests to return success. This is incorrect.
When two or more completeMultipartUpload requests arrive at different minio instances around the same time with different parts to be committed.
In this case the "last (successful) writer" wins, others may fail while appending parts since a successful completeMultipartUpload removes the parts.
When a completeMultipartUpload and a putObjectPart request arrive at different minio instances around the same time.
There are two possibilties,
a. the part uploaded is present in the completeMultipartUpload request
- if putObjectPart didn't complete on time, completeMultipartUpload would fail with part missing error.
b. the part uploaded isn't present in the completeMultipartUpload request
- in this case the part uploaded doesn't affect the result of completeMultipartUpload. It is possible that the uploadId directory and this part alone is not cleaned up.