Both the schema below have the following properties
- Crash-consistent during a completeMultipartUpload/abortMultipartUpload
- Concurrency-safe in presence of multiple concurrent uploads
These properties allows us to avoid fcntl(3) based locking in shared mode with FS backend.
.minio.sys.tmp
├── <uploadId> -----------------> created on initMultipartUpload
├── <eTag>.0 ---------------> created on initMultipartUpload
├── <eTag>.<partNumber> ----> created on putObjectPart
where,
<uploadId>/<eTag>.0
contains the following json object
{
"Bucket": "bucketName",
"Object": "objectName"
}
The following example contains 3 parts each of 2 concurrent uploads with uploadId
6e463bb8-35bd-4408-809e-78f509f558b3
7bed54f8-ad71-48b1-a4f8-bd2e2a8efbda
to mybucket/myobject
.
.minio.sys.tmp
├── 6e463bb8-35bd-4408-809e-78f509f558b3
│ ├── 467886be95c8ecfd71a2900e3f461b4f.0
│ ├── 467886be95c8ecfd71a2900e3f461b4f.1
│ ├── 467886be95c8ecfd71a2900e3f461b4f.2
│ └── 467886be95c8ecfd71a2900e3f461b4f.3
└── 7bed54f8-ad71-48b1-a4f8-bd2e2a8efbda
├── 467886be95c8ecfd71a2900e3f461b4f.0
├── 467886be95c8ecfd71a2900e3f461b4f.1
├── 467886be95c8ecfd71a2900e3f461b4f.2
└── 467886be95c8ecfd71a2900e3f461b4f.3
6e463bb8-35bd-4408-809e-78f509f558b3/467886be95c8ecfd71a2900e3f461b4f.0
would contain
{
"Bucket": "mybucket",
"Object": "myobject"
}
- Easier to support/debug since uploadId directory will have at most 10000 entries
- uploadId modTime will reflect last updated part for a given upload; simplifies detection of 'stale' uploads
- Limited number of entries per uploadId directory; listing during cleanup of stale upload parts will be faster
.minio.sys.tmp
├── <uploadId>.<eTag>.0 ---------------> created on initMultipartUpload
├── <uploadId>.<eTag>.<partNumber> ----> created on putObjectPart
where,
<uploadId>.<eTag>.0
contains the following json object
{
"Bucket": "bucketName",
"Object": "objectName"
}
The following example contains 3 parts each of 2 concurrent uploads with uploadId
6e463bb8-35bd-4408-809e-78f509f558b3
7bed54f8-ad71-48b1-a4f8-bd2e2a8efbda
to mybucket/myobject
.
.minio.sys.tmp
├── 6e463bb8-35bd-4408-809e-78f509f558b3.467886be95c8ecfd71a2900e3f461b4f.0
├── 6e463bb8-35bd-4408-809e-78f509f558b3.467886be95c8ecfd71a2900e3f461b4f.1
├── 6e463bb8-35bd-4408-809e-78f509f558b3.467886be95c8ecfd71a2900e3f461b4f.2
├── 6e463bb8-35bd-4408-809e-78f509f558b3.467886be95c8ecfd71a2900e3f461b4f.3
├── 7bed54f8-ad71-48b1-a4f8-bd2e2a8efbda.467886be95c8ecfd71a2900e3f461b4f.0
├── 7bed54f8-ad71-48b1-a4f8-bd2e2a8efbda.467886be95c8ecfd71a2900e3f461b4f.1
├── 7bed54f8-ad71-48b1-a4f8-bd2e2a8efbda.467886be95c8ecfd71a2900e3f461b4f.2
└── 7bed54f8-ad71-48b1-a4f8-bd2e2a8efbda.467886be95c8ecfd71a2900e3f461b4f.3
6e463bb8-35bd-4408-809e-78f509f558b3.467886be95c8ecfd71a2900e3f461b4f.0
contains
{
"Bucket": "bucketName",
"Object": "objectName"
}
- Single directory to hold all ongoing uploads including single PUT object
- Maximum object name supportable is limited by platform-specific path segment length limits. In GNU/Linux it's 255.
<bucket-name>
and<object-name>
embedded into part name limits object name length.<upload-id>
directory in schema-1 and its not clear about in schema-2