Skip to content

Instantly share code, notes, and snippets.

@krishnasrinivas
Last active April 25, 2017 22:30
Show Gist options
  • Save krishnasrinivas/4b216a6ec03925645253788d8e0e241d to your computer and use it in GitHub Desktop.
Save krishnasrinivas/4b216a6ec03925645253788d8e0e241d to your computer and use it in GitHub Desktop.

Took inspirtion from DigitalOcean REST APIs: https://developers.digitalocean.com/documentation/v2/

Convention:

  • Use GET, POST, DELETE http verbs. No case for PUT.
  • Use JSON request/responses.
  • Auth happens using S3V4
  • if a boolean query paramenter is passed without value then the value is assumed to be true. For ex ?recursive is same as ?recursive=true

API Categories

Version, Service, Locks, Heal, Config

Version

(Note that the request endpoint is not /minio/admin/v1/version)

Request: GET /minio/admin/version

Response Body:

{
   version: "1"
}

Client can use this endpoint to figure what REST API version the server is running so that it can use appropriate madmin version.

Service


Restart

Request: POST /minio/admin/v1/service

Request Body:

{
   action: restart
}

Response Body: EMPTY

Stop

Restart

Request: POST /minio/admin/v1/service

Request Body: EMPTY

{
   action: stop
}

Response Body: EMPTY

Status

Request : GET /minio/admin/v1/service

Request Body: EMPTY

Response Body:

   {
      "status": "stop",
      "service": true,
      "uptime": 121369476533489
   }

Locks


List locks

Request : GET /minio/admin/v1/locks?bucket=my-bucket&prefix=my-prefix&older-than=1231141

older-than is in seconds

Request Body: EMPTY

Response Body:

{
  "bucket": "my-bucket",
  "object": "my-prefix",
  "readLocks": 0,
  "writeLocks": 1,
  "lockOwners": [
    {
      "id": "5236849b-9ef5-4258-93aa-6b6d7c57dcf1",
      "source": "[web-handlers.go:284:(*webAPIHandlers).RemoveObject.func1()] LockType:WLock Status:Blocked Since:2017-03-22 04:40:37.031610943 +0000 UTC Duration:63h58m31.901491283s}",
      "type": "WLock",
      "status": "Blocked",
      "since": "2017-03-22 04:25:43.415475148 +0000 UTC",
      "duration": 64h13m25.517627078s
    }
  ]
}

Clear locks

Request: DELETE /minio/admin/v1/locks?bucket=my-bucket&prefix=my-prefix&older-than=1231141

older-than is in seconds

Request Body: EMPTY

Response Body:

{
  "bucket": "my-bucket",
  "object": "my-prefix",
  "readLocks": 0,
  "writeLocks": 1,
  "lockOwners": [
    {
      "id": "5236849b-9ef5-4258-93aa-6b6d7c57dcf1",
      "source": "[web-handlers.go:284:(*webAPIHandlers).RemoveObject.func1()] LockType:WLock Status:Blocked Since:2017-03-22 04:40:37.031610943 +0000 UTC Duration:63h58m31.901491283s}",
      "type": "WLock",
      "status": "Blocked",
      "since": "2017-03-22 04:25:43.415475148 +0000 UTC",
      "duration": 64h13m25.517627078s
    }
  ]
}

Heal

Heal initiate

Request: POST /minio/admin/v1/heal

Request body: { path: "testbucket/a/b" recursive: true, advanced: true, fake: true, healMultipart: true, remove: true }

Description:

  • path: bucket + prefix path
  • recursive: heal recursively
  • advanced: check bit rot
  • fake: dry run
  • healMultipart: heal uncommitted objects
  • remove: remove objects that can not be recovered

Response Body:

{
  healTaskID: "testbucket/a/b"
}

More detailed explanation: https://gist.github.com/krishnasrinivas/615eb3d1428c7eb5a8c7fa059ed199a2

Heal status

Request: GET /minio/admin/v1/heal/healTaskID

Example: GET /minio/admin/v1/heal/testbucket/a/b

Response Body:

{
  status: "running",
  objects: [
    {type: "minioMeta", metaFile: "format.json", disks: [1,2,3,4,5,6,7,8]},
    {type: "bucket", bucketName: "testbucket", disks: [1,2,3,4,5,6,7,8]},
    {type: "object", bucketName: "testbucket", objectName: "objectname1", disks: [1,2,4,6,7,8],
    {type: "object", bucketName: "testbucket", objectName: "objectname2", disks: [1,2,4,6,7,8],
    {type: "object", bucketName: "testbucket", objectName: "objectname3", disks: [1,2,4,6,7,8],
    {type: "multipart", bucketName: "testbucket", objectName: "objectname4", disks: [1,2,4,6,7,8],
    {type: "multipart", bucketName: "testbucket", objectName: "objectname5", disks: [1,2,4,6,7,8],
  ]
}

The list of objects healed is kept in a circular linked list of fixed length (FIFO).

After the heal task completes, the status will be "completed" and some information is kept in memory till a timeout so that clients can query the state of the heal task.

Heal Terminate

DELETE /minio/admin/v1/heal/healTaskID

Example: DELETE /minio/admin/v1/heal/testbucket/a/b

If the heal task is running then it will terminate it. If the heal task is completed then its reference is removed from the memory.

Config

Get Config

Request: GET /minio/admin/v1/config

Request Body: EMPTY

Response Body:

{
    //config.json
}

Set Config

Request: POST /minio/admin/v1/config

Request body:

{
    //config.json
}

Response Body: EMPTY

Set credential

Request: POST /minio/admin/v1/config/credential

Request body:

{
    accessKey: ACCESS_KEY,
    secretKey: SECRET_KEY
}

Response Body: EMPTY

@krisis
Copy link

krisis commented Apr 18, 2017

@krishnasrinivas What happens when a client knows to communicate only in version 1 protocol but the server is upgraded to communicate in version 2? Version API call would return version 2 as explained above. Some APIs may have difference in behavior or number of arguments passed. How does the client cope with version changes?

@vadmeste
Copy link

Will authorization mechanism be the same?

@vadmeste
Copy link

Is there any arguments why we are changing the API in the first place ?

@krishnasrinivas
Copy link
Author

@krisis when we move from version 1 to 2 on the server, we have two options, either support both version 1 and 2 or just support 2 and return error when a client sends request using v1. @abperiasamy said we can just return error to outdated clients. But madmin will itself support both v1 and v2, hence mc will support v1 and v2.

Will authorization mechanism be the same?

@vadmeste auth is S3V4

Is there any arguments why we are changing the API in the first place ?

Usually there is a REST API endpoint and query-params to customize it.
For ex. GET /minio/admin/v1/locks?older-than=1234 is an intuitive API.
But GET /?lock with x-minio-operation: list header is not an intuitive API. When you want to pass query params it becomes more un-intuitive for ex GET /?lock&older-than=1234
For other APIs when we use x-minio-operation: bucket and x-minio-operation: object for heal. x-minio-operation indicates that the value is a verb but bucket/object is not a verb.

@vadmeste
Copy link

Usually there is a REST API endpoint and query-params to customize it.
For ex. GET /minio/admin/v1/locks?older-than=1234 is an intuitive API.
But GET /?lock with x-minio-operation: list header is not an intuitive API. When you want to pass query params it becomes more un-intuitive for ex GET /?lock&older-than=1234
For other APIs when we use x-minio-operation: bucket and x-minio-operation: object for heal. x-minio-operation indicates that the value is a verb but bucket/object is not a verb.

👍

@donatello
Copy link

List/Clear Locks:

  1. older-than parameter's unit needs to be fixed clearly for the API spec.
  2. The returned body echoes the bucket and prefix, but not the older-than parameter. This should also be echoed back as good practice and because it makes the response self-contained.

@krisis
Copy link

krisis commented Apr 18, 2017

{
healMinioMeta: true,
healBucket: true,
recursive: true,
advanced: true,
fake: true,
healObject: true,
healMultipart: true,
remove: true,
bucketName: "testbucket",
objectPrefix: "testprefix/a/b"
}

what is advanced here? It's too generic.

@krisis
Copy link

krisis commented Apr 18, 2017

{type: "minioMeta", metaFile: "format.json", disks: [1,2,3,4,5,6,7,8]},
{type: "bucket", bucketName: "testbucket", disks: [1,2,3,4,5,6,7,8]},

How would admin SDK users or authors use metaFile and disks? Is there a use-case?

@krishnasrinivas
Copy link
Author

@donatello

The returned body echoes the bucket and prefix, but not the older-than parameter. This should also be echoed back as good practice and because it makes the response self-contained.

bucketName is part of the response so that in case the request was "heal all objects in all buckets" we have a way to differentiate on which bucket the object that was healed belongs to. Having older-than is not useful because it applies to all the responses.

@krisis

what is advanced here? It's too generic.

it's documented - advanced means checking bit-rot (for now, can be extended to more things in future if need be)

How would admin SDK users or authors use metaFile and disks? Is there a use-case?

Just to output to the UI/console on the heal event.

@krisis
Copy link

krisis commented Apr 22, 2017

How would admin SDK users or authors use metaFile and disks? Is there a use-case?

Just to output to the UI/console on the heal event.

This kind of disk-level specifics was asked to be omitted by AB, see minio/minio#4045 (comment).

@krisis
Copy link

krisis commented Apr 22, 2017

  "since": "2017-03-22 04:25:43.415475148 +0000 UTC",
  "duration": 64h13m25.517627078s

@krishnasrinivas, AB was not in favour of having both since and duration as part of API response. Has that changed?

@krisis
Copy link

krisis commented Apr 22, 2017

Having older-than is not useful because it applies to all the responses.

But it is still different for requests with different older-than values, right? It tells the caller that the locks listed or cleared have been held for longer than the duration in older-than parameter. In that sense it is useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment