Complete issue import API walkthrough with curl

complete-issue-api-walkthrough.md

This gist describes a new GitHub API for importing issues. This API hasn't been finalized yet, but when it is -- it will be announced on the official API blog.

Overview:

• General notes
• Create a repository
• Try to start an issue import with incomplete data
• Start an issue import
• Check status of issue import
• Check status of multiple issues
• Import fails because issue has an invalid milestone, assignee, creator or label
• Import fails because of unexpected comment creation error

General notes

To access the Issue Import API, you'll need to include application/vnd.github.golden-comet-preview in your request's Accept header. You'll also need to have admin permissions on the repository.

The maximum request size is 1MB.

Issues and comments created with this API do not trigger any notifications to mentioned users or users watching the repository into which issues are being imported. This is unlike the Issues API which does trigger notifications).

Like all other API requests, requests made to the Issue Import API are affected by regular API rate limits and abuse rate limits.

Issue imports started with this API are processed in the background asynchronously. When you start an import, you get back an import URL which you can use to check if the import completed, and whether it completed successfully or not. Because of this background processing, issue numbers are handed out to successfully imported issues once the processing is completed, and not when you start an import. As a result, if you use this API to submit two imports for issues A and B, and B is submitted after A, but before A's import completes, then it is possible that B will have a lower issue number than A. If you need to make sure that issues have specific numbers, then the recommended approach is that you start an import, wait for that import to complete and make sure it completed successfully, and only after that start a new import.

If you have any feedback or questions about the Issue Import API, please get in touch.

Create a repository

Create a repository using the web interface or the API.

Here is an example API request to create a repository:

curl -X POST -H "Authorization: token ${GITHUB_TOKEN}" \
  -d '{"name":"foo","private":true}' \
  https://api.github.com/user/repos

Try to start an issue import with incomplete data

The Issue Import API attempts to provide actionable feedback when a request does not meet validation criteria.

Request

curl -X POST -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  -d '{"issue":{"title":"My money, mo issues"}}' \
  https://api.github.com/repos/${GITHUB_USERNAME}/foo/import/issues

Response

{
  "message": "Validation Failed",
  "errors": [
    {
      "resource": "Issue",
      "code": "missing_field",
      "field": "body"
    }
  ],
  "documentation_url": "https://developer.github.com/v3"
}

Start an issue import

Just make a post request to the import issues API endpoint for the repository and include the issue and comments json as data. An issue only requires the following fields:

{
  "issue": {
    "title": "Imported from some other system",
    "body": "..."
  }
}

And an issue with a comment only needs the comment body added:

{
  "issue": {
    "title": "Imported from some other system",
    "body": "..."
  },
  "comments": [
    {
      "body": "talk talk"
    }
  ]
}

Request

curl -X POST -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  -d '{"issue":{"title":"Fix broken widgets","body":"- [ ] widget 1\n- [ ] widget 2","created_at":"2014-03-16T18:21:16Z"},"comments":[{"body":"Can we wrap this up soon?","created_at":"2014-03-16T17:15:42Z"}]}' \
  https://api.github.com/repos/${GITHUB_USERNAME}/foo/import/issues

Response

{
  "id": 3,
  "status": "pending",
  "url": "https://api.github.com/repos/jonmagic/foo/import/issues/3",
  "import_issues_url": "https://api.github.com/repos/jonmagic/foo/import/issues",
  "repository_url": "https://api.github.com/repos/jonmagic/foo"
}

Supported issue and comment fields

The complete json schema looks like this:

{
  "issue": {
    "title": "Imported from some other system",
    "body": "...",
    "created_at": "2014-01-01T12:34:58Z",
    "closed_at": "2014-01-02T12:24:56Z",
    "updated_at": "2014-01-03T11:34:53Z",
    "assignee": "jonmagic",
    "milestone": 1,
    "closed": true,
    "labels": [
      "bug",
      "low"
    ]
  },
  "comments": [
    {
      "created_at": "2014-01-02T12:34:56Z",
      "body": "talk talk"
    }
  ]
}

Note: The body of an issue or comment will be parsed and rendered using Markdown if the created_at value is set to a time after 2009-04-20T19:00:00Z, and using Textile if it's older. The body may be at most 65535 characters long. As of 2016-03-15 all newly created issues will be rendered with Markdown regardless of the created_at timestamp date.

Check status of issue import

After making the API request to import an issue you can check the status of the import by making a get request to the url value from the response in the previous post request.

Request

curl -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  https://api.github.com/repos/#{GITHUB_USERNAME}/foo/import/issues/3

Response

{
  "id": 3,
  "status": "imported",
  "url": "https://api.github.com/repos/jonmagic/foo/import/issues/3",
  "import_issues_url": "https://api.github.com/repos/jonmagic/foo/import/issues",
  "repository_url": "https://api.github.com/repos/jonmagic/foo",
  "issue_url": "https://api.github.com/repos/jonmagic/foo/issues/1"
}

Check status of multiple issues

You can check the status of issues imported since a date or date and time.

Request

curl -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  https://api.github.com/repos/#{GITHUB_USERNAME}/foo/import/issues?since=2015-03-15

Response

[
  {
    "id": 3,
    "status": "imported",
    "url": "https://api.github.com/repos/jonmagic/foo/import/issues/3",
    "import_issues_url": "https://api.github.com/repos/jonmagic/foo/import/issues",
    "repository_url": "https://api.github.com/repos/jonmagic/foo",
    "issue_url": "https://api.github.com/repos/jonmagic/foo/issues/1"
  }
]

Import fails because issue has an invalid milestone, assignee, creator or label

What happens if you try to import an issue with an invalid attributes?

Import issue request

curl -X POST -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  -d '{"issue":{"title":"Fix broken widgets","body":"...","milestone":5,
      "assignee":"bobbyfoobar","labels":["bug","invalid,label"]}}' \
  https://api.github.com/repos/${GITHUB_USERNAME}/foo/import/issues

Import issue response

{
  "id": 7,
  "status": "pending",
  "url": "https://api.github.com/repos/jonmagic/foo/import/issues/7",
  "import_issues_url": "https://api.github.com/repos/jonmagic/foo/import/issues",
  "repository_url": "https://api.github.com/repos/jonmagic/foo"
}

Import status request

curl -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  https://api.github.com/repos/#{GITHUB_USERNAME}/foo/import/issues/7

Import status response

{
  "id": 7,
  "status": "failed",
  "url": "https://api.github.com/repos/jonmagic/foo/import/issues/7",
  "import_issues_url": "https://api.github.com/repos/jonmagic/foo/import/issues",
  "repository_url": "https://api.github.com/repos/jonmagic/foo",
  "created_at": "2015-03-18T19:40:53-07:00",
  "updated_at": "2015-03-18T19:40:58-07:00",
  "errors": [
    {
      "location": "/issue/milestone",
      "resource": "Issue",
      "field": "milestone",
      "value": "5",
      "code": "invalid"
    },
    {
      "location": "/issue/assignee",
      "resource": "Issue",
      "field": "assignee",
      "value": "bobbyfoobar",
      "code": "invalid"
    },
    {
      "location": "/issue/labels[1]",
      "resource": "Label",
      "field": "name",
      "value": "invalid,label",
      "code": "invalid"
    }
  ]
}

Import fails because of unexpected comment creation error

Import issue request

curl -X POST -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  -d '{"issue":{"title":"foo","body":"1"},"comments":[{"body":"fail this comment somehow"}]}' \
  https://api.github.com/repos/${GITHUB_USERNAME}/foo/import/issues

Import issue response

{
  "id": 12,
  "status": "pending",
  "url": "https://api.github.com/repos/jonmagic/foo/import/issues/12",
  "import_issues_url": "https://api.github.com/repos/jonmagic/foo/import/issues",
  "repository_url": "https://api.github.com/repos/jonmagic/foo",
  "created_at": "2015-03-18T21:45:58-07:00",
}

Import status request

curl -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  https://api.github.com/repos/#{GITHUB_USERNAME}/foo/import/issues/12

Import status response

{
  "id": 12,
  "status": "failed",
  "url": "https://api.github.com/repos/jonmagic/foo/import/issues/12",
  "import_issues_url": "https://api.github.com/repos/jonmagic/foo/import/issues",
  "repository_url": "https://api.github.com/repos/jonmagic/foo",
  "created_at": "2015-03-18T21:45:58-07:00",
  "updated_at": "2015-03-18T21:46:05-07:00",
  "issue_url": "https://api.github.com/repos/jonmagic/foo/issues/7",
  "errors": [
    {
      "location": "/comments[0]",
      "resource": "IssueComment",
      "field": null,
      "value": null,
      "code": "error"
    }
  ]
}

The problem in this API is then that the owner of the token will be used for both the issue and all comments, which practically limits you to use a fictional user created for this.

Has there been any evolution on this ?

Just working with the new issues importing API and noticed that it’s not possible to set the author of a comment. All comments are imported using the username of the importer.

It would be great to be able to specify the author of a comment, like we can with the regular API.

Let me go ahead and third or fourth this. Given that the authorized user is an admin for the repo, even if comment authors should be limited to real accounts and not fictionalized, why not require access tokens for any accounts to be posted under and allow the specification of a comment author and token? Or at least allow the contribution of comments with dates for imported issues by additional requests using this API. This limitation basically forces us to utilize the standard Issues API for migrating issues, where we lose our original dates but maintain our team’s commenter/creator identities. With the “import” API, we can preserve dates but we lose our identities. It’s a bit frustrating.

UPDATE: Fixed

It seems the API is currently broken when importing a closed issue.

Importing closed issue: error

echo '{ "issue": { "title": "foo", "body": "bar", "closed": true } }' > issue.json
curl -X POST -H "Authorization: token ${token}" -H "Accept: application/vnd.github.golden-comet-preview+json" -d @issue.json https://api.github.com/repositories/190352707/import/issues
{
  "id": 2453079,
  "status": "pending",
  "url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues/2453079",
  "import_issues_url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues",
  "repository_url": "https://api.github.com/repos/stmllr/test-restored-6625",
  "created_at": "2019-06-05T09:13:24.000Z",
  "updated_at": "2019-06-05T09:13:24.000Z"
}
curl -H GET -H "Authorization: token ${token}" -H "Accept: application/vnd.github.golden-comet-preview+json" https://api.github.com/repositories/190352707/import/issues/2453079
{
  "id": 2453079,
  "status": "failed",
  "url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues/2453079",
  "import_issues_url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues",
  "repository_url": "https://api.github.com/repos/stmllr/test-restored-6625",
  "created_at": "2019-06-05T09:13:24.000Z",
  "updated_at": "2019-06-05T09:13:24.000Z",
  "issue_url": "https://api.github.com/repos/stmllr/test-restored-6625/issues/15",
  "errors": [
    {
      "location": "/issue",
      "resource": "Issue",
      "field": null,
      "value": null,
      "code": "error"
    }
  ]
}

The issue was created in the target repository, but state is open and status check returns "error".
Setting closed_at date did not change the behavior.

Importing open issue

If I set closed to false in the import issue, status check has no errors:

echo '{ "issue": { "title": "foo", "body": "bar", "closed": false } }' > issue.json
curl -X POST -H "Authorization: token ${token}" -H "Accept: application/vnd.github.golden-comet-preview+json" -d @issue.json https://api.github.com/repositories/190352707/import/issues
{
  "id": 2453082,
  "status": "pending",
  "url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues/2453082",
  "import_issues_url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues",
  "repository_url": "https://api.github.com/repos/stmllr/test-restored-6625",
  "created_at": "2019-06-05T09:19:13.000Z",
  "updated_at": "2019-06-05T09:19:13.000Z"
}
curl -H GET -H "Authorization: token ${token}" -H "Accept: application/vnd.github.golden-comet-preview+json" https://api.github.com/repositories/190352707/import/issues/2453082
{
  "id": 2453082,
  "status": "imported",
  "url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues/2453082",
  "import_issues_url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues",
  "repository_url": "https://api.github.com/repos/stmllr/test-restored-6625",
  "created_at": "2019-06-05T09:19:13.000Z",
  "updated_at": "2019-06-05T09:19:13.000Z",
  "issue_url": "https://api.github.com/repos/stmllr/test-restored-6625/issues/16"
}

BUG: If you add labels to the import, the time stamp on the event is now. This doesn't reflect last modified luckily but ideally the time stamp would conform to the creation date of the case or something?

Is there a way to import an issue with multiple assignees?

@jonmagic thank you for sharing this gist!

Issues and comments created with this API do not trigger any notifications to mentioned users or users watching the repository into which issues are being imported.

Is this still valid? We are planning to use this API to bulk import issues from a legacy issue tracker and we were wondering if watchers of the repo will get notified for each issue created/modified during the process.

Thank you upfront for your feedback.

Please contact support@github.com for latest best practices for imports. Thanks!

ok, thank you for your quick reply!

If anybody else comes across this gist be aware that emails to support@github.com now result in a request to submit requests via the web at https://support.github.com/

Is it possible to control the ID assigned to an imported issue? Or, at least, as a safeguard tell the API that the issue shall not be created if the assigned ID is not a given number that I'd expect?

Is it possible to control the ID assigned to an imported issue? Or, at least, as a safeguard tell the API that the issue shall not be created if the assigned ID is not a given number that I'd expect?

Afaik that is not possible and this API was never prioritized to be maintained and updated. As mentioned above submitting a request through https://support.github.com is the best way to seek help/give feedback. Thank you 🙏

I did. Contacted GitHub support, describing the task at hand and they referred me here, with no further information or updates to add.

This is an unofficial response from me personally:

We created this API specifically for https://opensource.googleblog.com/2015/03/farewell-to-google-code.html and then left it there for folks to continue using as is. The team that maintained it (my old team) was disbanded during a re-org in early '16 and the API never found a new owner.

I work in a different part of the company now and don't have visibility into it's future but I doubt improvements to this API will ever get prioritized and more likely it will be removed at some point.

Probably not the answer you want but the best I can give based on what I know.

Here's a little "hack" which might work for you:

1. Find the greatest issue number n from your import issue data.
2. Create n consecutive issues using the import issue API, use fake issue if you don't need the number, or a "real" issue if it matches one of your desired issue numbers.
3. Delete the fake issue afterwards.

This will only work once!

💚 to Jonathan for the above statement and 👍 to Steffen for the suggestion.

It's interesting to see that there is an API variant still being exposed (and functional as of today) that allows some more freedom over the date etc. of issues created, with apparently no team and responsibility assigned to it.

I have been trying to use this on an organization, after successfully testing this api on a repo created by my test user.

It fails with the result

{"documentation_url":"https:\/\/docs.github.com\/rest","message":"Not Found"}

I don't know how to figure this one out, I triple checked the org and repo name, but no success.

Does anybody have any idea?

@burner do you solve it?
i thought that github-csv-tools was broken, but with the curl example above and Thunder Client i get the same error

I re-tried a few days later and it worked. I don't know why. Result can be seen here https://github.com/dlang/visuald/issues

Thanks. I solve it using the GitHub CLI command of
https://docs.github.com/en/rest/issues/issues?apiVersion=2022-11-28#create-an-issue

Maybe curl request for new tokens are disable the first 24hr?