move-issues-from-jira-to-gitlab.py

import requests
from requests.auth import HTTPBasicAuth
import re
from StringIO import StringIO

JIRA_URL = 'https://your-jira-url.tld/'
JIRA_ACCOUNT = ('jira-username', 'jira-password')
# the JIRA project ID (short)
JIRA_PROJECT = 'PRO'
GITLAB_URL = 'http://your-gitlab-url.tld/'
# this is needed for importing attachments. The script will login to gitlab under the hood.
GITLAB_ACCOUNT = ('gitlab-username', 'gitlab-password')
# this token will be used whenever the API is invoked and
# the script will be unable to match the jira's author of the comment / attachment / issue
# this identity will be used instead.
GITLAB_TOKEN = 'get-this-token-from-your-profile'
# the project in gitlab that you are importing issues to.
GITLAB_PROJECT = 'namespaced/project/name'
# the numeric project ID. If you don't know it, the script will search for it
# based on the project name.
GITLAB_PROJECT_ID = None
# set this to false if JIRA / Gitlab is using self-signed certificate.
VERIFY_SSL_CERTIFICATE = False

# IMPORTANT !!!
# make sure that user (in gitlab) has access to the project you are trying to
# import into. Otherwise the API request will fail.
GITLAB_USER_TOKENS = {
    'jira-username': 'gitlab-private-token-for-this-user',
}

RE_TOKEN = "<meta content=\"(?P<token>.*)\" name=\"csrf-token\""

jira_issues = requests.get(
    JIRA_URL + 'rest/api/2/search?jql=project=%s+AND+resolution=Unresolved+ORDER+BY+priority+DESC&maxResults=10000' % JIRA_PROJECT,
    auth=HTTPBasicAuth(*JIRA_ACCOUNT),
    verify=VERIFY_SSL_CERTIFICATE,
    headers={'Content-Type': 'application/json'}
)


if not GITLAB_PROJECT_ID:
    # find out the ID of the project.
    for project in requests.get(
        GITLAB_URL + 'api/v3/projects',
        headers={'PRIVATE-TOKEN': GITLAB_TOKEN},
        verify=VERIFY_SSL_CERTIFICATE
    ).json():
        if project['path_with_namespace'] == GITLAB_PROJECT:
            GITLAB_PROJECT_ID = project['id']
            break

if not GITLAB_PROJECT_ID:
    raise Exception("Unable to find %s in gitlab!" % GITLAB_PROJECT)

for issue in jira_issues.json()['issues']:

    reporter = issue['fields']['reporter']['name']

    gl_issue = requests.post(
        GITLAB_URL + 'api/v3/projects/%s/issues' % GITLAB_PROJECT_ID,
        headers={'PRIVATE-TOKEN': GITLAB_USER_TOKENS.get(reporter, GITLAB_TOKEN)},
        verify=VERIFY_SSL_CERTIFICATE,
        data={
            'title': issue['fields']['summary'],
            'description': issue['fields']['description']
        }
    ).json()['id']

    # get comments and attachments
    issue_info = requests.get(
        JIRA_URL + 'rest/api/2/issue/%s/?fields=attachment,comment' % issue['id'],
        auth=HTTPBasicAuth(*JIRA_ACCOUNT),
        verify=VERIFY_SSL_CERTIFICATE,
        headers={'Content-Type': 'application/json'}
    ).json()

    for comment in issue_info['fields']['comment']['comments']:
        author = comment['author']['name']

        note_add = requests.post(
            GITLAB_URL + 'api/v3/projects/%s/issues/%s/notes' % (GITLAB_PROJECT_ID, gl_issue),
            headers={'PRIVATE-TOKEN': GITLAB_USER_TOKENS.get(author, GITLAB_TOKEN)},
            verify=VERIFY_SSL_CERTIFICATE,
            data={
                'body': comment['body']
            }
        )

    if len(issue_info['fields']['attachment']):
        # !!! HACK !!! obtain a session to gitlab in order to get a secret csrftoken
        with requests.Session() as s:
            token = re.search(
                RE_TOKEN,
                s.get(
                    GITLAB_URL + 'users/sign_in',
                    verify=VERIFY_SSL_CERTIFICATE
                ).content
            ).group('token')

            signin = s.post(
                GITLAB_URL + 'users/sign_in',
                headers={
                    "Referer": GITLAB_URL
                },
                verify=VERIFY_SSL_CERTIFICATE,
                data={
                    'authenticity_token': token,
                    'user[login]': GITLAB_ACCOUNT[0],
                    'user[password]': GITLAB_ACCOUNT[1],
                    'user[remember_me]': 0
                }
            )

            html = s.get(
                GITLAB_URL + '%s/issues/%s' % (GITLAB_PROJECT, gl_issue),
                verify=VERIFY_SSL_CERTIFICATE
            ).content

            token = re.search(RE_TOKEN, html).group('token')

            for attachment in issue_info['fields']['attachment']:
                author = attachment['author']['name']

                _file = requests.get(
                    attachment['content'],
                    auth=HTTPBasicAuth(*JIRA_ACCOUNT),
                    verify=VERIFY_SSL_CERTIFICATE,
                )

                _content = StringIO(_file.content)

                file_info = s.post(
                    GITLAB_URL + '%s/uploads' % GITLAB_PROJECT,
                    headers={
                        'X-CSRF-Token': token,
                    },
                    files={
                        'file': (
                            attachment['filename'],
                            _content
                        )
                    },
                    verify=VERIFY_SSL_CERTIFICATE
                )

                del _content

                # now we got the upload URL. Let's post the comment with an
                # attachment

                requests.post(
                    GITLAB_URL + 'api/v3/projects/%s/issues/%s/notes' % (GITLAB_PROJECT_ID, gl_issue),
                    headers={'PRIVATE-TOKEN': GITLAB_USER_TOKENS.get(author, GITLAB_TOKEN)},
                    verify=VERIFY_SSL_CERTIFICATE,
                    data={
                        'body': '[%s](%s)' % (
                            attachment['filename'],
                            file_info.json()['link']['url']
                        )
                    }
                )

            s.get(GITLAB_URL + 'users/sign_out')

Hello,
I have tried your script but when it tries to move attachments, there is an error:

Traceback (most recent call last):
  File "move-issues-from-jira-to-gitlab.py", line 98, in <module>
    verify=VERIFY_SSL_CERTIFICATE
AttributeError: 'NoneType' object has no attribute 'group'

ok, i found the problem.
i'm using GitLab 8.11
Line 32 should be: RE_TOKEN = "<meta name=\"csrf-token\" content=\"(?P<token>.*)\""

I've made an updated version that uses the uploads api and does not need the token anymore.
It also uses the sudo parameter instead of requiring you to add private tokens for each user.

https://gist.github.com/florisb/1266d3584dbfdbd2a8a55d31e2518edd

oh, I'm so sorry, I never got any emails from github about the comments :(

This is really interesting. I am not much of a python coder, but it does seem to be very complete code.

Do you think it would be difficult to go in the opposite direction, from Gitlab to JIRA? (I mean for the code, not for the users ;))

I've updated @florisb's version to also copy assignees and some Jira issue fileds as gitlab issue labels. It also closes automatically all "Done" labeled issues.

https://gist.github.com/amarruedo/e6d670ad0bdcc7049afc25b9c1b1f64b

for those interested, i did the reverse GitLab to JIRA : https://gist.github.com/mvisonneau/4a76401c2f980943cb08c0c8bd1132e1

Based on the script above we wrote a CLI tool since we also needed a mapping between gitlab properties and custom jira fields: https://gitlab.com/smallstack/jira2gitlab.

Based on the script above, the @florisb's usage of upload api and the @amarruedo updates, I have created my version : jira2gitlab.py

It translates contents from Jira Notation to Markdown. So, to keep the pictures in the description and comments, I had to move attachments before creating the issue in Gitlab.
I tried to port Jira Sprint Boards to Gitlab using labels.
I also tried to manage inactive Jira users and unicode usage in attachment names.

This script saved my ass! Thank you very much!

for those interested, i did the reverse GitLab to JIRA : https://gist.github.com/mvisonneau/4a76401c2f980943cb08c0c8bd1132e1

I updated this to work with current APIs as of June 2019. https://gist.github.com/secretrob/b11791f19bc8f72a9ca87943e283b3c4