Create Gitlab Personal Access Token using curl

gitlab_access_token.md

Create Gitlab Personal Access Token using curl

Prerequisites

• You need a Gitlab server up and running
• You need user credentials for a (admin) user on the Gitlab server
• You need curl and Perl on your server

What does it do?

1. Open the login page of Gitlab to get session cookie
2. Login to Gitlab using username and password to get authenticated session cookie
3. Open the Personal Access Tokens page
4. POSTing the Personal Access Token for to generate a personal access token
5. Scrape the personal access token from the returned HTML page

Code

gitlab_host="http://localhost:8080"
gitlab_user="root"
gitlab_password="12341234"

# 1. curl for the login page to get a session cookie and the sources with the auth tokens
body_header=$(curl -c cookies.txt -i "${gitlab_host}/users/sign_in" -s)

# grep the auth token for the user login for
#   not sure whether another token on the page will work, too - there are 3 of them
csrf_token=$(echo $body_header | perl -ne 'print "$1\n" if /new_user.*?authenticity_token"[[:blank:]]value="(.+?)"/' | sed -n 1p)

# 2. send login credentials with curl, using cookies and token from previous request
curl -b cookies.txt -c cookies.txt -i "${gitlab_host}/users/sign_in" \
	--data "user[login]=${gitlab_user}&user[password]=${gitlab_password}" \
	--data-urlencode "authenticity_token=${csrf_token}"

# 3. send curl GET request to personal access token page to get auth token
body_header=$(curl -H 'user-agent: curl' -b cookies.txt -i "${gitlab_host}/profile/personal_access_tokens" -s)
csrf_token=$(echo $body_header | perl -ne 'print "$1\n" if /authenticity_token"[[:blank:]]value="(.+?)"/' | sed -n 1p)

# 4. curl POST request to send the "generate personal access token form"
#      the response will be a redirect, so we have to follow using `-L`
body_header=$(curl -L -b cookies.txt "${gitlab_host}/profile/personal_access_tokens" \
	--data-urlencode "authenticity_token=${csrf_token}" \
	--data 'personal_access_token[name]=golab-generated&personal_access_token[expires_at]=&personal_access_token[scopes][]=api')

# 5. Scrape the personal access token from the response HTML
personal_access_token=$(echo $body_header | perl -ne 'print "$1\n" if /created-personal-access-token"[[:blank:]]value="(.+?)"/' | sed -n 1p)

Usage of token in API Requests

According to the Gitlab API documentation, you can now use the personal_access_token to make API requests:

curl --header "Private-Token: ${personal_access_token}" https://gitlab.example.com/api/v4/projects

+1 for this. super useful

Thanks for your work but do have any clue how to make it work with LDAP authentication ?
Thanks

It is giving
curl: (6) Could not resolve host: Private-Token with curl --header "Private-Token: ${personal_access_token}" https://gitlab.example.com/api/v4/projects

"gitlab.example.com" is literally an example. Here is an instance where you have your own organisation and repo and you want the issues that are assigned to you.

curl --header "PRIVATE-TOKEN: <Private_Token>" "https://gitlab.com/api/v4/groups/<your url-encoded project path with / converted to %2F>/issues?assignee_username[]=<your username>"

This results in a JSON encoded response like

[{"id":72111718,"iid":111,"project_id":21511178,"title":...

Hmm I think this doesn't work anymore, because of possibly one of two things. 1) Since GitLab does that thing where it checks your browser before letting you log in, or 2) it doesn't work because I have MFA set up on my GitLab account 😢

@michaellihs Does this script account for any of that?

@emmahsax Same here with an on prem install (13.8.x) without MFA. My guess is that something has been changed in Gitlab that this doesn't work anymore. The script just returns empty output

Came across the same issue for Gitlab IaC...got around it by using rails console to set an arbitrary token.

https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#programmatically-creating-a-personal-access-token

gitlab-rails runner can be replaced with bundle exec rails runner -e production if needed

Really appreciate your help.
Just for data integrity want to leave link to SO with the same question:
https://stackoverflow.com/questions/47948887/login-to-gitlab-with-username-and-password-using-curl

And a few notes from myself:

1. It did not work for 'as is';
2. added -L redirect parameter to all curl requests (got 302 before that);
3. changed url because as checked with UI there was extra -/ in path. So finally in script I use:

"${gitlab_host}/-/profile/personal_access_tokens"

4. Got some 422 errors, maybe my VM connection issue. At the 2-3rd try the TOKEN was created, received in script, confirmed in UI.

---

Self-hosted Gitlab as docker image gitlab/gitlab-ee:latest:
sha256:fb63c8733903dfffd659bdb31efbf586c9fb7bde127136c29d55ac2f26f4f2ee
version: 14.8.2-ee.0
my image page: https://hub.docker.com/layers/gitlab/gitlab-ee/latest/images/sha256-fb63c8733903dfffd659bdb31efbf586c9fb7bde127136c29d55ac2f26f4f2ee?context=explore

still useful, thanks man.

gitlab_host="http://localhost:8080"
gitlab_user="root"
gitlab_password="12341234"

# 1. curl for the login page to get a session cookie and the sources with the auth tokens
body_header=$(curl -c cookies.txt -i "${gitlab_host}/users/sign_in" -s)
# grep the auth token for the user login for
#   not sure whether another token on the page will work, too - there are 3 of them
csrf_token=$(echo $body_header | perl -ne 'print "$1\n" if /new_user.*?authenticity_token"[[:blank:]]value="(.+?)"/' | sed -n 1p)

# 2. send login credentials with curl, using cookies and token from previous request
curl -L -b cookies.txt -c cookies.txt -i "${gitlab_host}/users/sign_in" \
  --data-raw "user%5Blogin%5D=${gitlab_user}&user%5Bpassword%5D=${gitlab_password}" \
  --data-urlencode "authenticity_token=${csrf_token}" \
  --compressed \
  --insecure 2>&1 > /dev/null

# 3. send curl GET request to personal access token page to get auth token
body_header=$(curl -H 'user-agent: curl' -b cookies.txt -i "${gitlab_host}/-/profile/personal_access_tokens" -s)
csrf_token=$(echo $body_header | perl -ne 'print "$1\n" if /authenticity_token"[[:blank:]]value="(.+?)"/' | sed -n 1p)

# 4. curl POST request to send the "generate personal access token form"
#      the response will be a redirect, so we have to follow using `-L`
body_header=$(curl -L -b cookies.txt "${gitlab_host}/-/profile/personal_access_tokens" \
    --data-urlencode "authenticity_token=${csrf_token}" \
    --data 'personal_access_token[name]=golab-generated&personal_access_token[expires_at]=&personal_access_token[scopes][]=api')

# 5. Scrape the personal access token from the response json
personal_access_token=$(echo $body_header | perl -ne 'print "$1\n" if /new_token":"(.+?)"/' | sed -n 1p)

version GitLab Community Edition [15.4.2]

gitlab_host="https://"
gitlab_user="root"
gitlab_password=""

# 1. curl for the login page to get a session cookie and the sources with the auth tokens
body_header=$(curl -c cookies.txt -i "${gitlab_host}/users/sign_in" -s --insecure)
# grep the auth token for the user login for
#   not sure whether another token on the page will work, too - there are 3 of them
csrf_token=$(echo $body_header | perl -ne 'print "$1\n" if /new_user.*?authenticity_token"[[:blank:]]value="(.+?)"/' | sed -n 1p)
# 2. send login credentials with curl, using cookies and token from previous request
curl -L -b cookies.txt -c cookies.txt -i "${gitlab_host}/users/sign_in" \
  --data-raw "user%5Blogin%5D=${gitlab_user}&user%5Bpassword%5D=${gitlab_password}" \
  --data-urlencode "authenticity_token=${csrf_token}" \
  --compressed \
  --insecure 2>&1 > /dev/null

# 3. send curl GET request to personal access token page to get auth token
body_header=$(curl -H 'user-agent: curl' -b cookies.txt -i "${gitlab_host}/-/profile/personal_access_tokens" -s --insecure)

csrf_token=$(echo $body_header | perl -ne 'print "$1\n" if /csrf-token"[[:blank:]]content="(.+?)"/' | sed -n 1p)
# 4. curl POST request to send the "generate personal access token form"
#      the response will be a redirect, so we have to follow using `-L`
body_header=$(curl -L -b cookies.txt "${gitlab_host}/-/profile/personal_access_tokens" \
    --data-urlencode "authenticity_token=${csrf_token}" \
    --data 'personal_access_token[name]=golab-generated&personal_access_token[expires_at]=&personal_access_token[scopes][]=api' --insecure)
# 5. Scrape the personal access token from the response json
personal_access_token=$(echo $body_header | perl -ne 'print "$1\n" if /new_token":"(.+?)"/' | sed -n 1p)
echo $personal_access_token

version GitLab Community Edition [15.6]

Thanks for your work but do have any clue how to make it work with LDAP authentication ? Thanks
It is also interesting - is there an opportunity to adapt this method for programmatic LDAP authorization? I want to request pages with changes from the Github repository, but I can't go beyond authorization...

Does anyone tried the python module "python-gitlab" or cmdline tool glab, maybe it's a better choice ?