Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
script to clone all repositories in a bitbucket server (aka stash) project
# this script uses syntax for bitbucket server.
# For bitbucket cloud see https://confluence.atlassian.com/bitbucket/use-the-bitbucket-cloud-rest-apis-222724129.html
#
# Note: replace username, password and PROJECTNAME with your values
USERNAME=xxxx
PASSWORD=xxxx
PROJECTNAME=xxxxx
# jq syntax helpful links:
# - http://stackoverflow.com/questions/30601278/how-to-extract-keys-from-multiple-nested-arrays-using-jq
# - http://stackoverflow.com/questions/18592173/select-objects-based-on-value-of-variable-in-object-using-jq
curl -s -u $USERNAME:$PASSWORD https://bitbucket.example.com/rest/api/1.0/projects/$PROJECTNAME/repos/ \
| jq -r '.values[].links.clone[] | select(.name=="ssh") | .href' \
| xargs -I {} echo "git clone '{}'" > clone_repos.sh
@DeekshithSN

This comment has been minimized.

Copy link

@DeekshithSN DeekshithSN commented Jun 19, 2018

Thanks
this work fine if the number of repos are less than 25 , is there any way to get repos names if project contains more 25 repos .

if its more than 25 use https://bitbucket.example.com/rest/api/1.0/projects/$PROJECTNAME/repos/?limit\=1000

@eeichinger

This comment has been minimized.

Copy link
Owner Author

@eeichinger eeichinger commented Jul 5, 2018

tx for the tip - I posted an updated version that works for both, bitbucket server and bitbucket cloud here: https://gist.github.com/eeichinger/c2eb46fbe7d5e2f49eba3bfaf5471759

@jensim

This comment has been minimized.

Copy link

@jensim jensim commented May 11, 2020

I've built a cli for cloning and pulling git repos from bitbucket server
https://github.com/jensim/bitbucket_server_cli

@sreddy31

This comment has been minimized.

Copy link

@sreddy31 sreddy31 commented Feb 1, 2021

I've built a cli for cloning and pulling git repos from bitbucket server
https://github.com/jensim/bitbucket_server_cli

Hey Jensim,
Is the cli you created for cloning all the repo's in bitbucket or does it clone repo's of a specific user?
I get the following error:
Clone/update all projects yes
Failed fetching user & project repos. Failed loading user repos due to 'Failed fetching users from bitbucket, status code: 404 Not Found.'. Failed loading project repos due to 'Failed fetching projects from bitbucket, status code: 404 Not Found.'

@jensim

This comment has been minimized.

Copy link

@jensim jensim commented Feb 2, 2021

@sreddy31 Are you trying it against bitbucket cloud repositories? The CLI only works for bitbucket server
That's the reason I didn't pick the name bitbucket_cli

If your problem is with a bitbucket server installation, please tell me what version of bitbucket server you are running, so that im able to look at the change logs, to determine if they have deprecated some of the API/1.0 endpoints for fetching users and projects.
Or have you configured non-standard paths ? You should be able to add such to the base-path when the CLI asks you for it..

@sreddy31

This comment has been minimized.

Copy link

@sreddy31 sreddy31 commented Feb 2, 2021

@sreddy31 Are you trying it against bitbucket cloud repositories? The CLI only works for bitbucket server
That's the reason I didn't pick the name bitbucket_cli

If your problem is with a bitbucket server installation, please tell me what version of bitbucket server you are running, so that im able to look at the change logs, to determine if they have deprecated some of the API/1.0 endpoints for fetching users and projects.
Or have you configured non-standard paths ? You should be able to add such to the base-path when the CLI asks you for it..

@jensim, Many thanks for your reply.

I was adding a "/" at the end of my bitbucket URL, which tripped the program. The issue is now fixed.

I have found another rather interesting issue...
when I use the command:
bitbucket_server_cli clone --git-quiet --server https://bitbucket.myorg.mycompany.net --username bb_clone --password Password123 --all -H
the command fails with below:
etching projects [00:00:00] [########################################] 2/2 (eta:0s)
Failed loading user repos due to 'Failed fetching users from bitbucket, status code: 401 Unauthorized.
Cause: Body: '{"errors":[{"context":null,"message":"You are not permitted to access this resource","exceptionName":"com.atlassian.bitbucket.AuthorisationException"}]}''
Working repos [00:00:00] [########################################] 2/2 (eta:0s)

however, if I export the same password as env variable, and run the command to grab the password from env variable, it runs without any issues.
export BITBUCKET_PASSWORD=Password123
bitbucket_server_cli clone --git-quiet --server https://bitbucket.myorg.mycompany.net --username test123 --env-password --all -H
Fetching users [00:00:11] [########################################] 948/948 (eta:0s)
Fetching projects [00:00:00] [########################################] 70/70 (eta:0s)
Working repos [00:00:04] [>---------------------------------------] 3/1606 (eta:22m)

Please could you help.

@jensim

This comment has been minimized.

Copy link

@jensim jensim commented Feb 2, 2021

@sreddy31 Im glad it works

I think your problem might be with nbsp in the terminal you are using
I often trip on that the shell treats different whiteSpace differently
Hitting Space is not the same as Shift+Space
Have you tried sanitizing your input in some way
If you use any of JetBrains IDEs, they will highlight strange whiteSpace if you paste it into a scratch file..

@sreddy31

This comment has been minimized.

Copy link

@sreddy31 sreddy31 commented Feb 2, 2021

@Jenism checked and double checked. I am using Ubuntu 18.04 LTS.
Same issue on my Mac too.
Fetching projects [00:00:00] [########################################] 2/2 (eta:0s)
Failed loading user repos due to 'Failed fetching users from bitbucket, status code: 401 Unauthorized.'
Working repos [00:00:00] [########################################] 2/2 (eta:0s)

But, if I export the password as env variable and run the command, it works like a charm.

Your work has helped me save a lot of my time and I sincerely appreciate your efforts.

@MOURIK

This comment has been minimized.

Copy link

@MOURIK MOURIK commented Feb 3, 2021

i have issues with this script.
then after some changes as below it works:
curl -D -X GET -H "Content-Type: application/json" https://$USERNAAME:$user-token@bitbucket.example.com/rest/api/1.0/projects/$PROJECTNAME/repos/
thanks.

@jensim

This comment has been minimized.

Copy link

@jensim jensim commented Feb 3, 2021

@Jenism checked and double checked. I am using Ubuntu 18.04 LTS.
Same issue on my Mac too.
Fetching projects [00:00:00] [########################################] 2/2 (eta:0s)
Failed loading user repos due to 'Failed fetching users from bitbucket, status code: 401 Unauthorized.'
Working repos [00:00:00] [########################################] 2/2 (eta:0s)

But, if I export the password as env variable and run the command, it works like a charm.

Your work has helped me save a lot of my time and I sincerely appreciate your efforts.

Im really glad to hear it has helped another person out there 🥳
I created a issue on my project to track this issue
jensim/bitbucket_server_cli#21
Ill see to it as soon as i have some time to spare!

@yves-vogl

This comment has been minimized.

Copy link

@yves-vogl yves-vogl commented Jun 23, 2021

I've modified it a little bit.

curl -s -u "$ORG_USER:$ORG_PASS" "https://api.bitbucket.org/2.0/repositories/$ORG_TEAM?q=project.key%3D%22${ORG_PROJECT}%22&pagelen=100" \
| jq -r '.values[] | {slug:.slug, links:.links.clone[] } | select(.links.name=="ssh") | "git clone \(.links.href) \(.slug)"' | sort > clone-repos.sh

Make sure that you do not have more than 100 repos. In this case you need to add pagination, e.g. in a while-loop.

https://developer.atlassian.com/bitbucket/api/2/reference/meta/pagination

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment