Clone all repos from a GitHub organization

clone-all-twitter-github-repos.sh

curl -s https://api.github.com/orgs/twitter/repos?per_page=200 | ruby -rubygems -e 'require "json"; JSON.load(STDIN.read).each { |repo| %x[git clone #{repo["ssh_url"]} ]}'

clone all repos

• you need to install gh (github api)
• windows : choco install gh
• mac : brew install gh

username="ChaitanyaChandra" # user name or org name is required
limit=100
gh repo list $username --limit $limit | awk '{print $1}' > out.txt
while read -r line; do echo "git clone  https://github.com/$line.git" >> final.sh  ; done < out.txt
chmod +x final.sh
./final.sh

• Into $ORG folder
  mkdir $ORG; cd $ORG
• Loop clone repo
  gh repo list $ORG --limit 9999 --json url | jq '.[]|.url' | xargs -n1 git clone

• Into $ORG folder
  mkdir $ORG; cd $ORG
• Loop clone repo
  gh repo list $ORG --limit 9999 --json url | jq '.[]|.url' | xargs -n1 git clone

Github stopped https clones (username and passwords needed for every clone so instead use ssh clone
gh repo list $ORG --limit 9999 --json sshUrl | jq '.[]|.sshUrl' | xargs -n1 git clone

This also works whilst using gh command and auth instead of git

gh repo list $ORG --limit 9999 --json nameWithOwner | jq '.[]|.nameWithOwner' | xargs -n1 gh repo clone

this can be even simpler, assuming < 100 repos: curl -s https://api.github.com/orgs/<ORG_NAME>/repos\?per_page\=100 | jq '.[].html_url' | xargs -n 1 git clone

Just in case, if you need ssh url and a bit more than 100 repos:
curl -s "https://api.github.com/orgs/<ORG_NAME>/repos?per_page=100&page=<1,2...>" | jq '.[].ssh_url' | xargs -n 1 git clone

Hi all
I wrote a simple script to clone all the repositories from GitHub, either from a user or whole organization. Please check it out, I'm sure you'll find it useful

Here's a fully working script with things from here, SO + some improvements by me. Needs gh cli.

If the repo exists it will check out the main/master branch and pull.

#!/bin/bash

ORG="myorg" # Your organization
CLONE_PATH="/home/user/Documents/..." # Path in your filesystem where you want to clone the repos
PER_PAGE=100 # per_page maxes out at 100
TOKEN="" # Set your token from GH

for ((PAGE=1; ; PAGE+=1)); do
  # Page 0 and 1 are the same
  # Change authorization method as needed
  INPUT=$(curl -H "Authorization: token $TOKEN" -s "https://api.github.com/orgs/$ORG/repos?per_page=$PER_PAGE&page=$PAGE" | jq -r ".[].clone_url")
  if [[ -z "$INPUT" ]]; then
    echo "All repos processed, stopped at page=$PAGE"
    exit
  fi
  while read REPO_URL ; do
    echo $REPO_URL
    temp=${REPO_URL##*/}
    repo_name=${temp%.*}
    gh repo clone "$REPO_URL" "$CLONE_PATH/$repo_name" -- -q 2>/dev/null || (
        cd "$CLONE_PATH/$repo_name"
        # Handle case where local checkout is on a non-main/master branch
        # - ignore checkout errors because some repos may have zero commits, 
        # so no main or master
        git checkout -q main 2>/dev/null || true
        git checkout -q master 2>/dev/null || true
        git pull -q
    )
  done < <(echo "$INPUT")
done

And here's a "Perl-as-glue" version, that works on systems with command-line curl and git: https://gist.github.com/ology/b6087be12131bae0441d40d87505e658 :D

• | xargs -n1 git clone

we can essentially remove jq and directly use gh

gh repo list --json=sshUrl --limit 1000 -q ".[].sshUrl" | xargs -n1 git clone

though, it's asking for ssh passcode... any idea how to overcome that?

meh.. I have moved to ghorg. implemented it does everything I could do with my own scripts and is 10 times as fast to boot gh

…

On Mon, Jan 15, 2024 at 10:23 AM Vaisakh K M ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ - | xargs -n1 git clone we can essentially remove jq and directly use gh gh repo list --json=sshUrl --limit 1000 -q ".[].sshUrl" | xargs -n1 git clone — Reply to this email directly, view it on GitHub <https://gist.github.com/caniszczyk/3856584#gistcomment-4832274> or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNZZRED6QI6M2V6ZDBT4XTYOVXZXBFKMF2HI4TJMJ2XIZLTSKBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDHNFZXJJDOMFWWLK3UNBZGKYLEL52HS4DFVRZXKYTKMVRXIX3UPFYGLK2HNFZXIQ3PNVWWK3TUUZ2G64DJMNZZDAVEOR4XAZNEM5UXG5FFOZQWY5LFU4ZTQNJWGU4DJJ3UOJUWOZ3FOKTGG4TFMF2GK> . You are receiving this email because you commented on the thread. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub> .

meh.. I have moved to ghorg. implemented it does everything I could do with my own scripts and is 10 times as fast to boot gh
…
On Mon, Jan 15, 2024 at 10:23 AM Vaisakh K M @.> wrote: @.* commented on this gist. ------------------------------ - | xargs -n1 git clone we can essentially remove jq and directly use gh gh repo list --json=sshUrl --limit 1000 -q ".[].sshUrl" | xargs -n1 git clone — Reply to this email directly, view it on GitHub https://gist.github.com/caniszczyk/3856584#gistcomment-4832274 or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNZZRED6QI6M2V6ZDBT4XTYOVXZXBFKMF2HI4TJMJ2XIZLTSKBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDHNFZXJJDOMFWWLK3UNBZGKYLEL52HS4DFVRZXKYTKMVRXIX3UPFYGLK2HNFZXIQ3PNVWWK3TUUZ2G64DJMNZZDAVEOR4XAZNEM5UXG5FFOZQWY5LFU4ZTQNJWGU4DJJ3UOJUWOZ3FOKTGG4TFMF2GK . You are receiving this email because you commented on the thread. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

thanks.. i didn't know that existed...

Windows / PowerShell

gh repo list theUSERorORG --json=nameWithOwner --limit 1000 -q ".[].nameWithOwner"  | %{gh repo clone $_}