Skip to content

Instantly share code, notes, and snippets.

@RichardHightower
Last active May 5, 2023 20:47
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save RichardHightower/fd7cc422e1d572515cfa47c014cdc04c to your computer and use it in GitHub Desktop.
Save RichardHightower/fd7cc422e1d572515cfa47c014cdc04c to your computer and use it in GitHub Desktop.
github scan for log4j

install github command line

  1. This command will run a script called install-github-cli.sh.

  2. The script will install the GitHub command line interface (CLI) onto the user's computer.

  3. The CLI will allow the user to interact with the GitHub repository from the command line.

  4. The script will download the latest version of the CLI from GitHub and install it on the user's computer.

  5. Once the installation is complete, the user will be able to use the CLI to manage their GitHub repositories.

install-github-cli.sh

#!/bin/bash
set -Eeuo pipefail

# Create a log file
LOG_FILE="install_gh.log"
touch "$LOG_FILE"

# Function to handle errors
error_handler() {
  echo "An error occurred during the installation process. Please check the log file for details." | tee -a "$LOG_FILE"
}

# Register the error handler
trap error_handler ERR

# Check if GitHub CLI (gh) is installed
if ! command -v gh &>/dev/null; then
  echo "GitHub CLI not found, installing..." | tee -a "$LOG_FILE"

  # Detect operating system
  OS=$(uname)

  if [ "$OS" == "Darwin" ]; then
    # macOS
    if ! command -v brew &>/dev/null; then
      echo "Homebrew not found, installing..." | tee -a "$LOG_FILE"
      /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" | tee -a "$LOG_FILE"
    fi
    brew install gh | tee -a "$LOG_FILE"
  elif [ "$OS" == "MINGW64_NT-10.0" ] || [ "$OS" == "MSYS_NT-10.0" ]; then
    # Windows (Git Bash)
    if ! command -v choco &>/dev/null; then
      echo "Chocolatey not found, installing..." | tee -a "$LOG_FILE"
      /bin/bash -c "$(curl -fsSL https://chocolatey.org/install.ps1)" | tee -a "$LOG_FILE"
    fi
    choco install gh | tee -a "$LOG_FILE"
  else
    echo "Unsupported operating system. Please install GitHub CLI manually." | tee -a "$LOG_FILE"
    exit 1
  fi

  echo "GitHub CLI installed successfully!" | tee -a "$LOG_FILE"
else
  echo "GitHub CLI is already installed." | tee -a "$LOG_FILE"
fi

This command line runs a script to pull all of the repositories from a given GitHub organization.

  1. The command is running the pull-github-repos-org.sh script.
  2. This script will pull all the repositories from a GitHub organization and clone them onto the local machine.
  3. It will first authenticate with the GitHub organization using the credentials provided.
  4. It will then loop through the list of repositories in the organization and clone each one to the local machine.
  5. Once the cloning is complete, the script will finish and the repositories will be available on the local machine.

Here's a script that finds all Java repositories in a GitHub organization, clones them using SSH in the current directory, and is compatible with macOS Zsh and Windows Git Bash:

pull-github-repos-org.sh

#!/bin/bash

# Set the organization name
ORG_NAME="nats-io"

# Function to check if a command exists
command_exists() {
  command -v "$1" >/dev/null 2>&1
}

# Check if gh and jq are installed
if ! command_exists gh; then
  echo "gh command not found. Please install GitHub CLI."
  exit 1
elif ! command_exists jq; then
  echo "jq command not found. Please install jq."
  exit 1
fi

# Create a log file
LOG_FILE="clone_repos.log"
touch "$LOG_FILE"

# Function to handle errors
error_handler() {
  echo "An error occurred during the cloning process. Please check the log file for details." | tee -a "$LOG_FILE"
}

# Register the error handler
trap error_handler ERR

# Clone Java repositories in the organization
gh api "orgs/${ORG_NAME}/repos" --paginate | jq -r '.[] | select(.language == "Java") | .ssh_url' | while read -r repo_url; do
  echo "Cloning repository: ${repo_url}" | tee -a "$LOG_FILE"
  git clone --depth 1 "${repo_url}" | tee -a "$LOG_FILE"
done

echo "Finished cloning Java repositories." | tee -a "$LOG_FILE"

This script checks if gh and jq are installed, sets up a log file, and uses error handling to provide feedback if an issue occurs during the cloning process. It should run on macOS with Zsh and on Windows using Git Bash.

Please ensure that you have the necessary SSH keys set up for your GitHub account before running this script, as it uses SSH for cloning repositories. You might also need to authenticate with GitHub using gh auth login before running this script if you haven't done so already.

#!/bin/bash
# Set bash to log all commands run
set -x
set -Eeuo pipefail
error_handler() {
echo "An error occurred during the installation process for this project.
Please check the log file for details."
}
# Register the error handler
trap error_handler ERR
# Create the Python script file
cat > check_log4j_dependencies.py << 'EOF'
import os
import subprocess
from pathlib import Path
import xml.etree.ElementTree as ET
import xml.dom.minidom as minidom
class MavenDependency:
def __init__(self, repo, subproject, group_id, artifact_id, artifact_type, version, scope, parent=None):
self.group_id = group_id
self.artifact_id = artifact_id
self.artifact_type = artifact_type
self.version = version
self.scope = scope
self.parent = parent
self.repo = repo
self.subproject = subproject
@classmethod
def from_maven_output(cls, repo, subproject, maven_output):
stack = []
dependencies = []
for line in maven_output:
line = line.strip()
if line.startswith('[INFO]'):
line = line[len('[INFO] '):]
if '+-' in line:
parent = stack[-1] if stack else None
level, artifact_line = line.split('+-')
group_id, artifact_id, a_type, version, scope = artifact_line.split(':')
dependency = cls(repo, subproject, group_id.strip(), artifact_id, a_type, version, scope, parent)
dependencies.append(dependency)
stack.append(dependency)
elif '\-' in line:
parent = stack[-1] if stack else None
level, artifact_line = line.split('\-')
group_id, artifact_id, a_type, version, scope = artifact_line.split(':')
dependency = cls(repo, subproject, group_id.strip(), artifact_id, a_type, version, scope, parent)
dependencies.append(dependency)
if len(stack) > 0:
stack.pop()
return dependencies
files_and_dirs = os.listdir('.')
for repo in os.listdir('.'):
print(repo)
for root, _, files in os.walk(repo):
for file in files:
if file == 'pom.xml':
pom_file = os.path.join(root, file)
subproject_dir = os.path.dirname(pom_file)
print(pom_file, subproject_dir)
dependencies_txt = os.path.join(subproject_dir, 'dependencies.txt')
with open(dependencies_txt, 'w') as f:
print(f"Resolving dependencies for {pom_file}...")
subprocess.run(['mvn', '-f', pom_file, 'dependency:tree'], stdout=f, stderr=subprocess.DEVNULL)
with open(dependencies_txt, 'r') as f:
file_contents = f.read()
if ("log4j" in file_contents):
print(f"Found log4j in base dependencies for {pom_file}...")
with open(dependencies_txt, 'r') as f:
dependencies_lines = f.readlines()
#print(file_contents)
dependencies = MavenDependency.from_maven_output(os.path.basename(repo),
os.path.basename(subproject_dir),
dependencies_lines)
#
# for d in dependencies:
# print(f" Log4j {d.repo}/{d.subproject}:::{d.artifact_id}:{d.version}")
log4j_dependencies = [dependency for dependency in dependencies if 'log4j' in dependency.artifact_id ]
for d in log4j_dependencies:
print(f" Log4j {d.repo}/{d.subproject}:::{d.artifact_id}:{d.version}")
for d in log4j_dependencies:
parent = d.parent
rootParent = d.parent
while parent is not None:
rootParent = parent
parent = parent.parent
r = rootParent
print(f" Log4j {d.repo}/{d.subproject}:::{d.group_id}:{d.artifact_id}:{d.version}")
print(f" Root {r.repo}/{r.subproject}:::{r.group_id}:{r.artifact_id}:{r.version}")
# Create an XML element for the dependency with exclusions
dependency_elem = ET.Element('dependency')
groupId_elem = ET.SubElement(dependency_elem, 'groupId')
groupId_elem.text = r.group_id
artifactId_elem = ET.SubElement(dependency_elem, 'artifactId')
artifactId_elem.text = r.artifact_id
version_elem = ET.SubElement(dependency_elem, 'version')
version_elem.text = r.version
exclusions_elem = ET.SubElement(dependency_elem, 'exclusions')
exclusion_elem = ET.SubElement(exclusions_elem, 'exclusion')
groupId_elem = ET.SubElement(exclusion_elem, 'groupId')
groupId_elem.text = d.group_id
artifactId_elem = ET.SubElement(exclusion_elem, 'artifactId')
artifactId_elem.text = d.artifact_id
# Output the XML element as a string
xml_str = minidom.parseString(ET.tostring(dependency_elem)).toprettyxml(indent=" ")
print(xml_str)
EOF
# Set bash to error out if any command fails
set -e
# Detect operating system
OS=$(uname)
# Install Maven using the appropriate package manager
if [ "$OS" == "Darwin" ]; then
# macOS
if ! command -v brew &>/dev/null; then
echo "Homebrew not found, installing..."
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
fi
if ! command -v mvn &>/dev/null; then
echo "Maven not found, installing..."
brew install maven
fi
if ! command -v python3 &>/dev/null; then
echo "Python 3 not found, installing..."
brew install python@3
fi
elif [ "$OS" == "MINGW64_NT-10.0" ] || [ "$OS" == "MSYS_NT-10.0" ]; then
# Windows (Git Bash)
if ! command -v choco &>/dev/null; then
echo "Chocolatey not found, installing..."
/bin/bash -c "$(curl -fsSL https://chocolatey.org/install.ps1)"
fi
if ! command -v mvn &>/dev/null; then
echo "Maven not found, installing..."
choco install maven
fi
if ! command -v python3 &>/dev/null; then
echo "Python 3 not found, installing..."
choco install python --version 3.9.7
fi
else
echo "Unsupported operating system. Please install Maven and Python 3 manually."
exit 1
fi
echo "The Python script 'check_log4j_dependencies.py' has been created in the current directory."
#!/bin/bash
set -Eeuo pipefail
# Create a log file
LOG_FILE="install_gh.log"
touch "$LOG_FILE"
# Function to handle errors
error_handler() {
echo "An error occurred during the installation process. Please check the log file for details." | tee -a "$LOG_FILE"
}
# Register the error handler
trap error_handler ERR
# Check if GitHub CLI (gh) is installed
if ! command -v gh &>/dev/null; then
echo "GitHub CLI not found, installing..." | tee -a "$LOG_FILE"
# Detect operating system
OS=$(uname)
if [ "$OS" == "Darwin" ]; then
# macOS
if ! command -v brew &>/dev/null; then
echo "Homebrew not found, installing..." | tee -a "$LOG_FILE"
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" | tee -a "$LOG_FILE"
fi
brew install gh | tee -a "$LOG_FILE"
elif [ "$OS" == "MINGW64_NT-10.0" ] || [ "$OS" == "MSYS_NT-10.0" ]; then
# Windows (Git Bash)
if ! command -v choco &>/dev/null; then
echo "Chocolatey not found, installing..." | tee -a "$LOG_FILE"
/bin/bash -c "$(curl -fsSL https://chocolatey.org/install.ps1)" | tee -a "$LOG_FILE"
fi
choco install gh | tee -a "$LOG_FILE"
else
echo "Unsupported operating system. Please install GitHub CLI manually." | tee -a "$LOG_FILE"
exit 1
fi
# Install jq
if ! command -v jq &>/dev/null; then
echo "jq not found, installing..." | tee -a "$LOG_FILE"
if [ "$OS" == "Darwin" ]; then
# macOS
brew install jq | tee -a "$LOG_FILE"
elif [ "$OS" == "MINGW64_NT-10.0" ] || [ "$OS" == "MSYS_NT-10.0" ]; then
# Windows (Git Bash)
choco install jq | tee -a "$LOG_FILE"
else
echo "Unsupported operating system. Please install jq manually." | tee -a "$LOG_FILE"
exit 1
fi
fi
echo "GitHub CLI and jq installed successfully!" | tee -a "$LOG_FILE"
else
echo "GitHub CLI and jq are already installed." | tee -a "$LOG_FILE"
fi
#!/bin/bash
set -x
set -Eeuo pipefail
error_handler() {
echo "Problems pulling the repos"
}
# Register the error handler
trap error_handler ERR
# Set the organization name
ORG_NAME="nats-io"
# Function to check if a command exists
command_exists() {
command -v "$1" >/dev/null 2>&1
}
# Check if gh and jq are installed
if ! command_exists gh; then
echo "gh command not found. Please install GitHub CLI."
exit 1
elif ! command_exists jq; then
echo "jq command not found. Please install jq."
exit 1
fi
# Create a log file
LOG_FILE="clone_repos.log"
touch "$LOG_FILE"
# Function to handle errors
error_handler() {
echo "An error occurred during the cloning process. Please check the log file for details." | tee -a "$LOG_FILE"
}
# Register the error handler
trap error_handler ERR
# Clone Java repositories in the organization
gh api "orgs/${ORG_NAME}/repos" --paginate | jq -r '.[] | select(.language == "Java") | .ssh_url' | while read -r repo_url; do
echo "Cloning repository: ${repo_url}" | tee -a "$LOG_FILE"
git clone --depth 1 "${repo_url}" | tee -a "$LOG_FILE"
done
echo "Finished cloning Java repositories." | tee -a "$LOG_FILE"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment