Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Reposync - A better tool than mrepo. Use this to sync down all channels a RHEL system is subscribed to and turn them into locally exposed yum repositories.
#!/bin/bash
# This tool can be used to sync down Red Hat based packages from RHN using only Red Hat shipped tools
# Brian "Red Beard" Harrington <brian@dead-city.org>
# Copyright 2013
#
# To satisfy the pre-reqs for this script install the following two rpms:
# yum-utils
# createrepo_c (in RHEL 8 createrepo and createrepo_c have been combined)
# See https://github.com/rpm-software-management/createrepo_c#differences-in-behavior-between-createrepo_c-and-createrepo
# Additional Notes:
# - The command `reposync` is handled by a DNF plugin of the same name in
# RHEL 8. This plugin does not provide a flag for control of GPG key
# checking and so must be handled via DNF config files or `--setopt`
# - For old CentOS content (e.g. CentOS 6) one must target vault.centos.org
# - `repoview` was deprecated in RHEL8 due to python-KID not being maintained
# - This tool does nothing special in relation to Modularity nor does it
# perform any optimized file handling (e.g. hard linking of content).
# - For more information read the "RPM-Based Distributions" section of this
# blog entry https://www.percona.com/blog/2020/01/02/how-to-create-your-own-repositories-for-packages/
#
set -eu -o pipefail
RS_DOWNLOAD_DIR="${RS_DOWNLOAD_DIR:-/var/www/html/RHN}"
RS_LOG="${RS_LOG:-True}"
RS_LOG_PATH="${RS_LOG_PATH:-/var/log/reposync.log}"
function logger() {
echo "$(date):$(printf ' %q' "$@")" >> "${RS_LOG_PATH}"
"$@" 2>> "${RS_LOG_PATH}"
}
RS_LOG_CMD=""
if [ "${RS_LOG}" == "True" ]; then
RS_LOG_CMD="logger"
fi
# Perform synchronization of content
${RS_LOG_CMD} /usr/bin/reposync -m --download-metadata -p ${RS_DOWNLOAD_DIR}/
# Process all RPMs in directory structure, generating repodata.
for dirname in `find ${RS_DOWNLOAD_DIR} -maxdepth 1 -mindepth 1 -type d`; do
echo $dirname: | tee -a /var/log/reposync.log
COMPS=""
# Check to see if comps.xml exists if so use it for group information
if [ -f "${dirname}/comps.xml" ]; then
${RS_LOG_CMD} cp ${dirname}/comps.xml ${dirname}/Packages/
# Assign this to the comps path for use if comps.xml exists
COMPS="-g '${dirname}/Packages/comps.xml'"
fi
${RS_LOG_CMD} createrepo --update -p --workers 2 ${COMPS} ${dirname}
updateinfo=$(ls -1t ${dirname}/*-updateinfo.xml.gz 2>/dev/null | head -1 )
if [[ -f $updateinfo && $? -eq 0 ]]; then
${RS_LOG_CMD} echo "Updating errata information for ${dirname}"
${RS_LOG_CMD} \cp $updateinfo ${dirname}/updateinfo.xml.gz
${RS_LOG_CMD} gunzip -df ${dirname}/updateinfo.xml.gz
${RS_LOG_CMD} modifyrepo ${dirname}/updateinfo.xml ${dirname}/repodata/
else
${RS_LOG_CMD} echo "No errata information to be processed for ${dirname}"
fi
done
#vim: ts=4 sw=4 expandtab
@MortenMiddelthon
Copy link

MortenMiddelthon commented Feb 18, 2015

Excellent stuff!

Loading

@brianredbeard
Copy link
Author

brianredbeard commented Nov 11, 2015

Wow... unexpected bonus... This works great with Docker allowing for even easier synchronization completely different yum trees. From a Fedora 23 instance, i've added the script to a container and bind mounted in the path and am running it with the following command:

sudo docker run -ti -v /opt/repo/centos:/repo --security-opt label:disable csync

In this case the --security-opt label:disable is due to the MLS restrictions by sVirt used by Docker on RHEL.

Loading

@mkrsn
Copy link

mkrsn commented Feb 21, 2016

You don't need docker to sync completely different yum-trees. Reposync can handle it on it's own.

e.g.: I sync rhel5, rhel6 and rhel7 with reposync only on a centos6 Box. Works really great.

Loading

@elkekou
Copy link

elkekou commented Feb 23, 2016

Hi mkrsn,
Please could you explain how did you do to sync rhel5/6/7 on a centos box with the script above.

Loading

@calba
Copy link

calba commented Apr 19, 2016

Hi Brian,

First of all, thanks a lot for the script.

Out of curiosity: why moving the comps file to the Packages directory?

Thanks

Loading

@admin-pro
Copy link

admin-pro commented Nov 4, 2016

Hello,

Thank you for your script.

My question is same as Elkekou : how did you do sync rhel5/6/7 not on centos but on a redhat 7 please ?

Thank you very much.

Loading

@Firxiao
Copy link

Firxiao commented May 10, 2017

@admin-pro reposync could use specified yum.conf and multi-repo file. you could store rhel5/6/7 repo in one yum.repo.d forlder. then use reposync, and another question you need resolve is the server cloud visit rhel5/6/7 official repository at same time. give you a hint. look the repo file.

Loading

@agentguerry
Copy link

agentguerry commented May 16, 2017

Is there a way when using this script to say "exclude" certain packages?

Loading

@bp85
Copy link

bp85 commented Jun 12, 2018

Can anyone have an example config to reposync rhel6/7 from a rhel7 machine?

Loading

@brianredbeard
Copy link
Author

brianredbeard commented Jul 26, 2018

@bp85 More recently I've been running this via a container (normally i use systemd-nspawn, but docker should work too) to get things done.

Loading

@brianredbeard
Copy link
Author

brianredbeard commented Sep 21, 2018

Makes me realize there should also be a variant of this for container content.

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment