Do Depend from Lower Package Layer

README.md

Do Depend from Lower Package Layer

Install the dependancy package for from a Lower Layer Package Repo.

Use Case

In nearly every company I've been, we completely build our own ports tree and create our own package repos. There are mutliple reasons for this - "creating our distro" by freeze port sets, setting our own set of options for ports, which are not defaults, etc.

I'm currently doing this, and insanely ... I build everything (I blacklist 1428 ports). Now I want to create my own, company specific packages - for meta packaes, tools, binaries, code, etc - that are stricktly inernal things. (I've gone back and fort - put these in puppet/ansible config management - or put them in packages.)

But I'd like to be able to itterate quickly over a subset, but intergrating 1 pkg into the 30k+ pkg repo ... a simple package that contains a small script or two, takes too long.

poudriere's (ports?) poorly documented OVERLAY feature is ideal, but I want this meta packaqges to be a repo of its own, but pull all depends from a different repo - aka - work like Enterprise Linux (EL7, EL8) repos ... layer ontop. So I need the depenancies to come from the lower layer pkg repo, but the new packages to go to the upper layer packages repo.

Patch ports

Apply the dp_PACKAGES_LOWERLAYER.patch below to the files in /usr/ports/Mk. Outside of poudriere, in the directory of any /usr/port - I create this way :

sudo zfs snap nas0/poudriere/ports/default@20220524a
sudo zfs clone -o mountpoint=/usr/ports nas0/poudriere/ports/default@20220524a nas0/poudriere/ports/clones/default

Not sure it matters, nas0 is actually a zpool on an iSCSI lun from my nas.

make \
    PACKAGES_LOWERLAYER=/usr/local/poudriere/data/packages/13-1-stable-default \
    PACKAGES=/usr/ports/packages \
    USE_PACKAGE_DEPENDS_ONLY=1 \
    package

and this would very well, Built all your packages & create a repo.

> sudo pkg repo /usr/ports/packages
> tree /usr/ports/packages/
/usr/ports/packages/
├── All
│   ├── care2-meta-1.0.0_1.pkg
│   └── py38-passiveagent-0.2.0.pkg
├── meta.conf
├── meta.pkg
├── meta.txz -> meta.pkg
├── packagesite.pkg
└── packagesite.txz -> packagesite.pkg

dp_PACKAGES_LOWERLAYER.patch

diff -r -u /usr/local/poudriere/ports/default/Mk/Scripts/do-depends.sh /usr/ports/Mk/Scripts/do-depends.sh
--- /usr/local/poudriere/ports/default/Mk/Scripts/do-depends.sh	2021-04-06 11:32:08.000000000 +0000
+++ /usr/ports/Mk/Scripts/do-depends.sh	2022-03-25 00:16:04.250638000 +0000
@@ -11,7 +11,7 @@
 	dp_DEPENDS_CLEAN dp_DEPENDS_ARGS dp_USE_PACKAGE_DEPENDS \
 	dp_USE_PACKAGE_DEPENDS_ONLY dp_PKG_ADD dp_PKG_INFO dp_WRKDIR \
 	dp_PKGNAME dp_STRICT_DEPENDS dp_LOCALBASE dp_LIB_DIRS dp_SH \
-	dp_SCRIPTSDIR PORTSDIR dp_MAKE dp_MAKEFLAGS dp_OVERLAYS
+	dp_SCRIPTSDIR PORTSDIR dp_MAKE dp_MAKEFLAGS dp_OVERLAYS dp_PACKAGES_LOWERLAYER

 [ -n "${DEBUG_MK_SCRIPTS}" -o -n "${DEBUG_MK_SCRIPTS_DO_DEPENDS}" ] && set -x

diff -r -u /usr/local/poudriere/ports/default/Mk/Scripts/functions.sh /usr/ports/Mk/Scripts/functions.sh
--- /usr/local/poudriere/ports/default/Mk/Scripts/functions.sh	2021-04-06 11:32:08.000000000 +0000
+++ /usr/ports/Mk/Scripts/functions.sh	2022-03-25 00:13:46.524007000 +0000
@@ -257,6 +257,10 @@
 	local _makeflags _vars
 	local _portvar _var _line

+    	if [ -n "${dp_PACKAGES_LOWERLAYER}" -a -h "${dp_PACKAGES_LOWERLAYER}/All" ]; then
+	 	make_args="${make_args} PACKAGES=${dp_PACKAGES_LOWERLAYER} "
+   	 fi
+
 	_makeflags=
 	_vars=
 	shift 2
diff -r -u /usr/local/poudriere/ports/default/Mk/bsd.port.mk /usr/ports/Mk/bsd.port.mk
--- /usr/local/poudriere/ports/default/Mk/bsd.port.mk	2021-11-30 12:11:12.000000000 +0000
+++ /usr/ports/Mk/bsd.port.mk	2022-03-25 00:03:49.434872000 +0000
@@ -4044,6 +4044,7 @@
 		dp_SCRIPTSDIR="${SCRIPTSDIR}" \
 		PORTSDIR="${PORTSDIR}" \
 		dp_OVERLAYS="${OVERLAYS}" \
+		dp_PACKAGES_LOWERLAYER="${PACKAGES_LOWERLAYER}" \
 		dp_MAKE="${MAKE}" \
 		dp_MAKEFLAGS='${.MAKEFLAGS}' \
 		${SH} ${SCRIPTSDIR}/do-depends.sh