To optimize our CI builds we wanted to split our production dependencies from out dev dependencies. So that we would aquire two seperate archives which could be deployed seperatley.
For production we would only deploy production.tar.gz for testing we would deploy require.tar.gz
and require-dev.tar.gz
.
Unfortunately composer
only comes with the composer install/update -no-dev
that would ignore installation of dev dependencies.
Currently we are running two composer install
while moving the vendor/
into vendor_prod/
and so forth using rsync
to find the differenc between prod and dev installation and creating the proper tar balls. Sadly this creates a huge IO load.
To optimize the IO load I wanted to come up with a command that "only" installs dev dependencies, such as composer install --only-dev
. As there is no such command I come up with a bash script oneliner which does this
cat composer.json | jq -r '."require-dev"' | sed --regexp-extended 's/\{|\}|\s+"([^"]+)":.*/\1/' | xargs -I {} composer show {} -t | sed --regexp-extended 's/[^a-z0-9]*([a-z0-9_-]+\/[a-z0-9_-]+).*|.*/vendor\/\1/' | sed '/^$/d' | sort | uniq | tar zcf require-dev.tar.gz --ignore-failed-read -T -
The only tool you would need is jq
which can parse the json and extract the node of require-dev
dependencies.
- This will simply read your
composer.json
content to pipe it tojq
cat composer.json
- Extract
require-dev
node fromcomposer.json
jq -r '."require-dev"'
---
{
"zendframework/zend-expressive-tooling": "^1.2",
"behat/behat": "^3.5"
}
- Use
sed
to extract only package names
sed --regexp-extended 's/\{|\}|\s+"([^"]+)":.*/\1/'
---
zendframework/zend-expressive-tooling
behat/behat
- Use
composer show X -t
to get the tree of dependencies for each package
xargs -I {} composer show {} -t
---
zendframework/zend-expressive-tooling 1.2.1 Migration and development tooling for Expressive
├──ext-json *
├──ocramius/package-versions ^1.3
│ ├──composer-plugin-api ^1.0.0
│ └──php ^7.1.0
├──php ^7.1
├──symfony/console ^2.8 || ^3.0 || ^4.0
...
behat/behat 3.5.2 Description
├──symfony/console ^2.8 || ^3.0 || ^4.
...
- Use
sed
again to extract every packagename with certain pattern and remove everything else (like extension or language dependencies). Also addingvendor/
to the resulting package name to reflect package path.
sed --regexp-extended 's/[^a-z0-9]*([a-z0-9_-]+\/[a-z0-9_-]+).*|.*/vendor\/\1/'
---
vendor/zendframework/zend-expressive-tooling
vendor/ocramius/package-versions
vendor/symfony/console
vendor/behat/behat
vendor/symfony/console
- Remove empty lines using
sed
sed '/^$/d'
---
vendor/zendframework/zend-expressive-tooling
vendor/ocramius/package-versions
vendor/symfony/console
vendor/behat/behat
vendor/symfony/console
- Use
sort
anduniq
to remove duplicate package names
sort | uniq
---
vendor/behat/behat
vendor/ocramius/package-versions
vendor/symfony/console
vendor/zendframework/zend-expressive-tooling
- Pack every folder path into a output tar ball. This will also ignore every read package name that does not exist.
For example if you are using
roave/security-advisories
as its only a meta package without actual files.
tar zcf require-dev.tar.gz --ignore-failed-read -T -