There are currently three problems in openstack/tripleo-ui ci:
- Python wheels in tarballs.openstack.org/tripleo-ui/
- tripleo-ui-latest.tar.gz isn't updated in tarballs.openstack.org/tripleo-ui/
- tripleo-ui tarballs use unhelpful commits-since-tag naming scheme
The entry point for openstack ci configuration is the openstack-infra/project-config repository.
Here is the configuration for tripleo-ui:
- project:
name: openstack/tripleo-ui
templates:
- system-required
- nodejs8-jobs
- nodejs8-publish-to-npm
- release-notes-jobs
- translation-jobs
- translation-jobs-ocata
- translation-jobs-pike
post:
jobs:
- publish-openstack-python-tarball
The reader will notice that openstack/tripleo-ui is an npm project and doesn't ship any Python tarballs or wheels. This patch removes the Python tarball job. This solves problem number 1.
Let's focus on the following two jobs:
nodejs8-jobs
nodejs8-publish-to-npm
Here is the definition for nodejs8-jobs
in openstack-zuul-jobs:
- project-template:
name: nodejs8-jobs
description: |
Run lint and test jobs using Node 8.
check:
jobs:
- nodejs-npm-run-lint:
vars:
node_version: 8
- nodejs-npm-run-test:
vars:
node_version: 8
- build-javascript-tarball:
files:
- npm-shrinkwrap.json
- package.json
gate:
jobs:
- nodejs-npm-run-lint:
vars:
node_version: 8
- nodejs-npm-run-test:
vars:
node_version: 8
- build-javascript-tarball:
files:
- npm-shrinkwrap.json
- package.json
This job has two sections: check and gate: Check happens on every patchset, and
once the patch is approved, the gate happens. You can also have post
which
runs after a patch is merged.
So, we run three things: lint, test, and build tarball.
Building a tarball only happens when the patch touches the files
npm-shrinkwrap.json
or package.json
. This jobs is redundant, and there is a
patch to remove it. The job only tests that npm pack
will succeed, and
doesn't produce any artifacts.
You can see that nodejs8-jobs
doesn't handle any tarball updating/uploading.
Let's move on.
Here is the definition for nodejs8-publish-to-npm
in openstack-zuul-jobs:
- project-template:
name: nodejs8-publish-to-npm
release:
jobs:
- publish-openstack-javascript-tarball:
vars:
node_version: 8
- release-openstack-javascript:
vars:
node_version: 8
- announce-release:
dependencies:
- release-openstack-javascript
- publish-openstack-javascript-tarball
You will notice that all of the jobs are under the release
header. This
means that tarballs are published to tarballs.openstack.org only when a release
is made. Cool.
But, wait. How are the per-commit tarballs getting there?
Let's focus on the tripleo-ui-latest.tar.gz
file issue for a moment. Where is
that done? Some grepping reveals that it happens here, in
openstack-infra/zuul-jobs (note the difference between
openstack-zuul-jobs
and zuul-jobs
):
- name: Rename tarball for uploading
shell: |
mkdir -p dist
cp *.tgz dist/{{ zuul.project.short_name }}-{{ project_ver }}.tar.gz
cp *.tgz dist/{{ zuul.project.short_name }}-latest.tar.gz
This is in the fetch-javascript-tarball
role.
Who uses this role?
A bit more grepping two locations:
... well? Who uses that?
It's release-openstack-javascript
in project-config.
This job first runs pre.yaml
: installs nodejs, revokes sudo, installs npm
packages, etc. Then it runs tarball.yaml
which runs npm pack
. And then the
post.yaml
which is the subject of the current heading.
post.yaml
fetches javascript output (whatever that means), gets the package
version from git, and then fetches the javascript tarball (whatever that means
again). Then, finally, it runs the upload-npm
role.
So, what do the fetch-javascript-*
jobs do? What does upload-npm
do?
The more important thing is, do we even care?
That's right, we don't care. The release-openstack-javascript
job is only
run when a release is made as we saw earlier.
Let's move on.
Who uses this beast?
It's publish-openstack-javascript-tarball
in openstack-zuul-jobs
.
In pre.yaml
, it does npm setup.
In tarball.yaml
, it runs npm pack
.
In post.yaml
, it runs fetch-javascript-output
(whatever that means).
And in post-tarball.yaml
(subject of current heading) it does
version-from-git
, and fetch-javascript-tarball
.
Who runs publish-openstack-javascript-tarball
?
It's the release job in nodejs8-publish-to-npm
. So, it's only run when a
release is made, again.
Huh? So, let's recap: the fetch-javascript-tarball
role which produces the
tripleo-ui-image.tar.gz
is only run by playbooks/roles that are triggered on
releases. Well, dear reader, this is the cause of the problem of updating the
latest file. No one runs it in post
.
Given that our project inexplicably ships Python wheels, I this this was done by
an unsuspecting developer who saw a setup.py
file in our project's root.
project-config sha: ad40365e3b9258d191238d52e6ce4c4592cd70f8
openstack-zuul-jobs sha: f63942d86ca86acfdfc670baea04c2fb7ccdf9f0
zuul jobs sha 62836959666e8032738110be854303e52bc954d0