rfairley/updating-fedora-coreos-pinger.md

## updating-fedora-coreos-pinger.md

      
    Raw
  

              updating-fedora-coreos-pinger.md
            
          
    !!! Note: In process of reworking this - since modules are no longer being used for Rust packages, only an update to the RPM is necessary, but different from the usual workflow for RPM packages. Pending information from Fedora Rust folks on what the new processes without modules is.
Updating fedora-coreos-pinger

I run these steps from a fedora:rawhide container, with the RPM build tools installed inside the container.
To start and provision the container:
podman run --name=rust-dev-rawhide --privileged --net=host -v /path/to/working-directory:/srv/work -ti registry.fedoraproject.org/fedora:rawhide

# Run the following inside the shell for the container
dnf -y install rust-packaging dnf-plugins-core python3-rust2rpm python3-solv cargo rust fedora-packager fedpkg krb5-workstation rpmdevtools rpm-build git
# Add your git author info
git config --global --edit
# Authentication needed for fedpkg
kinit <your FAS id>@FEDORAPROJECT.ORG

Updating the package in Fedora


Clone the pinger package to your /path/to/working-directory (located at https://src.fedoraproject.org/rpms/rust-fedora-coreos-pinger):
fedpkg clone rust-fedora-coreos-pinger
cd rust-fedora-coreos-pinger


Update the specfile with the new version, changelog entry, and any other parts as needed. Then, execute:
# Download the crate specified by the version in the specfile.
spectool -g rust-fedora-coreos-pinger.spec
# Upload the new crate to Fedora package sources. This will also update and stage the `.gitignore` and `sources` files.
fedpkg new-sources

(see an example where this is done in Zincati: https://src.fedoraproject.org/rpms/rust-zincati/c/f977d18ca590e84b15d989b32347522945838182?branch=master).


Build the package locally:
fedpkg local

If the package doesn't build at first, it may be that dependencies specified by DynamicBuildRequires are not present on the system. Run the following on the .buildreqs.nosrc.rpm file that fedpkg local outputted (there should be a message in the console indicating the filename).
dnf -y builddep ./rust-fedora-coreos-pinger*.buildreqs.nosrc.rpm
fedpkg local

If the package still doesn't build, it may be that the crate dependencies are missing in Fedora, or are not yet updated to the version requested in fedora-coreos-pinger's Cargo.toml. Normally, relaxing the requirement to the version present in Fedora is sufficient. To do this, a patch to Cargo.toml needs to be applied by rust-fedora-coreos-pinger.spec. An example of this is the zincati-fix-metadata.diff file in the example linked in Step 2. An easy way to generate the patch is to do (outside of your rust-fedora-coreos-pinger checkout directory):
rust2rpm fedora-coreos-pinger --dynamic-buildrequires -p

In the editor, edit the manifest with the correct versions present in Fedora. Then add the fedora-coreos-pinger-fix-metadata.diff file to the rust-fedora-coreos-pinger repo and apply it as a patch from the specfile (see example: https://src.fedoraproject.org/rpms/rust-zincati/blob/f977d18ca590e84b15d989b32347522945838182/f/rust-zincati.spec#_17).
You can check what version of a rust crate exists in Fedora by searching rust-<crate name> in koji.fedoraproject.org (note there are some naming exceptions like https://src.fedoraproject.org/rpms/rust-futures0.1). If you need a more updated version, then the crate in Fedora will need patching with the update (you can open a PR to the crate's RPM package, ask in #fedora-rust on IRC for help). If the crate does not exist in Fedora, then it will need to be requested and accepted as a package in Fedora (see steps: https://fedoraproject.org/wiki/Join_the_package_collection_maintainers#Make_a_Package).
As a final sanity check, run: the following to catch common mistakes in updating the specfile:
rpmlint rust-fedora-coreos-pinger.spec


If the package builds successfully locally, you can verify the package in FCOS by adding as an override RPM in a local cosa build (see: https://github.com/coreos/coreos-assembler#overriding-rpms-using-new-different-or-locally-built-rpms). If all looks OK, run the following from the rust-fedora-coreos-pinger directory:
# Commit your changes for updating the specfile
git commit
# Push the update commit
fedpkg push
# Build the package in Koji
fedkg build


Updating the module in Fedora

(mostly following https://docs.fedoraproject.org/en-US/modularity/making-modules/updating-modules/#_updating_the_module, but for Rust there are tools to make updating the modulemd easier)


Clone the pinger module to your working directory:
fedpkg clone modules/fedora-coreos-pinger
cd fedora-coreos-pinger
fedpkg switch-branch rolling


If the upstream version changed (e.g. 0.0.3-2 -> 0.0.4-1), the modulemd will need updating. To do this, use the modularity-utils script.
git clone https://pagure.io/fedora-rust/modularity-utils.git

Then, follow the instructions in the README.md (replacing ripgrep with fedora-coreos-pinger).
Note that the updated rust-fedora-coreos-pinger RPM package might not be present in the Fedora Rawhide repos yet, since they are pulled in through a nightly compose. You can try the Koji repo instead, which may have the latest builds:
dnf --repofrompath=koji,https://kojipkgs.fedoraproject.org/repos/rawhide/latest/x86_64/ --repofrompath=koji-source,https://kojipkgs.fedoraproject.org/repos/rawhide/latest/src/ --nogpgcheck makecache

Then update ./generate-modulemd.py to point towards the koji and koji-src repos:
--- a/generate-modulemd.py
+++ b/generate-modulemd.py
@@ -65,12 +65,12 @@ pool = solv.Pool()
pool.setarch()

rbin = pool.add_repo('binary')
-f = solv.xfopen('/var/cache/dnf/rawhide.solv')
+f = solv.xfopen('/var/cache/dnf/koji.solv')
rbin.add_solv(f)
f.close()

rsrc = pool.add_repo('source')
-f = solv.xfopen('/var/cache/dnf/rawhide-source.solv')
+f = solv.xfopen('/var/cache/dnf/koji-source.solv')
rsrc.add_solv(f)
f.close()

Then regenerate the modulemd:
./generate-modulemd.py fedora-coreos-pinger > fedora-coreos-pinger.yaml

If dependencies still do not resolve when running the ./generate-modulemd.py script, folks in Freenode #fedora-rust will be able to help.
In progress script (which works, but code needs refactoring) for generating modulemd: https://pagure.io/fork/rfairley/fedora-rust/modularity-utils/c/9497e14ded3b5e36a159e5d497d3a389130eaf14?branch=rfairley-genmmd


Once the modulemd is generated, add it to the fedora-coreos-pinger module repo. Commit and push the changes. An example of what this looks like is: https://src.fedoraproject.org/modules/zincati/c/98a464371d4f3a7a9b16e377690ab9c6b7b99e71?branch=rolling


Build the module in Fedora. From the repo, execute:
fedpkg module-build

Sometimes, individual components can fail from a missing dependency, causing the module build to fail. To investigate, you can view the build at https://release-engineering.github.io/mbs-ui/modules. The build can also sometimes just hang due to an infra flake - in this case, just execute fedpkg module-build again to try to continue the build.


Once built, create a Bodhi update for the module as follows: https://docs.fedoraproject.org/en-US/modularity/making-modules/updating-modules/#_publishing_the_module.


Tagging the update in coreos-pool

Tagging the update in coreos-pool has the benefit of pulling the new pinger module version into testing builds of FCOS, before the Bodhi update for the module is pushed to stable (which can take 7-10 days).
To tag the module in the pool, we update a lockfile manifest-lock.overridesx86_64.json which is monitored by FCOS releng automation. Open a PR to the lockfile in the testing-devel branch, with an entry like so:
"zincati": {
  "evra": "0.0.5-1.module_f30+6699+1eaa40c6.x86_64"
}

An example PR where this is done is:
coreos/fedora-coreos-config#186
The releng automation will detect a change to the lockfile, and tag the specified package into coreos-pool once the PR is merged.