Skip to content

Instantly share code, notes, and snippets.

@dparalen

dparalen/pulp2_modular_errata_upload_howto.md

Last active September 7, 2018 15:26
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dparalen/779f8018af1d7e090bb380a34acce851 to your computer and use it in GitHub Desktop.
Save dparalen/779f8018af1d7e090bb380a34acce851 to your computer and use it in GitHub Desktop.
Download ZIP
How to upload an erratum into Pulp2 via the REST API
Raw
pulp2_modular_errata_upload_howto.md

About

This gist is based on the upload errata recipe, blendend together with the REST API content upload specification, with a quantum of debugging the pulp-admin command line utility.

Goal

Upload an erratum (JSON) into Pulp2 via the REST API, using httpie <3. This gist was run on a Pulp2 development environment

Howto

  • Create an (empty) upload object on the server: http POST https://admin:admin@pulp2.dev/pulp/api/v2/content/uploads/
HTTP/1.1 201 CREATED
Connection: Keep-Alive
Content-Length: 132
Content-Type: application/json; charset=utf-8
Date: Thu, 06 Sep 2018 13:37:34 GMT
Keep-Alive: timeout=5, max=10000
Location: /pulp/api/v2/content/uploads/14240437-c8a2-4b48-bf21-5c3a56361c28/
Server: Apache/2.4.33 (Fedora) OpenSSL/1.1.0h-fips mod_wsgi/4.5.15 Python/2.7

{
    "_href": "/pulp/api/v2/content/uploads/14240437-c8a2-4b48-bf21-5c3a56361c28/",
    "upload_id": "14240437-c8a2-4b48-bf21-5c3a56361c28"
}
  • create a metadata file, and the erratum unit: cat walrus_errata.json
{ "status": "stable",
  "updated": "2018-09-05 11:23:42",
  "rights": null,
  "from": "betelgeuze",
  "description": "Walrus Erratum; metadata-only",
  "title": "Walrus",
  "issued": "2002-02-20 00:22:00",
  "relogin_suggested": false,
  "restart_suggested": false,
  "solution": null,
  "summary": null,
  "pushcount": "1",
  "version": "1",
  "references": [],
  "release": "1",
  "reboot_suggested": false,
  "type": "enhancement",
  "severity": null,
  "pkglist": [
    { "name": "collection 0",
      "short": "0",
      "module": {
        "name": "walrus",
        "stream": "5.21",
        "version": "20180704144203",
        "arch": "x86_64",
        "context": "deadbeef"
      },
      "packages": []
    }
  ]
}
  • Create an import task: http POST https://admin:admin@pulp2.dev/pulp/api/v2/repos itories/modulerr/actions/import_upload/ unit_type_id=erratum upload_id=14240437-c8a2-4b 48-bf21-5c3a56361c28 unit_key:='{"id": "RHEA-2018:0042"}' unit_metadata:=@walrus_errata.json
HTTP/1.1 202 Accepted
Connection: Keep-Alive
Content-Length: 172
Content-Type: application/json; charset=utf-8
Date: Fri, 07 Sep 2018 15:00:56 GMT
Keep-Alive: timeout=5, max=10000
Server: Apache/2.4.33 (Fedora) OpenSSL/1.1.0h-fips mod_wsgi/4.5.15 Python/2.7

{
    "error": null,
    "result": null,
    "spawned_tasks": [
        {
            "_href": "/pulp/api/v2/tasks/89144206-51da-4a57-8bcf-e8e78f233fa2/",
            "task_id": "89144206-51da-4a57-8bcf-e8e78f233fa2"
        }
    ]
}
  • check the import task result: http GET https://admin:admin@pulp2.dev/pulp/api/v2/tasks/89144206-51da-4a57-8bcf-e8e78f233fa2/
HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 728
Content-Type: application/json; charset=utf-8
Date: Fri, 07 Sep 2018 15:03:46 GMT
ETag: "ba883dc86114eca7a5fd7123c21e87bc"
Keep-Alive: timeout=5, max=10000
Server: Apache/2.4.33 (Fedora) OpenSSL/1.1.0h-fips mod_wsgi/4.5.15 Python/2.7

{
    "_href": "/pulp/api/v2/tasks/89144206-51da-4a57-8bcf-e8e78f233fa2/",
    "_id": {
        "$oid": "5b9292a87f513442b074bb86"
    },
    "_ns": "task_status",
    "error": null,
    "exception": null,
    "finish_time": "2018-09-07T15:00:56Z",
    "id": "5b9292a87f513442b074bb86",
    "progress_report": {},
    "queue": "reserved_resource_worker-4@pulp2.dev.dq2",
    "result": {
        "details": {},
        "success_flag": true,
        "summary": ""
    },
    "spawned_tasks": [],
    "start_time": "2018-09-07T15:00:56Z",
    "state": "finished",
    "tags": [
        "pulp:repository:modulerr",
        "pulp:action:import_upload"
    ],
    "task_id": "89144206-51da-4a57-8bcf-e8e78f233fa2",
    "task_type": "pulp.server.managers.content.upload.import_uploaded_unit",
    "traceback": null,
    "worker_name": "reserved_resource_worker-4@pulp2.dev"
}

  • Publish the repository to check the upload worked OK pulp-admin rpm repo publish run --repo-id modulerr --force-full

  • Check the published errata: sudo zcat /var/lib/pulp/published/yum/master/yum_distributor/ modulerr/1536331091.85/repodata/180f1a280b541444001a3960d818cc154cde2cf885f967f42355f0a e52d4590d-updateinfo.xml.gz

<?xml version="1.0" encoding="utf-8"?>
<updates>
  <update status="stable" from="betelgeuze" version="1" type="enhancement">
    <id>RHEA-2018:0042</id>
    <issued date="2002-02-20 00:22:00" />
    <title>Walrus</title>
    <release>1</release>
    <pushcount>1</pushcount>
    <description>Walrus Erratum; metadata-only</description>
    <updated date="2018-09-05 11:23:42" />
    <references />
    <pkglist>
      <collection short="modulerr_1_walrus">
        <name>modulerr_1_walrus</name>
        <module context="deadbeef" version="20180704144203" arch="x86_64" name="walrus" stream="5.21" />
      </collection>
    </pkglist>
  </update>
  <!-- ------\>%---------- -->
</updates>

Notes

  • For the modular erratum to publish, there has to be a module associated with the repository the errata are being imported into

  • Packages mentioned in the errata collections are also filtered out publish-time unless already associated with the repository

  • The CLI pulp-admin command I used to base this gist on: [vagrant@pulp2 pulp_rpm]$ pulp-admin -vvv rpm repo uploads erratum --repo-id modulerr --erratum-id RHEA-2018:0042 --title Kangaroo_DUCK_Erratum --description "Kangaroo DUCK Erratum Description" --version 1 --release 1 --updated "2018-09-05 11:23:42" --issued "2002-02-20 00:22:00" --from betelgeuze --type enhancement --status stable --pkglist-csv packagelist.csv

+----------------------------------------------------------------------+
                              Unit Upload
+----------------------------------------------------------------------+

2018-09-06 13:08:01,863 - DEBUG - sending GET request to /pulp/api/v2/repositories/modu
lerr/
2018-09-06 13:08:01,908 - INFO - GET request to /pulp/api/v2/repositories/modulerr/ wit
h parameters None
2018-09-06 13:08:01,908 - INFO - Response status : 200

2018-09-06 13:08:01,908 - INFO - Response body :
 {
  "scratchpad": {
    "checksum_type": "sha256"
  },
  "display_name": null,
  "description": null,
  "last_unit_added": "2018-09-06T13:07:22Z",
  "notes": {
    "_repo-type": "rpm-repo"
  },
  "last_unit_removed": null,
  "content_unit_counts": {
    "modulemd_defaults": 3,
    "erratum": 7,
    "package_category": 1,
    "package_group": 2,
    "modulemd": 7,
    "package_langpacks": 1,
    "rpm": 35
  },
  "_ns": "repos",
  "_id": {
    "$oid": "5b87f6fa30f2524800fc330d"
  },
  "id": "modulerr",
  "_href": "/pulp/api/v2/repositories/modulerr/"
}

Extracting necessary metadata for each request...
... completed

Creating upload requests on the server...
[==================================================] 100%
Initializing upload
2018-09-06 13:08:01,909 - DEBUG - sending POST request to /pulp/api/v2/content/uploads/
2018-09-06 13:08:01,951 - INFO - POST request to /pulp/api/v2/content/uploads/ with par
ameters None
2018-09-06 13:08:01,952 - INFO - Response status : 201

2018-09-06 13:08:01,952 - INFO - Response body :
 {
  "upload_id": "fc78bd95-5b80-492a-881e-900e66c0386b",
  "_href": "/pulp/api/v2/content/uploads/fc78bd95-5b80-492a-881e-900e66c0386b/"
}

... completed

Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
canceled entirely using the cancel command.

Importing into the repository...
2018-09-06 13:08:01,953 - DEBUG - sending POST request to /pulp/api/v2/repositories/mod
ulerr/actions/import_upload/
2018-09-06 13:08:02,046 - INFO - POST request to /pulp/api/v2/repositories/modulerr/act
ions/import_upload/ with parameters {"override_config": {}, "unit_type_id": "erratum",
"upload_id": "fc78bd95-5b80-492a-881e-900e66c0386b", "unit_key": {"id": "RHEA-2018:0042
"}, "unit_metadata": {"status": "stable", "updated": "2018-09-05 11:23:42", "rights": n
ull, "from": "betelgeuze", "description": "Kangaroo DUCK Erratum Description", "title":
 "Kangaroo_DUCK_Erratum", "issued": "2002-02-20 00:22:00", "relogin_suggested": false,
"restart_suggested": false, "solution": null, "summary": null, "pushcount": "1", "versi
on": "1", "references": [], "release": "1", "reboot_suggested": false, "type": "enhance
ment", "pkglist": [{"packages": [{"src": "duck-0.7-1.noarch.src.rpm", "name": "duck", "
arch": "noarch", "sums": "ceb0f0bb58be244393cc565e8ee5ef0ad36884d8ba8eec74542ff47d299a3
4c1", "filename": "duck-0.7-1.noarch.rpm", "epoch": "1", "version": "0", "release": "0.
7", "type": "sha256"}], "name": "1", "short": ""}], "severity": null}}
2018-09-06 13:08:02,046 - INFO - Response status : 202

2018-09-06 13:08:02,046 - INFO - Response body :
 {
  "spawned_tasks": [
    {
      "_href": "/pulp/api/v2/tasks/648a13fe-4d58-4e30-8a3b-0c3d0b964a7f/",
      "task_id": "648a13fe-4d58-4e30-8a3b-0c3d0b964a7f"
    }
  ],
  "result": null,
  "error": null
}

This command may be exited via ctrl+c without affecting the request.


[-]
Running...
2018-09-06 13:08:03,048 - DEBUG - sending GET request to /pulp/api/v2/tasks/648a13fe-4d
58-4e30-8a3b-0c3d0b964a7f/
2018-09-06 13:08:03,091 - INFO - GET request to /pulp/api/v2/tasks/648a13fe-4d58-4e30-8
a3b-0c3d0b964a7f/ with parameters None
2018-09-06 13:08:03,091 - INFO - Response status : 200

2018-09-06 13:08:03,091 - INFO - Response body :
 {
  "exception": null,
  "task_type": "pulp.server.managers.content.upload.import_uploaded_unit",
  "_href": "/pulp/api/v2/tasks/648a13fe-4d58-4e30-8a3b-0c3d0b964a7f/",
  "task_id": "648a13fe-4d58-4e30-8a3b-0c3d0b964a7f",
  "tags": [
    "pulp:repository:modulerr",
    "pulp:action:import_upload"
  ],
  "finish_time": "2018-09-06T13:08:02Z",
  "_ns": "task_status",
  "start_time": "2018-09-06T13:08:02Z",
  "traceback": null,
  "spawned_tasks": [],
  "progress_report": {},
  "queue": "reserved_resource_worker-4@pulp2.dev.dq2",
  "state": "finished",
  "worker_name": "reserved_resource_worker-4@pulp2.dev",
  "result": {
    "details": {},
    "success_flag": true,
    "summary": ""
  },
  "error": null,
    "_id": {
    "$oid": "5b9126b17f513442b0683017"
  },
  "id": "5b9126b17f513442b0683017"
[\]
Running...

Task Succeeded


Deleting the upload request...
2018-09-06 13:08:03,092 - DEBUG - sending DELETE request to /pulp/api/v2/content/upload
s/fc78bd95-5b80-492a-881e-900e66c0386b/
2018-09-06 13:08:03,135 - INFO - DELETE request to /pulp/api/v2/content/uploads/fc78bd9
5-5b80-492a-881e-900e66c0386b/ with parameters None
2018-09-06 13:08:03,136 - INFO - Response status : 200

2018-09-06 13:08:03,136 - INFO - Response body :
 null

... completed
  • The contents of packagelist.csv:
duck,0,0.7,1,noarch,duck-0.7-1.noarch.rpm,ceb0f0bb58be244393cc565e8ee5ef0ad36884d8ba8eec74542ff47d299a34c1,sha256,duck-0.7-1.noarch.src.rpm
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment