Skip to content

Instantly share code, notes, and snippets.

@alopresto
Created June 19, 2019 02:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save alopresto/b596372005032b9c80063a0652a955ea to your computer and use it in GitHub Desktop.
Save alopresto/b596372005032b9c80063a0652a955ea to your computer and use it in GitHub Desktop.
Excerpted and annotated log output from NiFi InvokeHTTP.
# This is the LogAttribute in the "HandleHTTPRequest/UpdateAttribute/HandleHTTPResponse" flow (on the right). Please note that the flowfile content is present (the JSON snippet with id and message keys), but the attribute1 and authorization values are not present. They were not sent via HTTP request.
2019-06-18 19:46:50,750 INFO [Timer-Driven Process Thread-5] o.a.n.processors.standard.LogAttribute LogAttribute[id=6d9bac97-016b-1000-1aab-863142f46b52] logging for flow file StandardFlowFileRecord[uuid=b2c07e72-6f23-41b2-9910-78a71d590b54,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1560912333372-1, container=default, section=1], offset=251, length=56],offset=0,name=b2c07e72-6f23-41b2-9910-78a71d590b54,size=56]
--------------------------------------------------
Standard FlowFile Attributes
Key: 'entryDate'
Value: 'Tue Jun 18 19:46:50 PDT 2019'
Key: 'lineageStartDate'
Value: 'Tue Jun 18 19:46:50 PDT 2019'
Key: 'fileSize'
Value: '56'
FlowFile Attribute Map Content
Key: 'filename'
Value: 'b2c07e72-6f23-41b2-9910-78a71d590b54'
Key: 'http.context.identifier'
Value: '5ae1b488-fc88-4de3-bbf5-438c4f8bc712'
Key: 'http.headers.Accept-Encoding'
Value: 'gzip'
Key: 'http.headers.Connection'
Value: 'keep-alive'
Key: 'http.headers.Content-Length'
Value: '56'
Key: 'http.headers.Content-Type'
Value: 'application/octet-stream'
Key: 'http.headers.Date'
Value: 'Wed, 19 Jun 2019 02:46:50 GMT'
Key: 'http.headers.Host'
Value: 'localhost:9999'
Key: 'http.headers.User-Agent'
Value: 'okhttp/3.8.1'
Key: 'http.local.addr'
Value: '127.0.0.1'
Key: 'http.local.name'
Value: '127.0.0.1'
Key: 'http.locale'
Value: 'en_US'
Key: 'http.method'
Value: 'POST'
Key: 'http.protocol'
Value: 'HTTP/1.1'
Key: 'http.remote.addr'
Value: '127.0.0.1'
Key: 'http.remote.host'
Value: '127.0.0.1'
Key: 'http.request.uri'
Value: '/contentListener'
Key: 'http.request.url'
Value: 'http://localhost:9999/contentListener'
Key: 'http.server.name'
Value: 'localhost'
Key: 'http.server.port'
Value: '9999'
Key: 'http.servlet.path'
Value: ''
Key: 'mime.type'
Value: 'application/octet-stream'
Key: 'path'
Value: './'
Key: 'uuid'
Value: 'b2c07e72-6f23-41b2-9910-78a71d590b54'
--------------------------------------------------
{"id": "1", "message": "This is the flowfile content. "}
# This is the LogAttribute in the "GenerateFlowFile/InvokeHTTP" flow segment. Please note that the attribute1 and authorization values are present here as flowfile attributes.
2019-06-18 19:46:50,781 INFO [Timer-Driven Process Thread-6] o.a.n.processors.standard.LogAttribute LogAttribute[id=016b1000-ac97-1d9b-f15a-d927cc7df8b0] logging for flow file StandardFlowFileRecord[uuid=807802d5-6950-4380-8eec-24f473ee7344,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1560912333372-1, container=default, section=1], offset=307, length=56],offset=0,name=df1925f6-d969-4ff7-9a47-1bb25510a4ca,size=56]
--------------------------------------------------
Standard FlowFile Attributes
Key: 'entryDate'
Value: 'Tue Jun 18 19:46:50 PDT 2019'
Key: 'lineageStartDate'
Value: 'Tue Jun 18 19:46:50 PDT 2019'
Key: 'fileSize'
Value: '56'
FlowFile Attribute Map Content
Key: 'Date'
Value: 'Wed, 19 Jun 2019 02:46:50 GMT'
Key: 'Server'
Value: 'Jetty(9.4.11.v20180605)'
Key: 'Transfer-Encoding'
Value: 'chunked'
Key: 'attribute1'
Value: 'This is the value of "attribute1". '
Key: 'authorization'
Value: 'Bearer abcdefg...'
Key: 'filename'
Value: 'df1925f6-d969-4ff7-9a47-1bb25510a4ca'
Key: 'invokehttp.request.url'
Value: 'http://localhost:9999/contentListener'
Key: 'invokehttp.status.code'
Value: '200'
Key: 'invokehttp.status.message'
Value: 'OK'
Key: 'invokehttp.tx.id'
Value: '9117c184-fbe9-4748-9ee0-b6d50a35ef4a'
Key: 'path'
Value: './'
Key: 'uuid'
Value: '807802d5-6950-4380-8eec-24f473ee7344'
--------------------------------------------------
{"id": "1", "message": "This is the flowfile content. "}
# At this time, I changed the configuration of the InvokeHTTP processor "Attributes to Send" property descriptor from <empty> to ".*" (no quotes). This tells the processor to send all attributes as request headers.
# This is the LogAttribute in the "HandleHTTPRequest/UpdateAttribute/HandleHTTPResponse" flow (on the right). Please note that the flowfile content is present (the JSON snippet with id and message keys), AND the attribute1 and authorization values are NOW present. They were sent via HTTP request.
2019-06-18 19:53:59,138 INFO [Timer-Driven Process Thread-10] o.a.n.processors.standard.LogAttribute LogAttribute[id=6d9bac97-016b-1000-1aab-863142f46b52] logging for flow file StandardFlowFileRecord[uuid=31a94fd9-2652-4b50-83c5-b79313448887,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1560912333372-1, container=default, section=1], offset=587, length=56],offset=0,name=31a94fd9-2652-4b50-83c5-b79313448887,size=56]
--------------------------------------------------
Standard FlowFile Attributes
Key: 'entryDate'
Value: 'Tue Jun 18 19:53:59 PDT 2019'
Key: 'lineageStartDate'
Value: 'Tue Jun 18 19:53:59 PDT 2019'
Key: 'fileSize'
Value: '56'
FlowFile Attribute Map Content
Key: 'filename'
Value: '31a94fd9-2652-4b50-83c5-b79313448887'
Key: 'http.context.identifier'
Value: '89416894-babe-4979-9f1f-0dc677fb7a00'
Key: 'http.headers.Accept-Encoding'
Value: 'gzip'
Key: 'http.headers.Authorization'
Value: 'Bearer abcdefg...'
Key: 'http.headers.Connection'
Value: 'keep-alive'
Key: 'http.headers.Content-Length'
Value: '56'
Key: 'http.headers.Content-Type'
Value: 'application/octet-stream'
Key: 'http.headers.Date'
Value: 'Wed, 19 Jun 2019 02:53:59 GMT'
Key: 'http.headers.Host'
Value: 'localhost:9999'
Key: 'http.headers.User-Agent'
Value: 'okhttp/3.8.1'
Key: 'http.headers.attribute1'
Value: 'This is the value of "attribute1".'
Key: 'http.local.addr'
Value: '127.0.0.1'
Key: 'http.local.name'
Value: '127.0.0.1'
Key: 'http.locale'
Value: 'en_US'
Key: 'http.method'
Value: 'POST'
Key: 'http.protocol'
Value: 'HTTP/1.1'
Key: 'http.remote.addr'
Value: '127.0.0.1'
Key: 'http.remote.host'
Value: '127.0.0.1'
Key: 'http.request.uri'
Value: '/contentListener'
Key: 'http.request.url'
Value: 'http://localhost:9999/contentListener'
Key: 'http.server.name'
Value: 'localhost'
Key: 'http.server.port'
Value: '9999'
Key: 'http.servlet.path'
Value: ''
Key: 'mime.type'
Value: 'application/octet-stream'
Key: 'path'
Value: './'
Key: 'uuid'
Value: '31a94fd9-2652-4b50-83c5-b79313448887'
--------------------------------------------------
{"id": "1", "message": "This is the flowfile content. "}
# This is the LogAttribute in the "GenerateFlowFile/InvokeHTTP" flow segment. Please note that the attribute1 and authorization values are present here as flowfile attributes.
2019-06-18 19:53:59,169 INFO [Timer-Driven Process Thread-5] o.a.n.processors.standard.LogAttribute LogAttribute[id=016b1000-ac97-1d9b-f15a-d927cc7df8b0] logging for flow file StandardFlowFileRecord[uuid=a198d751-7266-40d9-ab9a-8e46a38666e4,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1560912333372-1, container=default, section=1], offset=643, length=56],offset=0,name=9e935f26-d538-43ba-bfe5-1edbf0f895c7,size=56]
--------------------------------------------------
Standard FlowFile Attributes
Key: 'entryDate'
Value: 'Tue Jun 18 19:53:59 PDT 2019'
Key: 'lineageStartDate'
Value: 'Tue Jun 18 19:53:59 PDT 2019'
Key: 'fileSize'
Value: '56'
FlowFile Attribute Map Content
Key: 'Date'
Value: 'Wed, 19 Jun 2019 02:53:59 GMT'
Key: 'Server'
Value: 'Jetty(9.4.11.v20180605)'
Key: 'Transfer-Encoding'
Value: 'chunked'
Key: 'attribute1'
Value: 'This is the value of "attribute1". '
Key: 'authorization'
Value: 'Bearer abcdefg...'
Key: 'filename'
Value: '9e935f26-d538-43ba-bfe5-1edbf0f895c7'
Key: 'invokehttp.request.url'
Value: 'http://localhost:9999/contentListener'
Key: 'invokehttp.status.code'
Value: '200'
Key: 'invokehttp.status.message'
Value: 'OK'
Key: 'invokehttp.tx.id'
Value: '4686a584-01c6-4f9c-b6ba-dd985d569f74'
Key: 'path'
Value: './'
Key: 'uuid'
Value: 'a198d751-7266-40d9-ab9a-8e46a38666e4'
--------------------------------------------------
{"id": "1", "message": "This is the flowfile content. "}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment