This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <?xml version="1.0" encoding="UTF-8" standalone="yes"?> | |
| <!-- | |
| Licensed to the Apache Software Foundation (ASF) under one or more | |
| contributor license agreements. See the NOTICE file distributed with | |
| this work for additional information regarding copyright ownership. | |
| The ASF licenses this file to You under the Apache License, Version 2.0 | |
| (the "License"); you may not use this file except in compliance with | |
| the License. You may obtain a copy of the License at | |
| http://www.apache.org/licenses/LICENSE-2.0 | |
| Unless required by applicable law or agreed to in writing, software |
alopresto
/ Process.java
Created
September 28, 2016 00:08
A Java class which prints the provided arguments to both System.out and System.err for debugging purposes
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package org.apache.nifi.process; | |
| import java.util.Arrays; | |
| public class Process { | |
| public static void main(String[] args) { | |
| String allArgs = String.join(" ", Arrays.asList(args)); | |
| System.out.println("[System out] Provided arguments: " + allArgs); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| echo "Bash before Java invocation '$@'" | |
| java -cp /Users/alopresto/Workspace/scratch/process-test/target/process-test-1.0-SNAPSHOT.jar org.apache.nifi.process.Process "$@" | |
| echo "Bash after Java invocation '$@'" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| hw12203:/Users/alopresto/Workspace/scratch/process-test (master) alopresto | |
| 🔒 142s @ 17:09:54 $ ll | |
| total 40 | |
| drwxr-xr-x 9 alopresto staff 306B Sep 27 17:07 ./ | |
| drwxr-xr-x 48 alopresto staff 1.6K Sep 27 16:49 ../ | |
| -rw-r--r--@ 1 alopresto staff 6.0K Sep 27 17:06 .DS_Store | |
| drwxr-xr-x 11 alopresto staff 374B Sep 27 17:09 .idea/ | |
| -rw-r--r-- 1 alopresto staff 642B Sep 27 16:56 pom.xml | |
| -rw-r--r-- 1 alopresto staff 1.0K Sep 27 17:03 process-test-java.iml | |
| -rwxr-xr-x 1 alopresto staff 82B Sep 27 17:07 script.sh* |
alopresto
/ Output.log
Created
September 28, 2016 00:27
Excerpt from `logs/nifi-app.log` showing the flowfile attributes and content including bash `echo` output
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2016-09-27 17:23:13,970 INFO [StandardProcessScheduler Thread-1] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled ExecuteProcess[id=6e0d4685-0157-1000-1616-34e3ea888ad6] to run with 1 threads | |
| 2016-09-27 17:23:13,974 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled LogAttribute[id=6e22ae1c-0157-1000-96e3-9be4680eb151] to run with 1 threads | |
| 2016-09-27 17:23:14,340 INFO [Timer-Driven Process Thread-1] o.a.n.processors.standard.LogAttribute LogAttribute[id=6e22ae1c-0157-1000-96e3-9be4680eb151] logging for flow file StandardFlowFileRecord[uuid=9442ca93-3f3c-40c9-a23f-6b2e6707eba6,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1475019971713-1, container=default, section=1], offset=1751, length=149],offset=0,name=70522280527566,size=149] | |
| -------------------------------------------------- | |
| Standard FlowFile Attributes | |
| Key: 'entryDate' | |
| Value: 'Tue Sep 27 17:23:14 PDT 2016' | |
| Key: 'lineageStartDate' | |
| Value: 'Tue Sep 27 17:23:14 PDT 2016' | |
| Key: 'fileSize' |
alopresto
/ execute_and_print.xml
Created
September 28, 2016 00:30
NiFi template containing a `ExecuteProcess` processor connected to a `LogAttribute` processor.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <?xml version="1.0" ?> | |
| <template encoding-version="1.0"> | |
| <description></description> | |
| <groupId>459549da-0157-1000-e381-c7ba87f8ff61</groupId> | |
| <name>ExecuteProcess and print output</name> | |
| <snippet> | |
| <connections> | |
| <id>6e22e0ec-0157-1000-0000-000000000000</id> | |
| <parentGroupId>459549da-0157-1000-0000-000000000000</parentGroupId> | |
| <backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold> |
alopresto
/ Output.log
Created
September 28, 2016 00:31
Excerpt of `logs/nifi-app.log` showing the output of a Java program invoked via bash script.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2016-09-27 17:17:59,973 INFO [Timer-Driven Process Thread-3] o.a.n.processors.standard.LogAttribute LogAttribute[id=6e22ae1c-0157-1000-96e3-9be4680eb151] logging for flow file StandardFlowFileRecord[uuid=9748ab80-3c9a-4598-be9d-4dc297975ab3,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1475019971713-1, container=default, section=1], offset=1595, length=78],offset=0,name=70207909674263,size=78] | |
| -------------------------------------------------- | |
| Standard FlowFile Attributes | |
| Key: 'entryDate' | |
| Value: 'Tue Sep 27 17:17:59 PDT 2016' | |
| Key: 'lineageStartDate' | |
| Value: 'Tue Sep 27 17:17:59 PDT 2016' | |
| Key: 'fileSize' | |
| Value: '78' | |
| FlowFile Attribute Map Content |
alopresto
/ expression_language_boolean.xml
Created
October 7, 2016 18:39
NiFi template: Generates an empty flowfile, uses a Groovy script to randomly assign either "test" or "prod" to the "db.table.name" attribute, updates the "expression" attribute with the (String) result of a boolean comparison of "db.table.name" to "test", and routes to "test" or "prod" based on the boolean evaluation of "expression".
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <?xml version="1.0" ?> | |
| <template encoding-version="1.0"> | |
| <description>Generates an empty flowfile, uses a Groovy script to randomly assign either "test" or "prod" to the "db.table.name" attribute, updates the "expression" attribute with the (String) result of a boolean comparison of "db.table.name" to "test", and routes to "test" or "prod" based on the boolean evaluation of "expression". </description> | |
| <groupId>0157100e-ae1c-1e22-9d02-c3092b285655</groupId> | |
| <name>Expression Language Boolean Evaluation</name> | |
| <snippet> | |
| <connections> | |
| <id>01571004-ae1c-1e22-0000-000000000000</id> | |
| <parentGroupId>0157100e-ae1c-1e22-0000-000000000000</parentGroupId> | |
| <backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold> |