Skip to content

Instantly share code, notes, and snippets.

@mattyb149
mattyb149 / settings.xml
Created May 13, 2015 15:33
settings.xml
<settings xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd">
<profiles>
<profile>
<id>pentaho</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<repositories>
@mattyb149
mattyb149 / marketplaces.xml
Created July 7, 2015 18:06
Custom Marketplaces file for PDI
<!-- Place this file in your $HOME/.kettle directory -->
<marketplaces>
<!--<marketplace>
<name>GitHub</name>
<entries_url>https://raw.githubusercontent.com/pentaho/marketplace-metadata/master/marketplace.xml</entries_url>
</marketplace>-->
<marketplace>
<name>MattyB</name>
<entries_url>/Users/mburgess/git/marketplace-metadata/marketplace.xml</entries_url>
</marketplace>
@mattyb149
mattyb149 / postal_code.ktr
Created July 8, 2015 04:20
Get Postal Code from Google Places
<?xml version="1.0" encoding="UTF-8"?>
<transformation>
<info>
<name>postal_code</name>
<description/>
<extended_description/>
<trans_version/>
<trans_type>Normal</trans_type>
<directory>&#x2f;</directory>
<parameters>
@mattyb149
mattyb149 / onStart.groovy
Created July 10, 2015 19:17
Add Drag & Drop support for CSV files in PDI
import org.w3c.dom.*
import org.pentaho.di.core.*
import org.pentaho.di.core.exception.*
import org.pentaho.di.core.gui.*
import org.pentaho.di.core.plugins.*
import org.pentaho.di.trans.step.*
import org.pentaho.di.ui.spoon.*
class CsvListener implements FileListener {
@mattyb149
mattyb149 / ExecuteScriptHelloWorldGroovy.xml
Created February 10, 2016 18:43
Hello World example for NiFi ExecuteScript processor (with Groovy)
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><template><description>This is a self-contained example of using a Groovy script in an ExecuteScript processor to update the filename attribute of an incoming flowfile. A dummy JSON file is created (which is overkill for this instance, but is included to encourage playing around with the processor)</description><name>ExecuteScriptHelloWorldGroovy</name><snippet><connections><id>09e8aa98-c9b8-43f0-afbd-513f1d417061</id><parentGroupId>67fb1165-c100-47df-bde3-aa0a4f57af67</parentGroupId><backPressureDataSizeThreshold>0 MB</backPressureDataSizeThreshold><backPressureObjectThreshold>0</backPressureObjectThreshold><destination><groupId>67fb1165-c100-47df-bde3-aa0a4f57af67</groupId><id>8e8aa799-d1ab-4f97-ab35-b06a5a104ae2</id><type>PROCESSOR</type></destination><flowFileExpiration>0 sec</flowFileExpiration><labelIndex>1</labelIndex><name></name><selectedRelationships>success</selectedRelationships><source><groupId>67fb1165-c100-47df-bde3-aa0a4f57af67</groupId><i
@mattyb149
mattyb149 / TellTaleHeartGroovy.xml
Created February 10, 2016 20:29
Apache NiFi template - ExecuteScript (Groovy) Word Count
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><template><description>This template is a self-contained example showing how to use ExecuteScript (with Groovy) to perform a classic &quot;Word Count&quot; of an incoming flow file (namely The Tell-Tale Heart by Edgar Allan Poe). The output file consists of lines, each containing a word in the corpus and its frequency of occurrence.</description><name>TellTaleHeartGroovy</name><snippet><connections><id>de8c38dd-af7b-4c23-8480-4b83a11635c7</id><parentGroupId>faeccde3-60c2-47af-86c7-3492795ef534</parentGroupId><backPressureDataSizeThreshold>0 MB</backPressureDataSizeThreshold><backPressureObjectThreshold>0</backPressureObjectThreshold><destination><groupId>faeccde3-60c2-47af-86c7-3492795ef534</groupId><id>388f347c-7e84-44ef-aaa3-ef42565a7cc6</id><type>PROCESSOR</type></destination><flowFileExpiration>0 sec</flowFileExpiration><labelIndex>1</labelIndex><name></name><selectedRelationships>success</selectedRelationships><source><groupId>faeccde3-60c2-47af-86c7
@mattyb149
mattyb149 / seed_hazelcast.groovy
Created February 11, 2016 00:10
Groovy script to seed Hazelcast instance for ExecuteScript example
@Grab(group='com.hazelcast', module='hazelcast', version='3.6')
@Grab(group='com.hazelcast', module='hazelcast-client', version='3.6')
import com.hazelcast.client.*
import com.hazelcast.client.config.*
import com.hazelcast.core.*
ClientConfig clientConfig = new ClientConfig();
clientConfig.getGroupConfig().setName("dev").setPassword("dev-pass");
clientConfig.getNetworkConfig().addAddress("192.168.99.100", "192.168.99.100:32780");
@mattyb149
mattyb149 / ScriptedHazelcastClient.xml
Created February 11, 2016 00:31
ExecuteScript example for Hazelcast client in JS and Groovy
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><template><description>This template uses Hazelcast JARs in ExecuteScript processors (both in Groovy and Javascript) to generate a flow file from a Hazelcast map.</description><name>ScriptedHazelcastClient</name><snippet><connections><id>840610c3-ce4a-413d-8f54-40ea0a1f1f90</id><parentGroupId>d657baec-19f5-4689-ab5f-f72604055136</parentGroupId><backPressureDataSizeThreshold>0 MB</backPressureDataSizeThreshold><backPressureObjectThreshold>0</backPressureObjectThreshold><destination><groupId>d657baec-19f5-4689-ab5f-f72604055136</groupId><id>91c52bb1-ba4d-4bce-9d41-e49dd4f03851</id><type>PROCESSOR</type></destination><flowFileExpiration>0 sec</flowFileExpiration><labelIndex>1</labelIndex><name></name><selectedRelationships>success</selectedRelationships><source><groupId>d657baec-19f5-4689-ab5f-f72604055136</groupId><id>87256841-565d-4559-bb6b-72fd9d6d014b</id><type>PROCESSOR</type></source><zIndex>0</zIndex></connections><connections><id>f6c2e6aa-f4f7-4a80-9
@mattyb149
mattyb149 / InvokeScriptedProcessor_Groovy_PrintFullName.xml
Created February 11, 2016 02:11
NiFi template to get CSV data from randomuser.me & capitalize first/last name then output full name
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><template><description>This template takes CSV data from randomuser.me and capitalizes the first name and last name from the input and outputs the full name</description><name>InvokeScriptedProcessor_Groovy_PrintFullName</name><snippet><connections><id>3287d7e9-d003-4b4e-8a84-83cba7972923</id><parentGroupId>af7523d7-add4-43b6-ba5d-efe10bfd727f</parentGroupId><backPressureDataSizeThreshold>0 MB</backPressureDataSizeThreshold><backPressureObjectThreshold>0</backPressureObjectThreshold><destination><groupId>af7523d7-add4-43b6-ba5d-efe10bfd727f</groupId><id>a2fc65ea-dea4-4e31-aaae-02adf8cdbce0</id><type>PROCESSOR</type></destination><flowFileExpiration>0 sec</flowFileExpiration><labelIndex>1</labelIndex><name></name><selectedRelationships>success</selectedRelationships><source><groupId>af7523d7-add4-43b6-ba5d-efe10bfd727f</groupId><id>76d2915c-5ae5-484a-baaf-7640b36ba9e5</id><type>PROCESSOR</type></source><zIndex>0</zIndex></connections><connections><id>a7c6a
@mattyb149
mattyb149 / GroovyJsonToJsonExample.xml
Last active January 21, 2019 23:23
An Apache NiFi template to transform a JSON document from one schema to another
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><template><description>This template illustrates how to use the ExecuteScript processor with Groovy to perform JSON-to-JSON transformations</description><name>GroovyJsonToJsonExample</name><snippet><connections><id>09e8aa98-c9b8-43f0-afbd-513f1d417061</id><parentGroupId>67fb1165-c100-47df-bde3-aa0a4f57af67</parentGroupId><backPressureDataSizeThreshold>0 MB</backPressureDataSizeThreshold><backPressureObjectThreshold>0</backPressureObjectThreshold><destination><groupId>67fb1165-c100-47df-bde3-aa0a4f57af67</groupId><id>8e8aa799-d1ab-4f97-ab35-b06a5a104ae2</id><type>PROCESSOR</type></destination><flowFileExpiration>0 sec</flowFileExpiration><labelIndex>1</labelIndex><name></name><selectedRelationships>success</selectedRelationships><source><groupId>67fb1165-c100-47df-bde3-aa0a4f57af67</groupId><id>b52a6649-9570-4db5-8541-d5c0302007c0</id><type>PROCESSOR</type></source><zIndex>0</zIndex></connections><connections><id>b032a9c3-c1d7-4e62-9646-25fe680a0957</id><p