Skip to content

Instantly share code, notes, and snippets.

Matthew Burgess mattyb149

Block or report user

Report or block mattyb149

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@mattyb149
mattyb149 / SequenceLookupExample.xml
Created Aug 22, 2018
A NiFi template with a ScriptedLookupService that retrieves a sequence number from an external database
View SequenceLookupExample.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description>This template gives an example of using a ScriptedLookupService to provide sequence numbers from a database.</description>
<groupId>5d7e55f8-0165-1000-1170-d42741815ddb</groupId>
<name>SequenceLookupExample</name>
<snippet>
<connections>
<id>0cf6aae2-8e5a-361f-0000-000000000000</id>
<parentGroupId>cedd92a0-903e-3f68-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
@mattyb149
mattyb149 / SplitRecord_w_Conversion.xml
Created Jul 11, 2018
NiFi SplitRecord example that converts CSV to Avro while splitting files
View SplitRecord_w_Conversion.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description></description>
<groupId>89f8c6b8-0164-1000-d937-82c55b6fdffd</groupId>
<name>SplitRecord_w_Conversion</name>
<snippet>
<controllerServices>
<id>350009ab-3f44-32bf-0000-000000000000</id>
<parentGroupId>fa0118f1-ff03-3030-0000-000000000000</parentGroupId>
<bundle>
@mattyb149
mattyb149 / ISP_template.py
Created Nov 7, 2017
An InvokeScriptedProcessor template (in Jython) for running ExecuteScript Jython scripts faster
View ISP_template.py
#////////////////////////////////////////////////////////////
#// imports go here
#////////////////////////////////////////////////////////////
from org.apache.nifi.processor import Processor,Relationship
from java.lang import Throwable
class E():
def __init__(self):
pass
def executeScript(self,session, context, log, REL_SUCCESS, REL_FAILURE):
@mattyb149
mattyb149 / testEL.groovy
Created Oct 25, 2017
Groovy script for testing NiFi Expression Language expressions
View testEL.groovy
@Grab(group='org.apache.nifi', module='nifi-expression-language', version='1.4.0')
import org.apache.nifi.attribute.expression.language.*
def cli = new CliBuilder(usage:'groovy testEL.groovy [options] [expressions]',
header:'Options:')
cli.help('print this message')
cli.D(args:2, valueSeparator:'=', argName:'attribute=value',
'set value for given attribute')
def options = cli.parse(args)
if(!options.arguments()) {
@mattyb149
mattyb149 / csv_to_avroschema.groovy
Created May 19, 2017
A Groovy script for NiFi ExecuteScript to extract the schema from the header line of a CSV file
View csv_to_avroschema.groovy
import groovy.json.*
def flowFile = session.get()
if(!flowFile) return
def delim = ','
try {
delim = delimiter?.value ?: ','
} catch (MissingPropertyException mpe) { }
@mattyb149
mattyb149 / ScriptedReaderTest.xml
Created Apr 25, 2017
A NiFi 1.2.0 template with sample data and script to illustrate the use of ScriptedReader
View ScriptedReaderTest.xml
<?xml version="1.0" ?>
<template encoding-version="1.1">
<description></description>
<groupId>a5a9dae4-015b-1000-7a1c-eaf2db85b188</groupId>
<name>ScriptedReaderTest</name>
<snippet>
<controllerServices>
<id>bb8e2ece-1712-3f89-0000-000000000000</id>
<parentGroupId>f3e4227c-c306-3c57-0000-000000000000</parentGroupId>
<bundle>
@mattyb149
mattyb149 / ConvertRecord_examples.xml
Created Apr 18, 2017
NiFi template with various tests / exercises of the ConvertRecord processor
View ConvertRecord_examples.xml
<?xml version="1.0" ?>
<template encoding-version="1.1">
<description></description>
<groupId>7ebaf566-015b-1000-b416-922a420ea741</groupId>
<name>ConvertRecord_examples</name>
<snippet>
<connections>
<id>81dd9a51-07c4-39f0-0000-000000000000</id>
<parentGroupId>868f7eec-2b0e-352b-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
@mattyb149
mattyb149 / CDC_MySQL_Replication.xml
Created Apr 17, 2017
NiFi template with end-to-end table replication from a source MySQL database to a target MySQL database
View CDC_MySQL_Replication.xml
<?xml version="1.0" ?>
<template encoding-version="1.1">
<description>This template shows an end-to-end example of how to replicate table(s)
from a source MySQL database to a target MySQL database</description>
<groupId>68961054-015b-1000-2c21-01b42e84dbe0</groupId>
<name>CDC_MySQL_Replication</name>
<snippet>
<connections>
<id>9a0f0bca-5c0c-3f1e-0000-000000000000</id>
<parentGroupId>23d80cd0-6e6e-31c9-0000-000000000000</parentGroupId>
@mattyb149
mattyb149 / EnforceOrder_Examples.xml
Created Apr 7, 2017
A NiFi Template with 2 example use cases for the EnforceOrder processor
View EnforceOrder_Examples.xml
<?xml version="1.0" ?>
<template encoding-version="1.1">
<description>This template contains two use cases for the EnforceOrder processor:
1) Random Delay: FlowFiles are routed to two separate paths and delayed by a random amount of time (0-100 millis)
2) MultipleSplits: FlowFiles are split twice with SplitText, the attributes must be updated to keep the original order
</description>
<groupId>48313cf9-015b-1000-cf31-bc9709ad2638</groupId>
<name>EnforceOrder_Examples</name>
<snippet>
@mattyb149
mattyb149 / CDC_Replication.xml
Created Mar 23, 2017
A test script for the GetChangeDataCaptureMySQL NiFi processor, taking CDC (binlog) events and transforming them to target SQL
View CDC_Replication.xml
<?xml version="1.0" ?>
<template encoding-version="1.0">
<description></description>
<groupId>faf788c5-015a-1000-f344-de24ceb9d7e7</groupId>
<name>CDC_Replication</name>
<snippet>
<connections>
<id>d21bc8ee-015a-1000-0000-000000000000</id>
<parentGroupId>faf788c5-015a-1000-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
You can’t perform that action at this time.