Skip to content

Instantly share code, notes, and snippets.

@JPercivall
Created August 19, 2016 13:26
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save JPercivall/f337b8abdc9019cab5ff06cb7f6ff09a to your computer and use it in GitHub Desktop.
Save JPercivall/f337b8abdc9019cab5ff06cb7f6ff09a to your computer and use it in GitHub Desktop.
Demo config.yml used for MiNiFi 0.0.1 MeetUp talks
Flow Controller:
name: MiNiFi S2S temp and humidity
comment: ''
Core Properties:
flow controller graceful shutdown period: 10 sec
flow service write delay interval: 500 ms
administrative yield duration: 30 sec
bored yield duration: 10 millis
max concurrent threads: 1
FlowFile Repository:
partitions: 4
checkpoint interval: 2 mins
always sync: false
Swap:
threshold: 200
in period: 5 sec
in threads: 1
out period: 5 sec
out threads: 4
Content Repository:
content claim max appendable size: 100 KB
content claim max flow files: 1
always sync: false
Provenance Repository:
provenance rollover time: 1 min
Component Status Repository:
buffer size: 10
snapshot frequency: 1 min
Security Properties:
keystore: './raspberrypi-cert.pfx'
keystore type: 'PKCS12'
keystore password: 'PUT YOUR PASSWORD HERE'
key password: 'PUT YOUR PASSWORD HERE'
truststore: './all-trusted.jks'
truststore type: 'JKS'
truststore password: 'PUT YOUR PASSWORD HERE'
ssl protocol: 'TLS'
Sensitive Props:
key:
algorithm: PBEWITHMD5AND256BITAES-CBC-OPENSSL
provider: BC
Processors:
- name: Determine where to send
class: org.apache.nifi.processors.standard.RouteOnAttribute
max concurrent tasks: 1
scheduling strategy: TIMER_DRIVEN
scheduling period: 0 sec
penalization period: 30 sec
yield period: 1 sec
run duration nanos: 0
auto-terminated relationships list: []
Properties:
Routing Strategy: Route to Property name
high humidity: ${humidity:ge("90")}
high temp: ${temperature:ge("85")}
- name: Extract Values from Line
class: org.apache.nifi.processors.standard.ExtractText
max concurrent tasks: 1
scheduling strategy: TIMER_DRIVEN
scheduling period: 0 sec
penalization period: 30 sec
yield period: 1 sec
run duration nanos: 0
auto-terminated relationships list: []
Properties:
Character Set: UTF-8
Enable Canonical Equivalence: 'false'
Enable Case-insensitive Matching: 'false'
Enable DOTALL Mode: 'false'
Enable Literal Parsing of the Pattern: 'false'
Enable Multiline Mode: 'false'
Enable Unicode Predefined Character Classes: 'false'
Enable Unicode-aware Case Folding: 'false'
Enable Unix Lines Mode: 'false'
Include Capture Group 0: 'true'
Maximum Buffer Size: 1 MB
Maximum Capture Group Length: '1024'
Permit Whitespace and Comments in Pattern: 'false'
humidity: '"humidity":(\d+)'
temperature: '"temperature":(\d+)'
- name: Invalid reading
class: org.apache.nifi.processors.standard.LogAttribute
max concurrent tasks: 1
scheduling strategy: TIMER_DRIVEN
scheduling period: 0 sec
penalization period: 30 sec
yield period: 1 sec
run duration nanos: 0
auto-terminated relationships list:
- success
Properties:
Attributes to Ignore:
Attributes to Log:
Log Level: warn
Log Payload: 'true'
Log prefix:
- name: Normal
class: org.apache.nifi.processors.standard.LogAttribute
max concurrent tasks: 1
scheduling strategy: TIMER_DRIVEN
scheduling period: 0 sec
penalization period: 30 sec
yield period: 1 sec
run duration nanos: 0
auto-terminated relationships list:
- success
Properties:
Attributes to Ignore:
Attributes to Log:
Log Level: warn
Log Payload: 'true'
Log prefix:
- name: SplitText
class: org.apache.nifi.processors.standard.SplitText
max concurrent tasks: 1
scheduling strategy: TIMER_DRIVEN
scheduling period: 0 sec
penalization period: 30 sec
yield period: 1 sec
run duration nanos: 0
auto-terminated relationships list:
- failure
- original
Properties:
Header Line Count: '0'
Line Split Count: '1'
Remove Trailing Newlines: 'true'
- name: TailFile
class: org.apache.nifi.processors.standard.TailFile
max concurrent tasks: 1
scheduling strategy: TIMER_DRIVEN
scheduling period: 100 ms
penalization period: 30 sec
yield period: 1 sec
run duration nanos: 0
auto-terminated relationships list: []
Properties:
File Location: Local
File to Tail: /home/pi/Documents/logs/temperature.log
Initial Start Position: Beginning of File
Rolling Filename Pattern:
State File:
Connections:
- name: Determine where to send/high humidity/b7024a6d-5928-4738-9048-14b4895fca0b
source name: Determine where to send
source relationship name: high humidity
destination name: b7024a6d-5928-4738-9048-14b4895fca0b
max work queue size: 0
max work queue data size: 0 MB
flowfile expiration: 0 sec
queue prioritizer class: ''
- name: Determine where to send/high temp/64ec6ffa-0b13-4cdf-8795-81c8e85dc490
source name: Determine where to send
source relationship name: high temp
destination name: 64ec6ffa-0b13-4cdf-8795-81c8e85dc490
max work queue size: 0
max work queue data size: 0 MB
flowfile expiration: 0 sec
queue prioritizer class: ''
- name: Determine where to send/unmatched/Normal
source name: Determine where to send
source relationship name: unmatched
destination name: Normal
max work queue size: 0
max work queue data size: 0 MB
flowfile expiration: 0 sec
queue prioritizer class: ''
- name: Extract Values from Line/matched/Determine where to send
source name: Extract Values from Line
source relationship name: matched
destination name: Determine where to send
max work queue size: 0
max work queue data size: 0 MB
flowfile expiration: 0 sec
queue prioritizer class: ''
- name: Extract Values from Line/unmatched/Invalid reading
source name: Extract Values from Line
source relationship name: unmatched
destination name: Invalid reading
max work queue size: 0
max work queue data size: 0 MB
flowfile expiration: 0 sec
queue prioritizer class: ''
- name: SplitText/splits/Extract Values from Line
source name: SplitText
source relationship name: splits
destination name: Extract Values from Line
max work queue size: 0
max work queue data size: 0 MB
flowfile expiration: 0 sec
queue prioritizer class: ''
- name: TailFile/success/SplitText
source name: TailFile
source relationship name: success
destination name: SplitText
max work queue size: 0
max work queue data size: 0 MB
flowfile expiration: 0 sec
queue prioritizer class: ''
Remote Processing Groups:
- name: NiFi Flow
url: https://demo-instance.com:8443/nifi
comment: ''
timeout: 30 sec
yield period: 10 sec
Input Ports:
- id: 64ec6ffa-0b13-4cdf-8795-81c8e85dc490
name: minifi-high-temp
comment: ''
max concurrent tasks: 1
use compression: false
- id: b7024a6d-5928-4738-9048-14b4895fca0b
name: minifi-high-humidity
comment: ''
max concurrent tasks: 1
use compression: false
Provenance Reporting:
comment:
scheduling strategy: TIMER_DRIVEN
scheduling period: 30 sec
destination url: https://demo-instance.com:8443/
port name: minifi-provenance
originating url: http://${hostname(true)}:8081/nifi
use compression: true
timeout: 30 secs
batch size: 1000
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment