ijokarumawak

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
    <description></description>
    <groupId>7bf539f4-0167-1000-97f1-3a91b5de75c4</groupId>
    <name>NIFI-5826 Fix backslash escaping</name>
    <snippet>
        <processGroups>
            <id>cf87c2e1-c49b-303b-0000-000000000000</id>
            <parentGroupId>1e08db49-ce5e-30c7-0000-000000000000</parentGroupId>
            <position>

ijokarumawak

import java.io.File;
import java.io.IOException;

import java.nio.file.Path;
import java.nio.file.Files;

import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;

ijokarumawak

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
    <description></description>
    <groupId>39379f66-0167-1000-9951-3cf7c004e310</groupId>
    <name>Merge XML Records</name>
    <snippet>
        <controllerServices>
            <id>36c4d83a-ff47-38e2-0000-000000000000</id>
            <parentGroupId>376efa9a-48fc-3e3d-0000-000000000000</parentGroupId>
            <bundle>

ijokarumawak

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
    <description></description>
    <groupId>29cd7683-0167-1000-0886-c9dc91c022a5</groupId>
    <name>AddTimestamp</name>
    <snippet>
        <connections>
            <id>09785868-682a-3058-0000-000000000000</id>
            <parentGroupId>818236f9-e91f-324b-0000-000000000000</parentGroupId>
            <backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>

ijokarumawak

This NiFi flow template illustrates how incoming FlowFile attributes are carried to the InvokeHTTP output FlowFile.

ijokarumawak

This example flow can be used to process files with following requirements:

• A group of files can only be processed when every files for a specific group is ready

• Each filename has groupId (e.g. 123_456) and a type name (e.g. ab/cd/ef)

• Example set of files for group '123_456'

  • file_123_456_ab.ex1
  • file_123_456_cd.ex1
  • file_123_456_ef.ex1

• file_123_456.ex2

ijokarumawak

How to use Dante SOCKS proxy server

Create sockd.conf file locally on the docker host machine and map it with -v option of docker run.

Run socks server

docker run -d -p 1080:1080 -v /etc/sockd.conf:/etc/sockd.conf --name socks-server vimagick/dante

Add user

ijokarumawak

Execute following commands from hbase shell to setup a test table and visibility labels. HBase needs to be configured to enable visibility label, too.

create 't1', {NAME => 'f1'}
add_labels [ 'admin', 'service', 'developer', 'test' ]
grant 'nifi', 'RW', 't1'

ijokarumawak

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
    <description></description>
    <groupId>d1fdab79-0162-1000-ac9f-bef682631714</groupId>
    <name>NIFI-5090</name>
    <snippet>
        <processGroups>
            <id>096d130c-0d82-3e30-0000-000000000000</id>
            <parentGroupId>8f9ca121-e577-3715-0000-000000000000</parentGroupId>
            <position>

ijokarumawak