Skip to content

Instantly share code, notes, and snippets.

Avatar

Pierre Villard pvillard31

View GitHub Profile
View FTP-UTF8-Test.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.3">
<description></description>
<groupId>2e083aef-0162-1000-80bc-8c6bbfc00ca8</groupId>
<name>FTP-UTF8-Test</name>
<snippet>
<processGroups>
<id>3982c999-cd5f-3ac7-0000-000000000000</id>
<parentGroupId>16e377d4-8369-3464-0000-000000000000</parentGroupId>
<position>
View TestQueryRecord.java
@Test
public void testNoMatchingRecords() throws InitializationException {
TestRunner runner = getRunner();
final JsonTreeReader jsonReader = new JsonTreeReader();
runner.addControllerService("reader", jsonReader);
runner.setProperty(jsonReader, SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY, SchemaInferenceUtil.INFER_SCHEMA);
final JsonRecordSetWriter jsonWriter = new JsonRecordSetWriter();
runner.addControllerService("writer", jsonWriter);
View terraform-deploy
pvillard@cloudshell:~/nifi-gcp-terraform/gcp-cluster-secured-nifi-oidc (nifi-dev-project)$ /bin/sh deploy.sh nifi-dev-project nifi_bin
Updated property [core/project].
Initializing the backend...
Initializing provider plugins...
The following providers do not have any version constraints in configuration,
so the latest version was installed.
View deploy-nifi-cluster-terraform
git clone https://github.com/pvillard31/nifi-gcp-terraform.git
cd nifi-gcp-terraform/gcp-cluster-secured-nifi-oidc
/bin/sh deploy.sh <gcp-project-id> <gcs-bucket>
View deploy-nifi-terraform.sh
git clone https://github.com/pvillard31/nifi-gcp-terraform.git
cd nifi-gcp-terraform/gcp-single-secured-nifi-oidc/
terraform init
terraform apply
View nifi-gcp-quickstart
gcloud beta compute instances create-with-container my-nifi-instance --tags=nifi --container-image=apache/nifi
gcloud compute firewall-rules create allow-nifi-unsecured --action=ALLOW --rules=tcp:8080 --target-tags=nifi
View startup.sh
#! /bin/bash
# This script is used when starting a docker image based GCE instance
# of the NiFi Registry. It is intended to configure the NiFi Registry
# so that the persistence provider is the Google Cloud Source Repo.
# Docker volumes (directory type)
# /tmp/config => /home/nifi/.ssh - Read only
# /tmp/ssh/id_rsa => /id_rsa - Read only
# /tmp/nifi-flow-repository => /nifi-flow-repository - Read/Write
@pvillard31
pvillard31 / keypairs
Last active Jul 2, 2019
Generate SSH key pairs
View keypairs
$ ssh-keygen -t rsa -b 4096 -m PEM -C "NiFi Registry"
Generating public/private rsa key pair.
Enter file in which to save the key (~/.ssh/id_rsa): ~/temp/id_rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in ~/temp/id_rsa.
Your public key has been saved in ~/temp/id_rsa.pub.
The key fingerprint is:
SHA256:/hu6FZLvcvDigP4ixPFPEGzfpY1RHOoTRytjqYOoYHo NiFi Registry
View execute.sh
$ time curl --data-binary "@test.csv" -H "Content-Type: text/csv" -X POST https://minifi-csvjson-t546z2l6aq-uc.a.run.app
[ {
"name" : "Pierre Villard",
"company" : "Google"
} ]
real 0m0.725s
user 0m0.036s
sys 0m0.026s
View publish.sh
git clone git@github.com:pvillard31/nifi-cloudrun-example.git
cd nifi-cloudrun-example
gcloud auth configure-docker
# docker build -t [HOSTNAME]/[PROJECT-ID]/[IMAGE]:[TAG] .
docker build -t eu.gcr.io/my-gcp-project/minifi-csvjson:0.0.1 .
# docker push [HOSTNAME]/[PROJECT-ID]/[IMAGE]:[TAG] .
docker push eu.gcr.io/my-gcp-project/minifi-csvjson:0.0.1