Skip to content

Instantly share code, notes, and snippets.

Pierre Villard pvillard31

Block or report user

Report or block pvillard31

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View deploy-nifi-terraform.sh
git clone https://github.com/pvillard31/nifi-gcp-terraform.git
cd nifi-gcp-terraform/gcp-single-secured-nifi-oidc/
terraform init
terraform apply
View nifi-gcp-quickstart
gcloud beta compute instances create-with-container my-nifi-instance --tags=nifi --container-image=apache/nifi
gcloud compute firewall-rules create allow-nifi-unsecured --action=ALLOW --rules=tcp:8080 --target-tags=nifi
View startup.sh
#! /bin/bash
# This script is used when starting a docker image based GCE instance
# of the NiFi Registry. It is intended to configure the NiFi Registry
# so that the persistence provider is the Google Cloud Source Repo.
# Docker volumes (directory type)
# /tmp/config => /home/nifi/.ssh - Read only
# /tmp/ssh/id_rsa => /id_rsa - Read only
# /tmp/nifi-flow-repository => /nifi-flow-repository - Read/Write
@pvillard31
pvillard31 / keypairs
Last active Jul 2, 2019
Generate SSH key pairs
View keypairs
$ ssh-keygen -t rsa -b 4096 -m PEM -C "NiFi Registry"
Generating public/private rsa key pair.
Enter file in which to save the key (~/.ssh/id_rsa): ~/temp/id_rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in ~/temp/id_rsa.
Your public key has been saved in ~/temp/id_rsa.pub.
The key fingerprint is:
SHA256:/hu6FZLvcvDigP4ixPFPEGzfpY1RHOoTRytjqYOoYHo NiFi Registry
View execute.sh
$ time curl --data-binary "@test.csv" -H "Content-Type: text/csv" -X POST https://minifi-csvjson-t546z2l6aq-uc.a.run.app
[ {
"name" : "Pierre Villard",
"company" : "Google"
} ]
real 0m0.725s
user 0m0.036s
sys 0m0.026s
View publish.sh
git clone git@github.com:pvillard31/nifi-cloudrun-example.git
cd nifi-cloudrun-example
gcloud auth configure-docker
# docker build -t [HOSTNAME]/[PROJECT-ID]/[IMAGE]:[TAG] .
docker build -t eu.gcr.io/my-gcp-project/minifi-csvjson:0.0.1 .
# docker push [HOSTNAME]/[PROJECT-ID]/[IMAGE]:[TAG] .
docker push eu.gcr.io/my-gcp-project/minifi-csvjson:0.0.1
View Dockerfile
FROM apache/nifi-minifi:latest
USER root
ENV MINIFI_HOME /opt/minifi/minifi-0.5.0
ADD config.yml $MINIFI_HOME/conf/config.yml
ADD *.nar $MINIFI_HOME/lib/
RUN chown -R minifi:minifi $MINIFI_HOME
View medium-cloudrun-2
# command to convert the template into yml file for MiNiFi
$MINIFI_HOME/bin/config.sh transform templateConvertRecord.xml config.yml
View medium-cloudrun-1
$ cat test.csv
name,company
Pierre Villard,Google
$ curl --data-binary "@test.csv" -H "Content-Type: text/csv" -X POST http://localhost:9090/
[ {
"name" : "Pierre Villard",
"company" : "Google"
} ]
View oozie.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description></description>
<groupId>2e083aef-0162-1000-80bc-8c6bbfc00ca8</groupId>
<name>oozie</name>
<snippet>
<processGroups>
<id>5a720561-d2b1-3f86-0000-000000000000</id>
<parentGroupId>16e377d4-8369-3464-0000-000000000000</parentGroupId>
<position>
You can’t perform that action at this time.