rmoff

1. Download Instant Client:

• instantclient-basic-macos.x64-11.2.0.4.0.zip
• instantclient-sdk-macos.x64-11.2.0.4.0.zip
• instantclient-sqlplus-macos.x64-11.2.0.4.0.zip

2. Unzip and move to /opt

3. Create symlink

rmoff

# RNM 20140124
#
# https://twitter.com/rmoff/
# --------------------
# Required
from suds.client import Client
client = Client('http://obiee-server:9704/analytics-ws/saw.dll/wsdl/v7')
#------------
# Logon
sessionid = client.service['SAWSessionService'].logon('weblogic','Password01')

rmoff

Live Coding a KSQL Application

Introduction

rmoff

This is a set of instructions for use with the blog article Streaming data from Oracle using Oracle GoldenGate and Kafka Connect.

@rmoff / September 15, 2016

---

Preparing the Database for Oracle GoldenGate

First we prepare the database for Oracle GoldenGate. Connect as SYSDBA:

rmoff

$ ./bin/kafka-console-consumer \
  --bootstrap-server localhost:9092 \
  --from-beginning \
  --property print.key=true \
  --topic connect-offsets
["jdbc_source_mysql_foobar_01",{"table":"foobar"}]      {"timestamp_nanos":0,"timestamp":1501796305000}
["jdbc_source_mysql_smt_foobar_02",{"table":"wibble"}]	{"timestamp_nanos":0,"timestamp":1501800342000}
["jdbc_source_mysql_smt_foobar_02",{"table":"wibble"}]	{"timestamp_nanos":0,"timestamp":1501800707000}

rmoff

From Mayank Patel on http://cnfl.io/slack

Problem:

org.apache.kafka.connect.errors.ConnectException: Tolerance exceeded in error handler
	at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:178)
	at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execute(RetryWithToleranceOperator.java:104)
	at org.apache.kafka.connect.runtime.WorkerSourceTask.convertTransformedRecord(WorkerSourceTask.java:284)
	at org.apache.kafka.connect.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:309)

rmoff

kafkacat -b localhost:9092 -P -t testdata-json4 <<EOF
{ "schema": { "type": "struct", "fields": [ { "type": "map", "keys": { "type": "string", "optional": false }, "values": { "type": "string", "optional": false }, "optional": false, "field": "tags" }, { "field": "sn", "optional": false, "type": "string" }, { "field": "value", "optional": false, "type": "float" } ], "optional": false, "version": 1 }, "payload": { "tags": { "tagnum": "5" }, "sn": "FOO", "value": 500.0 } }
EOF

curl -i -X PUT -H "Accept:application/json" \
        -H  "Content-Type:application/json" http://localhost:8083/connectors/SINK_INFLUX_01/config \
        -d '{
            "connector.class"               : "io.confluent.influxdb.InfluxDBSinkConnector",
            "value.converter": "org.apache.kafka.connect.json.JsonConverter",
            "value.converter.schemas.enable": "true",

rmoff

Asciidoc code formatting

Example 01

-- SQL Code block
SELECT SOME_FIELD FROM FOO;

rmoff

kafkacat -b localhost:9092 \
         -t _kafka-connect-group-01-status \
         -C \
         -o-3 \
         -c3 \
         -f 'Topic %t / Partition %p / Offset: %o / Timestamp: %T\nHeaders: %h\nKey (%K bytes): %k\nPayload (%S bytes): %s\n--\n'

rmoff

When you use knit_expand it appears that the inclusion of the Rmd is done on the first pass, and then the complete document evaluated. This means that a Rmd block referenced in loop with knit_expand will only evaluate changing variables at their last value.

This can be worked around by passing the literal value of the variable at the time of the knit_expand with {{var}} syntax.

This is documented in the knitr_expand docs, but less clear (to an R noob like me) for embedded documents rather than strings.