Skip to content

Instantly share code, notes, and snippets.

View j3works's full-sized avatar

Jeffrey Jonathan Jennings j3works

View GitHub Profile
@j3works
j3works / GetAvroSchema.java
Last active February 11, 2021 12:12
Get Avro Value/Key Schemas from the Confluent Schema Registry and write them to a file.
package io.acertus.marketplace;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.FileInputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
@j3works
j3works / KafkaProducerExample.java
Last active February 11, 2021 13:22
Confluent Cloud Kafka Avro Example in Java
package io.vinlocity.examples;
import io.confluent.kafka.serializers.KafkaAvroSerializer;
import org.apache.kafka.clients.producer.Callback;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.clients.producer.RecordMetadata;
import org.apache.kafka.common.errors.SerializationException;
import org.slf4j.Logger;
@j3works
j3works / Formatted MM_dd_YYYY HH_mm_ss date to ....json
Last active May 3, 2020 03:09
Couple examples of converting a date string into a different format and into UNIX time (epoch)
{
"pipelineConfig" : {
"schemaVersion" : 6,
"version" : 16,
"pipelineId" : "FormattedMMddYYYYHHmmssdateto677507fb-d4d8-44bf-b01f-98350efbf5de",
"title" : "Formatted MM_dd_YYYY HH_mm_ss date to ...",
"description" : "",
"uuid" : "ddf1c8ab-07dd-4318-bf7c-30fa2070ccc4",
"configuration" : [ {
"name" : "executionMode",
@j3works
j3works / lam_1st_interview_round.ipynb
Last active June 20, 2019 13:40
LAM 1st Interview Round Middle Piece
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@j3works
j3works / lam_pyspark_q121.ipynb
Last active June 15, 2019 10:11
LAM Q121 PySpark and Pandas Solutions
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@j3works
j3works / edgar.py
Last active October 13, 2021 16:52
Using Python to process Text and XML data (i.e., Security Listing, Company Info, S-1 and 10-K filing) from the U.S. SEC EDGAR database
"""
This module is responsibile for communicating with the
U.S. SEC EDGAR database
"""
import datetime
import time
import http.client
from io import BytesIO
import requests
@j3works
j3works / KillUserSessions.java
Created May 17, 2019 22:07
Killing Stale User Sessions Automatically in SAP BI
/*****************************************************************************************************
* Author : Jeffrey Jonathan Jennings *
* Module : KillUserSessions.java *
* Purpose: The purpose of this program object's class is to delete all non-excluded active user *
* sessions that is equal or passed its designated kill age. *
*****************************************************************************************************/
import com.crystaldecisions.sdk.exception.SDKException;
import com.crystaldecisions.sdk.framework.IEnterpriseSession;
import com.crystaldecisions.sdk.plugin.desktop.program.IProgramBase;
@j3works
j3works / SetWebiDocUniverse.java
Created May 17, 2019 22:03
Using a SAP BI Program Object that utilizes the RESTful Web services SDK to change a Webi doc’s universe data provider from one universe to another
/*****************************************************************************************************
* *
* Author : Jeffrey Jonathan Jennings *
* Module : SetWebiDocUniverse.java *
* *
*****************************************************************************************************/
import com.crystaldecisions.sdk.exception.SDKException;
import com.crystaldecisions.sdk.framework.IEnterpriseSession;
import com.crystaldecisions.sdk.occa.infostore.*;
@j3works
j3works / aws-s3-uploader.go
Created May 17, 2019 21:26
Using GO to upload a file to AWS S3
package main
import "github.com/aws/aws-sdk-go/aws"
import "github.com/aws/aws-sdk-go/aws/session"
import "github.com/aws/aws-sdk-go/service/s3/s3manager"
import "os"
import "fmt"
func main() {
@j3works
j3works / Dockerfile
Last active June 14, 2019 18:14
Dockerize Spark
FROM openjdk:8-alpine
RUN apk --update add wget tar bash python
RUN wget http://apache.mirror.anlx.net/spark/spark-2.4.3/spark-2.4.3-bin-hadoop2.7.tgz
RUN tar -xzf spark-2.4.3-bin-hadoop2.7.tgz && mv spark-2.4.3-bin-hadoop2.7 spark && rm spark-2.4.3-bin-hadoop2.7.tgz
RUN printf "#!/bin/sh\n/spark/bin/spark-class org.apache.spark.deploy.master.Master --host \$SPARK_MASTER_HOST --port \$SPARK_MASTER_PORT --webui-port \$SPARK_MASTER_WEBUI_PORT" > /start-master.sh
RUN chmod +x /start-master.sh
RUN printf "#!/bin/sh\n/spark/bin/spark-class org.apache.spark.deploy.worker.Worker --webui-port \$SPARK_WORKER_WEBUI_PORT \$SPARK_MASTER_URL" > /start-worker.sh
RUN chmod +x /start-worker.sh