Skip to content

Instantly share code, notes, and snippets.

View yaravind's full-sized avatar
💭
Constraints Liberate. Liberties Constrain.

Aravind Yarram yaravind

💭
Constraints Liberate. Liberties Constrain.
View GitHub Profile
@staltz
staltz / introrx.md
Last active May 3, 2024 13:00
The introduction to Reactive Programming you've been missing
@Chaser324
Chaser324 / GitHub-Forking.md
Last active May 2, 2024 05:49
GitHub Standard Fork & Pull Request Workflow

Whether you're trying to give back to the open source community or collaborating on your own projects, knowing how to properly fork and generate pull requests is essential. Unfortunately, it's quite easy to make mistakes or not know what you should do when you're initially learning the process. I know that I certainly had considerable initial trouble with it, and I found a lot of the information on GitHub and around the internet to be rather piecemeal and incomplete - part of the process described here, another there, common hangups in a different place, and so on.

In an attempt to coallate this information for myself and others, this short tutorial is what I've found to be fairly standard procedure for creating a fork, doing your work, issuing a pull request, and merging that pull request back into the original project.

Creating a Fork

Just head over to the GitHub page and click the "Fork" button. It's just that simple. Once you've done that, you can use your favorite git client to clone your repo or j

@seddonm1
seddonm1 / xsd-schema.scala
Last active April 5, 2024 18:56
Makes a Spark Schema (StructType) from an input XSD file
// need to add the Apache WS XMLSchema library to spark/jars (does not have dependencies)
// https://repo1.maven.org/maven2/org/apache/ws/xmlschema/xmlschema-core/2.2.5/xmlschema-core-2.2.5.jar
import org.apache.ws.commons.schema.XmlSchemaCollection
import java.io.StringReader
import scala.collection.JavaConverters._
import org.apache.ws.commons.schema._
import org.apache.ws.commons.schema.constants.Constants
import org.apache.spark.sql.types._
@wsargent
wsargent / win10-dev.md
Last active March 21, 2024 04:27
Windows Development Environment for Scala

Windows 10 Development Environment for Scala

This is a guide for Scala and Java development on Windows, using Windows Subsystem for Linux, although a bunch of it is applicable to a VirtualBox / Vagrant / Docker subsystem environment. This is not complete, but is intended to be as step by step as possible.

Harden Windows 10

Read the entire Decent Security guide, and follow the instructions, especially:

@ashrithr
ashrithr / kafka.md
Last active March 14, 2024 21:16
kafka introduction

Introduction to Kafka

Kafka acts as a kind of write-ahead log (WAL) that records messages to a persistent store (disk) and allows subscribers to read and apply these changes to their own stores in a system appropriate time-frame.

Terminology:

  • Producers send messages to brokers
  • Consumers read messages from brokers
  • Messages are sent to a topic
@MarkRoddy
MarkRoddy / parse_s3_access_logs.sql
Last active February 28, 2024 18:42
DuckDB: Query S3 Access Logs
/*
Usage: you'll want to search for the strings <bucket> and <prefix>, and insert the S3 bucket where your access
logs are being delivered. Use (or delete) <prefix> to filter to a subset of your logs.
*/
/*
These commented out configuration settings you can either run yourself in the REPL and source this file using
`.read parse_s3_access_logs.sql`, or you can uncomment them and supply values for yourself.
@rosswd
rosswd / multi-git-win.md
Last active February 28, 2024 09:46
Setting up a Github and Bitbucket account on the same computer on Mac OS. Now with a guide for Windows 10.

Setting up github and bitbucket on the same computer (Windows)

Guide for Windows

mix3d asked for some help using this guide with windows so here we go. This was tested with Windows 10. Run all commands in Git Bash once it's installed.

Github will be the main account and bitbucket the secondary.

Git for Windows

  • Download and install Git for Windows
    • In the installer, select everything but decide if you want a desktop icon (2nd step)
@jdegoes
jdegoes / fpmax.scala
Created July 13, 2018 03:18
FP to the Max — Code Examples
package fpmax
import scala.util.Try
import scala.io.StdIn.readLine
object App0 {
def main: Unit = {
println("What is your name?")
val name = readLine()
@amalgjose
amalgjose / download_adls_directory.py
Last active December 4, 2023 16:15
Python program to download a complete directory or file from Microsoft Azure ADLS. This program is capable of recursively download a complete directory from Azure Data Lake Storage. This uses Azure Blob Storage API to iterate over the directories, files and download the data. This is tested as of 05-October-2020. For more details, refer to https…
# coding: utf-8
import os
from azure.storage.blob import BlobServiceClient
class DownloadADLS:
def __init__(self, connection_string, container_name):
service_client = BlobServiceClient.from_connection_string(connection_string)
self.client = service_client.get_container_client(container_name)
@chinshr
chinshr / Jenkinsfile
Last active October 16, 2023 09:25
Best of Jenkinsfile, a collection of useful workflow scripts ready to be copied into your Jenkinsfile on a per use basis.
#!groovy
# Best of Jenkinsfile
# `Jenkinsfile` is a groovy script DSL for defining CI/CD workflows for Jenkins
node {
}