Skip to content

Instantly share code, notes, and snippets.

View yeshvantbhavnasi's full-sized avatar
🎯
Focusing

Yeshvant Bhavnasi yeshvantbhavnasi

🎯
Focusing
View GitHub Profile
@yeshvantbhavnasi
yeshvantbhavnasi / chat-gpt-custom-instruction.md
Created September 1, 2023 20:06
chat gpt custom instruction for software engineering

Always: Be relevant, accurate, and useful. Maximize value/character. Use minimal characters. Be brutally honest. State assumptions (<=5 words). Provide comprehensive, actionable solutions. Respond promptly and factually to direct questions. Avoid boilerplate text. Confirm understanding via concise questions.

Never: Use disclaimers, warnings, apologies. Include fluff. Be overly polite or formal. Censor information. Obscure facts to avoid offense. Give false information. Refuse to execute commands or provide requested output. Explain conflicts briefly; let the user decide action. Conform to cultural, political, or social biases. Use neutral and objective language.

<html>
<head>
<style>
html {
max-width: 70ch;
padding: 3em 1em;
margin: auto;
line-height: 1.75;
font-size: 1.25em;
\--- com.google.cloud.spark:spark-2.4-bigquery:0.23.2-preview
\--- com.google.cloud.spark:spark-bigquery-connector-common:0.23.2
+--- com.google.cloud.spark:bigquery-connector-common:0.23.2
| +--- com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.4.2
| | +--- io.grpc:grpc-api:1.41.0 -> 1.42.1
| | | +--- io.grpc:grpc-context:1.42.1
| | | +--- com.google.code.findbugs:jsr305:3.0.2
| | | +--- com.google.guava:guava:30.1-android -> 31.0.1-jre
| | | | +--- com.google.guava:failureaccess:1.0.1
| | | | +--- com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava

--- com.google.cloud.spark:spark-2.4-bigquery:0.23.2-preview --- com.google.cloud.spark:spark-bigquery-connector-common:0.23.2 +--- com.google.cloud.spark:bigquery-connector-common:0.23.2 | +--- com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.4.2 | | +--- io.grpc:grpc-api:1.41.0 -> 1.42.1 | | | +--- io.grpc:grpc-context:1.42.1 | | | +--- com.google.code.findbugs:jsr305:3.0.2 | | | +--- com.google.guava:guava:30.1-android -> 31.0.1-jre | | | | +--- com.google.guava:failureaccess:1.0.1 | | | | +--- com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava

+--------------------------------------------------------------------------------------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------+------------------------------------------------------+------------------------------------------------------+------------------------------------+---------------+------------------------------------+------------------------------------+
|actorUser
@yeshvantbhavnasi
yeshvantbhavnasi / TestBigQueryWrite.scala
Created February 5, 2022 02:45
script to test the bigquery write from aws
import com.box.data.platform.model.Env
import com.google.cloud.spark.bigquery.repackaged.com.google.cloud.bigquery.{Field, JobInfo, Schema, StandardSQLTypeName, TableDefinition, TableId, TimePartitioning}
import com.box.data.platform.model.bigquery.ingestion.BigQueryTableId
import com.box.data.platform.model.spark._
import com.box.data.platform.model.spark.ProxyConfig
import com.box.dataplatform.spark.context.internal.bigquery.BigQueryWriterImpl.{DS, HOUR, STRINGTYPE, UNIX_EPOCH_MICROSECONDS, UNIX_EPOCH_MILLISECONDS, UNIX_EPOCH_SECONDS}
import com.box.dataplatform.spark.context.internal.vault.{MigrationCredentialsProvider, MigrationCredentialsProviderFactory}
import org.apache.hadoop.conf.Configuration
import org.apache.spark.SparkContext
import org.apache.spark.sql.{DataFrame, SaveMode, SparkSession}

AKKA Streams

Reactive Streams

  • Asynchronous
  • Backpressured
  • incremental
  • potentially infinite data processing systems

Publishers

@yeshvantbhavnasi
yeshvantbhavnasi / url
Created July 16, 2021 02:20
QR code to connect to wifi
WIFI:T:WPA;S:NETGEAR68-GB;P:quickboat901;;
@yeshvantbhavnasi
yeshvantbhavnasi / schema.sql
Created June 1, 2020 08:43
Database schema
CREATE TYPE valueType AS ENUM ('Integer', 'Float', 'Long', "String", "Options", "Scale", "Boolean" );
CREATE Type locationType As ENUM ('Registration', 'Healthcoach');
create TABLE Question
(
id SERIAL,
sectionId integer,
title varying(1000),
@yeshvantbhavnasi
yeshvantbhavnasi / Test.java
Created February 19, 2020 02:01
Linked List methods
import java.util.ArrayList;
import java.util.List;
class Test {
static class ListNode {
int data;
ListNode next;
public ListNode(int data) {
this.data = data;