Skip to content

Instantly share code, notes, and snippets.

Avatar
💭
Awesome

Anjaiah Methuku anjijava16

💭
Awesome
View GitHub Profile
View csv_to_csv.py
import csv
from datetime import datetime
INPUT_FILE_PATH = 'C:/Tech_Learn_welcome/Python_Utils/FastAPI/sai_workspace/fast_api_welcome/files/input.csv'
OUTPUT_FILE_PATH = 'C:/data/python_output/input.csv'
HEADER_SKIP = True
OUTPUT_HEADER = 'id,first,last,ssn,address,firstname_lastname,process_date'
def read_file():
View hive-site.xml
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
View python_environment_s.sh
# Virtual Environmnet
1. There are other great third-party tools for creating virtual environments, such as
1. virtualenv
2. conda
3. Pipenv
4. Poetry
# Step1
View mac_shortcuts.txt
Jump to beginning of a line – Command+Left Arrow
Jump to end of a line – Command+Right Arrow
Jump to beginning of current word – Option+Right Arrow
Jump to end of current word – Option+Right Arrow
To backspace on a Mac, press the fn and Delete keys, as shown below
command +Space ==> QUick search
Control + LeftArrow ==> Left Open one
Control+ RightArrow ==> Right Opne one
command + c==> COPY
command + p ==> Past
View zprofile.sh
vi .zprofile
# Setting PATH for Python 3.8
# The original version is saved in .zprofile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.8/bin:${PATH}"
# Hadoop
export HADOOP_HOME=/Users/welcome/Desktop/hadoop/hadoop-3.2.1/
export HADOOP_INSTALL=$HADOOP_HOME
View mac_setup_softwares.sh
# Hadoop setup here
1. https://blog.contactsunny.com/data-science/installing-hadoop-on-the-new-m1-pro-and-m1-max-macbook-pro
# Hive Setup Here
1. https://dbmstutorials.com/hive/hive-setup-on-mac.html
View .zprofile
welcome@Anjaiahs-MacBook-Pro ~ % more .zprofile
# Setting PATH for Python 3.8
# The original version is saved in .zprofile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.8/bin:${PATH}"
export PATH
welcome@Anjaiahs-MacBook-Pro ~ %
View Python_environment.py
mkdir delab_f
cd delab_f
python3 -m venv delab-venv
source delab-venv/bin/activate
pip install jupyterlab
View cosmosdb_azure.scala
# Read data
dfCustomer = spark.read\
.format("cosmos.oltp")\
.option("spark.synapse.linkedService", "AdventureWorksSQL")\
.option("spark.cosmos.container", "Customer")\
.load()
display(dfCustomer.limit(10))
# Write data
View Spark_Cosmosdb_cassandra.scala
# Spark Streaming with Cassandra (Azure Cosmos DB)
1. https://github.com/mspnp/azure-databricks-streaming-analytics