Skip to content

Instantly share code, notes, and snippets.

View dheerajinampudi's full-sized avatar

Dheeraj Inampudi dheerajinampudi

View GitHub Profile
@dheerajinampudi
dheerajinampudi / search_types.py
Created February 5, 2024 01:44
search examples
# Full-Text Search Example
import re
text = "Exploring the universe of AI and ML."
search_term = "universe"
result = re.findall(search_term, text)
print(result)
# Semantic Search Example (using spaCy)
import spacy
nlp = spacy.load("en_core_web_sm")
@dheerajinampudi
dheerajinampudi / get_analyze_comments.py
Created October 28, 2023 05:34
Get google data using youtube API and analyze using sklearn
# %%
import pandas as pd
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
from collections import defaultdict
from sklearn.feature_extraction.text import CountVectorizer
import configparser
# %%
config = configparser.ConfigParser()
@dheerajinampudi
dheerajinampudi / conda_env_commands.txt
Last active September 12, 2023 03:55
conda commands to create, list, activate, update and delete environments
# conda environment commands
# CREATE
conda create --name {env_name}
# LIST
conda list environments
# ACTIVATE
conda activate {env_name}
@dheerajinampudi
dheerajinampudi / clean_code.md
Created May 5, 2023 03:01 — forked from wojteklu/clean_code.md
Summary of 'Clean code' by Robert C. Martin

Code is clean if it can be understood easily – by everyone on the team. Clean code can be read and enhanced by a developer other than its original author. With understandability comes readability, changeability, extensibility and maintainability.


General rules

  1. Follow standard conventions.
  2. Keep it simple stupid. Simpler is always better. Reduce complexity as much as possible.
  3. Boy scout rule. Leave the campground cleaner than you found it.
  4. Always find root cause. Always look for the root cause of a problem.

Design rules

@dheerajinampudi
dheerajinampudi / cloudwatch_get_s3_bucket_size.py
Created April 26, 2023 07:55
get last 1 month's size of the s3 bucket
def get_bucket_size_from_cloudwatch(bucket_name, region_name):
cloudwatch = boto3.client('cloudwatch', region_name=region_name)
response = cloudwatch.get_metric_statistics(
Namespace='AWS/S3',
MetricName='BucketSizeBytes',
Dimensions=[{'Name': 'BucketName', 'Value': bucket_name}, {'Name': 'StorageType', 'Value': 'StandardStorage'}],
StartTime=datetime.utcnow() - timedelta(days=30),
EndTime=datetime.utcnow(),
Period=86400,
Statistics=['Average']
@dheerajinampudi
dheerajinampudi / .py
Created February 17, 2023 03:21
Parsing DynamoDB's to JSON Format
from boto3.dynamodb.types import TypeDeserializer, TypeSerializer
def unmarshall(dynamo_obj: dict) -> dict:
"""Convert a DynamoDB dict into a standard dict."""
deserializer = TypeDeserializer()
return {k: deserializer.deserialize(v) for k, v in dynamo_obj.items()}
def marshall(python_obj: dict) -> dict:
@dheerajinampudi
dheerajinampudi / patriQL_CRUD.py
Last active May 10, 2023 09:39
dynamodb_patriQL_CRUD_statements
#%%
import boto3
client = boto3.client('dynamodb')
table_name = ''
#%%
# INSERT INTO -- CREATE A RECORD WITH Multiple Values
#example args
primary_key = ''
secondary_key = ''
@dheerajinampudi
dheerajinampudi / s3_upload_build_folder.py
Created January 3, 2022 01:32
upload all files and folders of build folder to s3 for static website hosting
import boto3
import os
key_id = ''
secret_key = ''
region = ''
bucket_name = ''
def upload_files(path,bucket_name):
session = boto3.Session(
aws_access_key_id=key_id,
aws_secret_access_key=secret_key,
@dheerajinampudi
dheerajinampudi / endpoints-cf-config-diff.csv
Last active May 1, 2021 03:07
cloudfront distribution over difference endpoint configurations
We can make this file beautiful and searchable if this error is corrected: Unclosed quoted field in line 8.
S3 Website endpoint,Diff,S3 REST API endpoint and Example
Step 1: Register a custom domain with Route 53,✅,e.g. example.com
Step 2: Create two buckets,✅,"e.g. First Bucket: example.com
Second Bucket: www.example.com"
Step 3: Configure your root domain bucket for website hosting,❌,"Not required. Instead provide rest api endpoint as Origin Domain Name
Refer Cloudfront Origin Settings"
Step 4: Configure your subdomain bucket for website redirect,✅,www.example.com → website hosting → redirection → https://example.com
Step 5: Configure logging for website traffic,✅,Refer Cloudfront Origin Settings
Step 6: Upload index and website content,✅,Upload website content.
"Step 7: Upload an error document
@dheerajinampudi
dheerajinampudi / s3-endpoint-diff.csv
Last active May 1, 2021 02:22
difference between s3 website endpoint and REST API endpoint
Website Endpoint Rest API
Bucket is publicly available Only Accessible via cloudfront endpoint
Must be a public bucket Need not be a public bucket
Less secured due to s3 global read access More secured because of OAI configuration
Users can access your files through CloudFront and S3 bucket directly users can only access your files through CloudFront not directly from the S3 bucket
Makes Auditing difficult as buckets need public access option to be ON at all times Meet compliance by disabling public access to all buckets by default