Skip to content

Instantly share code, notes, and snippets.

View emmanuelnk's full-sized avatar

Emmanuel emmanuelnk

26 followers · 24 following
View GitHub Profile
@emmanuelnk
emmanuelnk / db.py
Last active February 14, 2024 07:57
Python SQLAlchemy Basic Model, Session, DB Connection Classes
from sqlalchemy import event
import os
import logging
import sqlalchemy
import boto3
import base64
import json
from botocore.exceptions import ClientError
logger = logging.getLogger()
@emmanuelnk
emmanuelnk / displayAwsAccountAlias.js
Last active January 19, 2024 18:49
GreaseMonkey script to show an AWS account alias in the top nav bar
// ==UserScript==
// @name DisplayAWSAccountAlias
// @version 1
// @grant none
// @match https://*.console.aws.amazon.com/*
// ==/UserScript==
// Replace with your account numbers and desired alias
// The key is the account number and the value is the alias to display
const accountAliases = {
@emmanuelnk
emmanuelnk / cleanEmpty.js
Created May 13, 2020 08:07
Clean js object by recursive removal of undefined, null, NaN and empty strings
// A SO answer to https://stackoverflow.com/questions/286141/remove-blank-attributes-from-an-object-in-javascript
// based off the recursive cleanEmpty function by @chickens.
// This one can also handle Date objects correctly
// and has a defaults list for values you want stripped.
const cleanEmpty = function(obj, defaults = [undefined, null, NaN, '']) {
if (defaults.includes(obj)) return
if (Array.isArray(obj))
return obj
@emmanuelnk
emmanuelnk / useful_lambda_log_insights_queries.md
Created December 7, 2023 19:25
AWS Cloudwatch Log Insights Useful Lambda Queries

Queries

General Performance

This query runs against Lambda function logs to get the following information:

  • InvocationCount -- This is the number of times the Lambda function was invoked.
  • ColdStartCount -- This is the number of times the Lambda function had initialization overhead (cold start).
  • InitDuration (min, avg, max) -- This is the amount of time it took the Lambda function to initialize (cold start time).
  • Duration (min, avg, max) -- This is the amount of time it took the Lambda function to run after initialization.
  • AllocatedMemory -- This is the amount of memory allocated to the Lambda function in MB.
  • MaxMemoryUsed (min, avg, max) -- This is the max amount of memory that the Lambda function used during its invocation in MB.
@emmanuelnk
emmanuelnk / openpr.md
Created November 1, 2023 18:03
Open Github PR from Terminal (SSH Authorization)

  1. Install Dependencies

    brew install jq
brew install hub

  2. Log into hub (from any repo just do hub pr list)

    • use your github username for username
    • use a personal access token for password (token should have repo access)
@emmanuelnk
emmanuelnk / github-actions-json-outputs.md
Created September 20, 2023 21:29
Extract Job or step output JSON object to GITHUB_ENV in Github Actions

If you have a job or step that has many outputs that you would like to set as envs (in the shell), do this:

name:

workflow_dispatch:

jobs:
  test:
 runs-on: ubuntu-latest
@emmanuelnk
emmanuelnk / How to make zsh like fish
Last active June 24, 2023 14:05 — forked from abhigenie92/How to make zsh like fish
1. Install oh-my-zsh
sh -c "$(wget https://raw.githubusercontent.com/robbyrussell/oh-my-zsh/master/tools/install.sh -O -)"
2. Clone necessary plugins.
git clone https://github.com/zsh-users/zsh-autosuggestions $ZSH_CUSTOM/plugins/zsh-autosuggestions
git clone https://github.com/zsh-users/zsh-history-substring-search ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-history-substring-search
git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting
3. Add plugins to ~/.zshrc as
plugins = ( [plugins...] zsh-autosuggestions zsh-history-substring-search zsh-syntax-highlighting)
@emmanuelnk
emmanuelnk / DynamoDBClient.py
Last active June 1, 2023 02:13
Simple client wrapper library for Dynamodb access in Python (boto3)
# based off the excellent gist by Martina Pugliese
# https://gist.github.com/martinapugliese/cae86eb68f5aab59e87332725935fd5f
import boto3
from boto3.dynamodb.conditions import Key, Attr
from boto3.dynamodb.types import DYNAMODB_CONTEXT
dynamodb = boto3.resource("dynamodb")
@emmanuelnk
emmanuelnk / Queries.md
Last active April 26, 2023 16:25
Useful Athena Queries

A collection of useful Athena Queries

Web Application Firewall

Create WAF logs with projection (no need to partition)

  • See the docs for reference

  • Replace the values BUCKET_NAME, AWS_ACCOUNT_NUMBER, AWS_REGION, WEB_ACL_NAME accordingly

  • Replace 'projection.date.range' = '2023/01/01/00/00,NOW', with the date your WAF logging begins (find in S3)

    Query 
    

    • 




  


  




    


          
    

      

        
        

          

              @emmanuelnk
          

          

            
                emmanuelnk
                / mwaa_cloudwatch_get_worker_count.md
            
            

              Created
              April 21, 2023 17:00
            

              
                Managed Worklows for Apache Airflow get worker Count from Cloudwatch Insights
              
          

        

      

        

  

      

    
Cloudwatch does not provide metrics on how many workers you are running in a MWAA Environment.
Fortunately we can get this information from Cloudwatch Insights using the MWAA_ENVIRONMENT_NAME-Worker log group.
Every log stream has the ip address of the worker in the @logStream field


    

  1. Open the relevant MWAA_ENVIRONMENT_NAME-Workerlog group
    2. 

  2. Open in CloudWatch Insights and paste and run the following query for your desired time range.
    3. 

  3. you can then add the visualization to your desired Cloudwatch dashboard
    4. 



fields @timestamp