Skip to content

Instantly share code, notes, and snippets.

@pcolazurdo
pcolazurdo / opensearch_index_management.sh
Created October 19, 2023 15:52
OpenSearch Serverless - some tips
# pip install awscurl
export COLLECTION_ID=j04odjdwa8f5xxxxxxxx
export OPENSEARCHHOST=`aws opensearchserverless batch-get-collection --ids ${COLLECTION_ID} | jq '.collectionDetails[] | .dashboardEndpoint'`
# Delete all indexes that follow a specific pattern
delete_old_indexes() {
# TARGETDATE should look like YYYY.MM.DD where date is 1 month before now.
export TARGETDATE=`date -d "-1 month" +"%Y.%m.%d"`
export INDEXLIST=$(awscurl --service aoss "${OPENSEARCHHOST}/_cat/indices" | grep ocsf | grep ${TARGETDATE} | awk '{print $1}')
echo "${INDEXLIST}" | while read index; do awscurl --service aoss -X DELETE "${OPENSEARCHHOST}/${index}"; done
@pcolazurdo
pcolazurdo / readme.md
Created October 17, 2023 11:34
VPC Flow Logs - Network Flow visualization for Security Lake

This is a customized snippet using Vega.

The original idea is from https://github.com/aws-solutions/centralized-logging-with-opensearch, but this is customised to consume OCSF logs injected into Security Lake

Some tips:

  • To debug Vega scripts, you can use VEGA_DEBUG.view.data('rawData') into your browser console to retrieve the data in rawData (look at the beginning of the file above)
  • Not sure how to programatically inject this code, but if you need to create this in your own dashboard, you can add a new visualization as Vega, and copy and paste the code above.
@pcolazurdo
pcolazurdo / .local browsing.md
Last active July 30, 2023 18:15
Smart home with Raspberry

how to fix browsing the internal network

MacOS issues with .local

In MacOS, you can do dig whatever.local and get some results if you have the entry in a local DNS (like pi-hole) but curl, or browsing will fail. This is because Apple enforces that .local domain is only discovered by the mDNS Bonjour service (more info)

To solve this, I decided to run the avahi-daemon in my local Raspberry-pi to publish additional services.

I decided to use the avahi-aliases project to simplify publishing more than one service on the same IP as the default avahi-daemon doesn't allow this at the moment

@pcolazurdo
pcolazurdo / play_with_dates.py
Created October 25, 2022 16:09
Get specific dates using Python
from datetime import datetime, timedelta
# Every day of the week (starting from tomorrow) for the past 50 weeks
now = datetime.now() + timedelta(days=1)
for i in range (50):
delta = timedelta(days=7*i)
print ('"{}"'.format((now-delta).strftime("%b %-d, %Y")))
# Every first Monday of the year
@pcolazurdo
pcolazurdo / golang-panic-analysis.sh
Created April 10, 2022 09:52
How to analyse GOLANG Panic results
# Assumes you are capturing the output of your golang app panic into /tmp/crash
# Reason of the panic
head -3 /tmp/crash
# Register status (22 may change in different architectures)
tail -22 /tmp/crash
# Number of goroutines
cat /tmp/crash | grep goroutine | wc -l
@pcolazurdo
pcolazurdo / GOLANG-modules.md
Last active April 4, 2022 17:12
Interesting GOLANG resources

Userful Libraries/modules

  1. OpenAPI toolkit common string formats: github.com/go-openapi/strfmt
  2. Seamless printing to the terminal (stdout) and logging to a io.Writer (file) that’s as easy to use as fmt.Println: https://github.com/spf13/jwalterweatherman
  3. Go package for dealing with maps, slices, JSON and other data: https://github.com/stretchr/objx
  4. Efficient JSON beautifier and compactor for Go: https://github.com/tidwall/pretty
  5. bbolt is a fork of Ben Johnson's Bolt key/value store: https://pkg.go.dev/go.etcd.io/bbolt
  6. Formatters for units to human friendly sizes: github.com/dustin/go-humanize
  7. Package httpsnoop provides an easy way to capture http related metrics (i.e. response time, bytes written, and http status code) from your application's http.Handlers: https://github.com/felixge/httpsnoop
  8. Staticcheck - The advanced Go linter: https://github.com/dominikh/go-tools
@pcolazurdo
pcolazurdo / clean_up_unused_policies.sh
Created March 11, 2022 15:50
#CAUTION: This will help you to delete all AWS IAM Policies that aren't attached to any resource Role. Treat carefully
#!/bin/bash
TEMP_DIR=$(mktemp -d)
echo Output Directory: ${TEMP_DIR}
confirm() {
#
# syntax: confirm [<prompt>]
#
# Prompts the user to enter Yes or No and returns 0/1.
@pcolazurdo
pcolazurdo / observe.js
Created February 7, 2022 18:00
Browser: Observe events when an element is visible in the screen
var observer = new IntersectionObserver(function (entries) {
// isIntersecting is true when element and viewport are overlapping
// isIntersecting is false when element and viewport don't overlap
if (entries[0].isIntersecting === true)
console.log(entries[0].target.id);
}, { threshold: [0] });
// Observe all H2 headings and get their id printed into the console whenever they are in focus
document.querySelectorAll("h2").forEach(function (item) {
observer.observe(document.querySelector("#" + item.id));
@pcolazurdo
pcolazurdo / list_all_resources.sh
Created October 5, 2021 21:31
CloudControl for the win
# very basic but you can see the gist of it:
aws cloudformation list-types --type RESOURCE --visibility PUBLIC | \
jq '.TypeSummaries[].TypeName' | \
cut -d\" -f2 | \
while read a
do
aws cloudcontrol list-resources --type-name $a 2>/dev/null| \
jq '.ResourceDescriptions[].Identifier'
done