Skip to content

Instantly share code, notes, and snippets.

View anderson-marques's full-sized avatar
🤖
Hacking as usual...

Anderson Carvalho anderson-marques

🤖
Hacking as usual...
View GitHub Profile

Keybase proof

I hereby claim:

  • I am anderson-marques on github.
  • I am mccanderson (https://keybase.io/mccanderson) on keybase.
  • I have a public key ASB1xgOAbqJl320g1Mf5iQQsEuPVj5M5xs80133Cb8gAOQo

To claim this, I am signing this object:

O nosso data flow é:
(S3) -> (Lambda/Python) -> Glue (Transferencia e Metadados) -> (S3) -> Glue -> Redshift (DW) -> QuickSight (Visualização)
-> Athena (Queries nos dados crus) ou QuickSight (Visualização)
- AWS Glue para o processo de extração dos dados "crus". Estou exporando o DBT.
- A partir de dados armazendos no S3 (DB, arquivos) - voce faz a primeira filtragem e gera um novo Dataset-
--------------
- https://www.databricks.com/product/data-intelligence-platform
- https://www.databricks.com/product/data-intelligence-platform
@anderson-marques
anderson-marques / playbook.md
Created November 3, 2023 11:50
AWS SSM using Maintenance Windows to Start and Stop EC2 Instances

How to set up an EC2 schedule using AWS Systems Manager (SSM) Maintenance Windows

Prerequisites:

  1. Make sure you have AWS CLI installed and configured.
  2. Ensure you have an IAM Role with the necessary permissions for SSM and EC2 start/stop operations.

Step-by-Step Playbook:

  1. Create the Maintenance Window for Stopping the EC2 instance:
https://chat.openai.com/share/c6913b30-41ab-4e81-9b21-066d96842907
@anderson-marques
anderson-marques / gitlab_auth.js
Created May 18, 2022 12:46
OpenID Connect Login to Gitlab using JavaScript
const http = require('http');
const GITLAB_OAUTH_AUTHORIZATION_URL = 'https://gitlab.com/oauth/authorize'
const GITLAB_OAUTH_CLIENT_ID = 'change-me'
const SERVER_PORT = 4321
let config = {
'base_url': GITLAB_OAUTH_AUTHORIZATION_URL,
'client_id': GITLAB_OAUTH_CLIENT_ID,
'redirect_uri': `http://localhost:${SERVER_PORT}/cb`,
@anderson-marques
anderson-marques / iptables-ddns.sh
Created February 9, 2022 11:14 — forked from eusonlito/iptables-ddns.sh
Update iptables firewall with dynamic DNS updates from cron job
#!/bin/bash
# Set as cronjob
# * * * * * /root/scripts/iptables-ddns.sh >> /root/logs/iptables-ddns.log 2>&1
log () {
echo "[$(date "+%F +%T")] [$1] $2" >> "$LOGS/changes.log"
}
HOSTS="mydynamichost.ddns.net"
@anderson-marques
anderson-marques / gist:0a56ad8ad4872f0eda45d9686964794d
Created October 19, 2021 12:00
Podman run with read-only volume from the host
podman run --rm -it --env-file credentials.env -v /mnt$(pwd)/my-folder:/${WORKDIR}//my-folder/:Z ${PROJECT_NAME}:base sh
import { SendWelcomeMessageInteractor } from "../marketing/control/send-welcome-message";
import { CustomerEventNames, CreateCustomerEvent } from "../customer/entity/customer-event";
import { S3PutEvent, S3PutEventEntry } from "../s3/entity/put-event-entry";
import { DataLakeGateway } from "../datalake/boundary/datalake-gateway";
const dataLakeGateway = DataLakeGateway.newInstance();
const sendWelcomeMessageInteractor = SendWelcomeMessageInteractor.newInstance();
const extractEventIdAndEventName = (
s3PutEventEntry: S3PutEventEntry
@anderson-marques
anderson-marques / template.yaml
Created May 25, 2021 23:10
SAM Template using S3 Event Source - Hypothetical CustomerEventsProcessor
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: >
SAM Template - Example of S3 Event processing
Process related events: CustomerCreated, CustomerDeleted, CustomerUpdated
Parameter:
DataLakeBucket:
Type: String
Default: com-mycompany-datalake-bucket
@anderson-marques
anderson-marques / s3-put-event.ts
Created May 25, 2021 22:59
S3 Put Event - TypeScript
export type S3PutEventEntry = {
eventVersion: string;
eventsource: string;
awsRegion: string;
eventTime: string;
eventName: string;
userIdentity: Record<string, unknown>;
requestParameters: Record<string, unknown>;
responseElements: Record<string, unknown>;
s3: {