Skip to content

Instantly share code, notes, and snippets.

View antweiss's full-sized avatar

Ant Weiss antweiss

View GitHub Profile
@antweiss
antweiss / gist:a2a827934761abb5f825b1429e0a1c5c
Last active May 8, 2024 15:11
CloudLinux Jenkins training
git clone https://github.com/otomato-gh/jenkins-playground.git
cd jenkins-playground
./setup.sh
cd non-docker
./setup_jenkins.sh
#######
Go to <your machine public IP>:8080
Follow the instructions to find and set the password and install the plugins
#####
Fork https://github.com/otomato-gh/bunsysinfo to your own github account.
@antweiss
antweiss / gist:938f254c1aaeacfb76e8e6a819f4eae1
Created February 14, 2024 17:35
Parse kubernetes nodes external IPs with yq
kubectl get node -oyaml | yq '.items[].status.addresses[] | select(.type=="ExternalIP") | .address')
import logging
from airflow import DAG
from datetime import datetime, timedelta
from airflow.providers.postgres.hooks.postgres import PostgresHook
from airflow.operators.python import PythonOperator
# Change these to your identifiers, if needed.
POSTGRES_CONN_ID = "postgres_default"
from airflow.operators.python_operator import ShortCircuitOperator
from airflow.operators.dummy_operator import DummyOperator
from airflow.models import DAG
import airflow.utils.helpers
from airflow.operators.bash_operator import BashOperator
import random
args = {
'owner': 'antweiss',
'start_date': airflow.utils.dates.days_ago(0)
@antweiss
antweiss / bash-with-template.py
Last active April 25, 2023 10:51
Aitflow bash operator with templating
from textwrap import dedent
# ...
templated_command = dedent(
"""
{% for i in range(5) %}
echo "{{ ds }}" #a variable (datestamp)
echo "{{ macros.ds_add(ds, 7)}}" #a built-in macro
{% endfor %}
"""
)
@antweiss
antweiss / pythonOperatorSleep.py
Created April 23, 2023 15:32
airflow pythonOperator example
from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.utils.dates import days_ago
import time
with DAG(
'second',
start_date=days_ago(0),
description='Our new little DAG',
tags=['second']
) as dag:
@antweiss
antweiss / zip_operator.py
Created March 1, 2023 10:23
Airflow Zip Operator
class ZipOperator(BaseOperator):
"""
An operator which takes in a path to a file and zips the contents to a location you define.
:param path_to_file_to_zip: Full path to the file you want to Zip
:type path_to_file_to_zip: string
:param path_to_save_zip: Full path to where you want to save the Zip file
:type path_to_save_zip: string
"""
template_fields = ('path_to_file_to_zip', 'path_to_save_zip')
template_ext = []
@antweiss
antweiss / gist:15f645186703ae07d86eb6bdf5bdb149
Created January 21, 2023 15:06
AWS cli - get account id
#assign to an env variable:
export ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text)
@antweiss
antweiss / get image names
Created January 20, 2023 10:30
Get all images in a multi-container pod
kubectl get pod my-pod -ojson | jq ".spec.containers[] | .image"
@antweiss
antweiss / gist:20b99c06e5a08301bbdf854f6479c7a0
Last active December 20, 2022 10:30
Get all available eksctl addons for a cluster
#the verbosity 0 prints out plain json and then we can parse out addon names
eksctl utils describe-addon-versions --cluster my-cluster --region=my-region --verbose 0 | jq -r '.Addons[] | .AddonName'