Skip to content

Instantly share code, notes, and snippets.

View jerowe's full-sized avatar

Jillian Rowe jerowe

View GitHub Profile
@jerowe
jerowe / airflow-docker-compose.yml
Last active June 15, 2023 06:29
Airflow Docker Compose Configuration - Includes airflow scheduler, airflow worker, airflow webserver, rabbitmq, and postgresql
version: '3'
# Run as
# docker-compose build; docker-compose up -d
# Check with
# docker ps
# Then check the logs with
# docker logs --tail 50 $container_id
# docker-compose images
# docker-compose logs --tail 20 repo_name
@jerowe
jerowe / spark-example-sort.py
Created March 2, 2019 06:54
Sort a file with spark
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
@jerowe
jerowe / docker-compose-swarm.yml
Last active February 12, 2019 16:18
Deploy a Celery Queue with Docker Swarm
version: '3'
# Deploy the stack
# docker stack deploy -f docker-compose-swarm.yml celery
# Investigate the service with
# docker service ls
# docker service logs celery_rabbit
# Scale the service with
# docker service scale celery_job_queue_flask_app=N
# docker service rm celery_rabbit celery_job_queue_flask_app celery_job_queue_celery_worker job_queue_celery_flower
version: '3'
# Run as
# docker-compose build; docker-compose up -d
# Check with
# docker ps
# Then check the logs with
# docker logs --tail 50 $container_id
# docker-compose logs --tail 20 tf_counts
@jerowe
jerowe / deploy-celery-minimal.py
Created February 9, 2019 08:33
A minimal celery example.
import time
from celery import Celery
import os
from flask import Flask
app = Flask(__name__)
def make_celery(app):
celery = Celery(
app.import_name,
@jerowe
jerowe / docker-compose.yml
Created February 6, 2019 13:19
Deploy a full stack flask and angular app with traefik
version: '3'
# Run as
# docker-compose build; docker-compose up -d
# Check with
# docker ps
# Then check the logs with
# docker logs --tail 50 $container_id
# docker-compose logs --tail 20 tf_counts
@jerowe
jerowe / Dockerrun.aws.json
Created January 26, 2019 13:41
deploy-python-flask-docker-series-Dockerrun.aws.json
{
"AWSEBDockerrunVersion": 2,
"volumes": [
],
"containerDefinitions": [
{
"name": "flask-app-server",
"image": "quay.io/jerowe/flask-app-server:latest",
"update": true,
"essential": true,
version: '3'
services:
flask-app-server:
build:
context: flask_app
dockerfile: Dockerfile
ports:
- "5000:5000"
volumes:
FROM continuumio/miniconda3:4.5.11
RUN apt-get update -y; apt-get upgrade -y
# You don't technically need these, but I get kind of twitchy if I don't have vim installed
RUN apt-get install -y vim-tiny vim-athena ssh
# Add a user and switch, using the default root user is BAD
RUN adduser --home /home/flask flask
#!/usr/bin/env bash
set -x -e
gunicorn --workers=2 --bind=0.0.0.0:5000 --keep-alive=2000 --timeout=2000 --log-level=debug flask_app:app