Skip to content

Instantly share code, notes, and snippets.

Avatar

Mehmet Ali "Mali" Akmanalp makmanalp

View GitHub Profile
@makmanalp
makmanalp / gist:ddffd79bdbd75fbff5126c69eb07c1bb
Created Mar 11, 2019
ads-0-backup-1552296000-l2zs8 backu
View gist:ddffd79bdbd75fbff5126c69eb07c1bb
ads-0-backup-1552296000-l2zs8 backup INFO 2019/03/11 19:22:01 tablet prod_iad-1360915300 still has decreasing replication lag of 208.710618394 seconds, will continue waiting
ads-0-backup-1552296000-l2zs8 backup INFO 2019/03/11 19:24:01 tablet prod_iad-1360915300 has caught up on replication
ads-0-backup-1552296000-l2zs8 backup INFO 2019/03/11 19:24:01 (prod_iad-1360915300) checking health
ads-0-backup-1552296000-l2zs8 backup INFO 2019/03/11 19:24:06 (prod_iad-1360915300) succeeded 1 of 3 healthchecks
ads-0-backup-1552296000-l2zs8 backup INFO 2019/03/11 19:24:11 (prod_iad-1360915300) succeeded 2 of 3 healthchecks
ads-0-backup-1552296000-l2zs8 backup INFO 2019/03/11 19:24:16 (prod_iad-1360915300) succeeded 3 of 3 healthchecks
ads-0-backup-1552296000-l2zs8 backup INFO 2019/03/11 19:24:16 getting replication status for replicas
ads-0-backup-1552296000-l2zs8 backup INFO 2019/03/11 19:24:16 getting replication status for master
ads-0-backup-1552296000-l2zs8 backup INFO 2019/03/11 19:24:16 comparing GTIDSets for err
@makmanalp
makmanalp / README.md
Last active Feb 6, 2019
Orchestrator OOM investigation
View README.md

Summary:

When we run a rolling restart on our orchestrator statefulset, the node that is the previous master will get stuck in a crash loop.

Findings so far:

@makmanalp
makmanalp / validate.py
Last active Nov 16, 2018
SQLAlchemy validator event example
View validate.py
from sqlalchemy import Column, Integer, String, DateTime, Boolean
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import event
import datetime
Base = declarative_base()
def validate_int(instance, value, oldvalue, initiator):
# Assigning a string to an Integer column will try to coerce it to the
@makmanalp
makmanalp / network.py
Created Sep 12, 2018
Read / write d3 style network JSON files with pandas, preserving order and types
View network.py
import pandas as pd
import json
def read_network(file_name, nodes_field="nodes", edges_field="edges"):
network = None
with open(file_name, "r") as f:
network = json.loads(f.read())
nodes = network[nodes_field]
edges = network[edges_field]
other_fields = {x:network[x] for x in network.keys()
@makmanalp
makmanalp / get_circleci_artifact.py
Last active Jan 1, 2019
Quick and dirty ansible module for fetching CircleCI build artifacts (latest on a branch, or by build num & git SHA)
View get_circleci_artifact.py
#!/usr/bin/env python
import requests
from ansible.module_utils.basic import AnsibleModule
import traceback
try:
from urllib.parse import quote
except ImportError:
@makmanalp
makmanalp / data_store.py
Created Jul 30, 2018
Simple filesystem organization wrapper
View data_store.py
"""
Simple filesystem organization scheme. You have:
- Objects: A logical "thing", e.g. a document or a page, with unique IDs
- Keys: A type of data that we're storing about the object, like the
location of margins on a page, or the locations of each text box.
- Files: For a specific object under a specific key, you can have multiple
files, e.g. image files for each column in the page
Generally you might want to store data in a specific object's key:
@makmanalp
makmanalp / stata_dask.py
Last active Jul 30, 2020
Read STATA .dta files chunk by chunk (streaming) into dask with pandas's read_stata / StataReader and some hackery
View stata_dask.py
import dask.dataframe as dd
from dask.dataframe.utils import make_meta
from dask.delayed import delayed
import pandas as pd
from itertools import chain
def get_stata_dask_meta(file_name, meta_chunksize=10000, *args, **kwargs):
"""Load up first bit of the file for type metadata info. We have to resort
@makmanalp
makmanalp / .block
Last active Feb 23, 2018
Multivariate radar charts with different axes
View .block
license: mit
scrolling: yes
@makmanalp
makmanalp / selfjoin.py
Created Sep 11, 2017
Readable double self join / tree traversal in SQLAlchemy
View selfjoin.py
FourDigit = aliased(HSProduct)
TwoDigit = aliased(HSProduct)
Section = aliased(HSProduct)
product_data = db.session\
.query(
FourDigit.id.label("product_id"),
FourDigit.code.label("product_code"),
FourDigit.name_en.label("product_name"),
Section.id.label("section_id"),
@makmanalp
makmanalp / eb.md
Created Apr 6, 2017
Elastic beanstalk python gotchas
View eb.md

Config not being run / used

Don't forget to actually commit the config or run eb deploy --staged. Also eb deploy -v gives you a detailed view of which config files get pushed.

Where are my application logs?

/var/log/httpd/error.log (unless you put them elsewhere, this is where stdout goes)

I set WSGIPath right and it's still not working, I get 500 errors

You need to name the variable with the WSGI application application. Naming it app as is flask convention won't work.