Skip to content

Instantly share code, notes, and snippets.

View danizen's full-sized avatar

Dan Davis danizen

View GitHub Profile
@danizen
danizen / dump_fields_in_ipython.py
Created August 28, 2017 21:31
Trying to achieve title in generated OpenAPI Schema
from rest_framework import schemas
g = schemas.SchemaGenerator()
s = g.get_schema()
for f in s['adapter'].links['create'].fields:
print(f)
@danizen
danizen / MongoJobStatusStore.java
Last active January 17, 2018 21:45
MongoJobStatusStore and unit tests
package gov.nih.nlm.occs.norconex;
import java.io.IOException;
import java.io.Reader;
import java.io.StringReader;
import java.io.StringWriter;
import java.io.Writer;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
@danizen
danizen / ConfigurationDumper.java
Created January 19, 2018 17:29
ConfigurationDumper example
package gov.nih.nlm.occs.norconex.util;
import static com.norconex.collector.core.AbstractCollectorLauncher.ARG_CONFIG;
import static com.norconex.collector.core.AbstractCollectorLauncher.ARG_VARIABLES;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.io.StringReader;
@danizen
danizen / targets.py
Last active February 7, 2018 21:55
A target that causes luigi to run again after a file reaches a maximum age
class RecentlyModifiedLocalTarget(luigi.LocalTarget):
"""
The local file must have been recently enough modified to "exist", driving luigi's DAG
"""
def __init__(self, *args, **kwargs):
"""
Extract an optional maxage parameter
"""
maxage = kwargs.pop('maxage', None)
@danizen
danizen / a-reference.json
Last active February 12, 2018 16:39
aggregation of crawled references using mongodb
{
"referrerLinkText": null,
"isRootParentReference": false,
"sitemapLastMod": null,
"parentRootReference": null,
"referrerLinkTag": null,
"sitemapChangeFreq": null,
"crawlState": "REJECTED",
"isValid": false,
"contentType": "text/html",
@danizen
danizen / testlinter.py
Last active May 3, 2018 01:06
test python linter for some basics
"""
Test the linter
"""
def real_uninitialized_problem(x):
"""
Real problem of uninitialized variable
"""
return x + y
@danizen
danizen / etl.py
Created May 18, 2018 20:53
petl from Django utility
from django.db.backends.base.base import BaseDatabaseWrapper
from django.db import connections
import cx_Oracle
__all__ = (
'get_etl_cursor',
)
class CursorProxy(object):
@danizen
danizen / excerpt from navbar.html
Last active August 27, 2018 20:19
Django template for authentication
{% if user.is_authenticated %}
{# ... other menu options ... #}
<li class="dropdown">
<a class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspop="true" aria-expanded="false">
<i class="fa fa-user"></i> {{ user.nih_login_id }}
</a>
<ul class="dropdown-menu">
<li role="menuitem">
<a href="{% url 'casauth:profile' %}"><i class="fa fa-user"></i> Profile</a>
</li>
@danizen
danizen / leakcheck.py
Last active November 9, 2018 22:35
A WSGI middleware to check for leaks
import tracemalloc
import pathlib
import random
import csv
import os
from datetime import datetime
import re
import linecache
import logging
import threading
@danizen
danizen / checkwinsafe.py
Created February 13, 2019 03:36
A tiny utility to check whether a python file has strings unsafe for output to the Windows command prompt
#!/usr/bin/env python
import argparse
import ast
import glob
import sys
class CheckStrings(ast.NodeVisitor):
filename = 'unknown'