Skip to content

Instantly share code, notes, and snippets.

View landonstewart's full-sized avatar

Landon Stewart landonstewart

  • Canada
View GitHub Profile

Keybase proof

I hereby claim:

  • I am landonstewart on github.
  • I am refactored (https://keybase.io/refactored) on keybase.
  • I have a public key ASAE_7bsHbSxukgu1gM4wiN73hVuCucQXIw7nfjfbpHfwwo

To claim this, I am signing this object:

@landonstewart
landonstewart / gist:647801915aed935c1c6ac6a824af86c2
Created June 10, 2021 18:30
Return a unique list of elements sorted by frequency
from collections import Counter
def unique_list(*args):
counted = Counter(args[0])
return [value for value, count in counted.most_common()]
Example:
>>> unique_list([1,2,3,4,5,6,6,6,5,4,4])
[4, 6, 5, 1, 2, 3]
@landonstewart
landonstewart / models.py
Last active June 10, 2021 18:31
Overriding the save() method for a Django model to dynamically obtain the hostname and IP addresses for a URL if none were given
from socket import getaddrinfo
from socket import IPPROTO_TCP # pylint: disable=no-name-in-module
def save(self, *args, **kwargs):
"""Override save."""
if self.url:
if not self.hostname:
self.hostname = urlparse(self.url).hostname
if self.hostname:
if not self.ip_addrs:
#!/usr/bin/env python
"""Convert to spacy binary format."""
import random
import pandas as pd
import spacy
from spacy.training import Example
from spacy.util import minibatch
from tqdm import tqdm
@landonstewart
landonstewart / tasks.py
Created March 4, 2021 21:40
Django dynamic field name timedelta filter using F()
def filter_urls(urls, hours_ago=None, filter_on='created', all_records=False):
"""Generate queryset from URL model.
All records or just the ones from the urls parameter
Restrict the records to those updated before hours_ago or not at all
Args:
urls (list): A list of URLs
Kwargs:
@landonstewart
landonstewart / helper_funcs.py
Last active March 4, 2021 19:51
Various dns/ip/ipwhois related functions
@cache_memoize(1200)
def dns_lookup(hostname):
"""Lookup both A and AAAA.
Args:
hostname (str): Hostname that you'd like to get the ip address(es) for
port (int): Port against which to do the lookup
Returns:
A list of IP addresses
@landonstewart
landonstewart / find_key_in_dict.py
Created May 11, 2018 19:51
Find's the value of a key matching key in dictionary recursively
def find_key_in_dict(key, dictionary):
for k, v in dictionary.items():
if k == key:
yield v
elif isinstance(v, dict):
for result in find_key_in_dict(key, v):
yield result
elif isinstance(v, list):
for d in v:
for result in find_key_in_dict(key, d):
@landonstewart
landonstewart / hosts file to block facebook
Created March 21, 2018 07:12
Block facebook IPv4 and IPv6
# Hosts file snippet
# ...
# ...
# Block Facebook IPv4
127.0.0.1 api.ak.facebook.com
127.0.0.1 api.connect.facebook.com
127.0.0.1 api.facebook.com
127.0.0.1 app.facebook.com
127.0.0.1 apps.facebook.com
127.0.0.1 ar-ar.facebook.com