Skip to content

Instantly share code, notes, and snippets.

View DustinAlandzes's full-sized avatar

Dustin Alandzes DustinAlandzes

82 followers · 66 following
View GitHub Profile
@DustinAlandzes
DustinAlandzes / gist:936d5c2770eaeef79b6ac96c0c2a170f
Created June 17, 2019 03:42
Use a Lua script to get an HTML response with cookies, headers, body and method set to correct values; lua_source argument value is cached on Splash server and is not sent with each request (it requires Splash 2.1+):
import scrapy
from scrapy_splash import SplashRequest
script = """
function main(splash)
splash:init_cookies(splash.args.cookies)
assert(splash:go{
splash.args.url,
headers=splash.args.headers,
http_method=splash.args.http_method,
@DustinAlandzes
DustinAlandzes / gist:26e0b6c3e7cdd5f0caf8a9dba75a7c80
Created June 14, 2019 23:45
[2019-06-14 23:44:26,825] ERROR [Process: MainProcess Id:55826] [Thread: MainThread Id: 140736302633920] [scrapy.core.scraper._itemproc_finished:234] Error processing {'case_json': {'aws_cache_link': 'https://s3.console.aws.amazon.com/s3/buckets/scrapy-appeal-spiders/staging/mn-appeals/parse_item_page/A19-0915/2019-06-14-23:43.html',
'case_number': 'A19-0915',
'case_type': 'Civil',
'external_source_link': 'http://macsnc.courts.state.mn.us/ctrack/view/publicCaseMaintenance.do?csNameID=94074&csInstanceID=106682',
'internal_sub_project': 'mn-appeals',
'start_date': '06/13/2019',
'status': 'Pre-Briefing',
'title': 'TMT Management Group, LLC, et al., Appellants, vs. '
'U.S. Bank National Association, et al., Respondents, '
'Wilbur Tate, Respondent, United Credit Recovery, LLC, '
@DustinAlandzes
DustinAlandzes / gist:a10a05232e3207b8eb67a8af1b5b9112
Created June 14, 2019 23:44
[2019-06-14 23:43:56,035] ERROR [Process: MainProcess Id:55826] [Thread: MainThread Id: 140736302633920] [scrapy.core.scraper._itemproc_finished:234] Error processing {'case_json': {'aws_cache_link': 'https://s3.console.aws.amazon.com/s3/buckets/scrapy-appeal-spiders/staging/mn-appeals/parse_item_page/A19-0914/2019-06-14-23:43.html',
'case_number': 'A19-0914',
'case_type': 'Civil',
'external_source_link': 'http://macsnc.courts.state.mn.us/ctrack/view/publicCaseMaintenance.do?csNameID=94073&csInstanceID=106681',
'internal_sub_project': 'mn-appeals',
'start_date': '06/13/2019',
'status': 'Pre-Briefing',
'title': 'In re the Matter of the Welfare of the Children of: '
'R. L. W. and J. P. W., Parents.'},
'parties_list': [{'attorneys': [],
@DustinAlandzes
DustinAlandzes / test.csv
Created May 31, 2019 19:55
We can make this file beautiful and searchable if this error is corrected: Unclosed quoted field in line 6.
CaseCategory,CaseTypeDescription,CaseNumber,Title,FilingDate,PartyType,FirstName,MiddleName,LastName/CompanyName,PartyAddress,Attorney
"CV","Mortgage Foreclosure - Commercial -2- $50,001 - $249,999","19-CA-005467","Amirhouses.com, LLC vs Lewis, Henry","05/28/2019","Plaintiff","","","Amirhouses.com, LLC","1627 US Hwy 92 W, Auburndale, FL 33823","No Attorney"
"CV","Mortgage Foreclosure - Commercial -2- $50,001 - $249,999","19-CA-005467","Amirhouses.com, LLC vs Lewis, Henry","05/28/2019","Defendant","Henry","","Lewis","2620 E 33rd Ave, Tampa, FL 33610","No Attorney"
"CV","Mortgage Foreclosure - Commercial -2- $50,001 - $249,999","19-CA-005467","Amirhouses.com, LLC vs Lewis, Henry","05/28/2019","Defendant","Lyette","","Pate","9604 N Aster Ave, Tampa, FL 33612","No Attorney"
"CV","Mortgage Foreclosure - Commercial -2- $50,001 - $249,999","19-CA-005467","Amirhouses.com, LLC vs Lewis, Henry","05/28/2019","Defendant","Cornelius","","Pate","9604 N Aster Ave, Tampa, FL 33612","No Attorney"
"CV","Mortgage Foreclosure -
@DustinAlandzes
DustinAlandzes / hurst.py
Last active April 30, 2019 04:30
#http://www.long-memory.com/hurst/hurst.cpp
#The calculation requires at least 4 data points.
# https://en.wikipedia.org/wiki/Rescaled_range
data = []
array1 = []
array2 = []
for n in range(len(data)):
totalR = 0
totalS = 0
@DustinAlandzes
DustinAlandzes / orderedList.py
Created March 16, 2019 01:30
class Node:
def __init__(self,initdata):
self.data = initdata
self.next = None
def getData(self):
return self.data
def getNext(self):
return self.next
@DustinAlandzes
DustinAlandzes / unorderedList.py
Created March 16, 2019 01:29
class Node:
def __init__(self,initdata):
self.data = initdata
self.next = None
def getData(self):
return self.data
def getNext(self):
return self.next
@DustinAlandzes
DustinAlandzes / orderedSequentialSearch.py
Created March 6, 2019 03:27
(unorderd and orderd) linear/sequential search implemented with python lists: http://interactivepython.org/runestone/static/pythonds/SortSearch/TheSequentialSearch.html
def orderedSequentialSearch(alist, item):
'''
O(n/2) in item is not present average case
'''
pos = 0
found = False
stop = False
while pos < len(alist) and not found and not stop:
if alist[pos] == item:
found = True
@DustinAlandzes
DustinAlandzes / queue.py
Created March 6, 2019 02:53
a Queue class using Python lists: http://interactivepython.org/runestone/static/pythonds/BasicDS/ImplementingaQueueinPython.html
class Queue:
def __init__(self):
self.items = []
def isEmpty(self):
return self.items == []
def enqueue(self, item):
self.items.insert(0,item)
@DustinAlandzes
DustinAlandzes / stack.py
Created March 6, 2019 02:45
a Stack class using Python lists: http://interactivepython.org/runestone/static/pythonds/BasicDS/ImplementingaStackinPython.html
class Stack:
def __init__(self):
self.items = []
def isEmpty(self):
return self.items == []
def push(self, item):
self.items.append(item)