This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<head> | |
<script type="text/javascript" src="https://code.jquery.com/jquery-3.2.1.min.js"></script> | |
<script type="text/javascript" src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script> | |
<script type="text/javascript" src="script.js"></script> | |
<link rel="stylesheet" type="text/css" href="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/css/bootstrap.min.css"> | |
<link rel="stylesheet" type="text/css" href="https://netdna.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.css"> | |
<link rel="stylesheet" type="text/css" href="style.css"> | |
</head> | |
<body> | |
<div class="container"> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
_time | sender | receiver | message | |
---|---|---|---|---|
2017-05-12 01:00 | Mario | World 1 | Start world 1 | |
2017-05-12 01:30 | Mario | World 1-1 | Move to World 1-1 | |
2017-05-12 02:00 | World 1-1 | Mario | Received Mushroom | |
2017-05-12 02:30 | Mario | World 1-2 | Move to World 1-2 | |
2017-05-12 03:00 | World 1-2 | Mario | Received Fire flower | |
2017-05-12 03:30 | Mario | World 1-3 | Move to World 1-3 | |
2017-05-12 04:00 | World 1-3 | Mario | Received Warp Whistle | |
2017-05-12 04:30 | Mario | World 1-4 | Move to World 1-4 | |
2017-05-12 05:00 | World 1-4 | Mario | Received One-up |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[ | |
{ | |
"Date": "2017-04-28", | |
"DRK": "0.01136728", | |
"BTC": 0, | |
"LTC": 0 | |
}, | |
{ | |
"Date": "2017-04-29", | |
"DRK": "0.01008568", |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[ | |
{ | |
"Time": "2017-05-03 23:17", | |
"BTC": 19.7, | |
"DOGE": 0, | |
"ETH": 78.88, | |
"LTC": 20.81, | |
"REP": 0 | |
}, | |
{ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[{"name":"3M Canada Company","Community Involvement":10,"Employee Engagement & Performance":7,"Financial Benefits & Compensation":9,"Health & Family-Friendly Benefits":9,"Physical Workplace":10,"Training & Skills Development":9,"Vacation & Personal Time-Off":6,"Work Atmosphere & Communications":10,"Total Score":70},{"name":"Aboriginal Peoples Television Network Inc. \/ APTN","Community Involvement":9,"Employee Engagement & Performance":6,"Financial Benefits & Compensation":7,"Health & Family-Friendly Benefits":9,"Physical Workplace":7,"Training & Skills Development":9,"Vacation & Personal Time-Off":9,"Work Atmosphere & Communications":9,"Total Score":65},{"name":"Accenture Inc.","Community Involvement":10,"Employee Engagement & Performance":9,"Financial Benefits & Compensation":7,"Health & Family-Friendly Benefits":9,"Physical Workplace":7,"Training & Skills Development":7,"Vacation & Personal Time-Off":6,"Work Atmosphere & Communications":9,"Total Score":64},{"name":"Agrium Inc.","Community Involvement":10," |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" main.py """ | |
from argparse import ArgumentParser | |
from lxml import etree as ET | |
import getpass | |
import glob | |
import os | |
import re | |
import requests | |
from requests.auth import HTTPBasicAuth | |
from requests.packages.urllib3.exceptions import InsecureRequestWarning |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
for ((i=1998;i<=2019;i++)); | |
do | |
screen -dmS $i bash -c 'source activate my-env; python retrieval.py '$i'; exec bash' | |
done |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
version: '3.2' | |
services: | |
scraper: | |
build: | |
context: . | |
dockerfile: ./Dockerfile | |
command: scrapy crawl insolvencydirect | |
environment: | |
- HUB_PORT_4444_TCP_ADDR=hub | |
- HUB_PORT_4444_TCP_PORT=4444 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import pandas as pd | |
from azure.cosmosdb.table.tableservice import TableService | |
CONNECTION_STRING = "DUMMYSTRING" | |
SOURCE_TABLE = "DUMMYTABLE" | |
def set_table_service(): | |
""" Set the Azure Table Storage service """ | |
return TableService(connection_string=CONNECTION_STRING) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import boto3 | |
from io import StringIO | |
def _write_dataframe_to_csv_on_s3(dataframe, filename): | |
""" Write a dataframe to a CSV on S3 """ | |
print("Writing {} records to {}".format(len(dataframe), filename)) | |
# Create buffer | |
csv_buffer = StringIO() | |
# Write dataframe to buffer | |
dataframe.to_csv(csv_buffer, sep="|", index=False) |
OlderNewer