Skip to content

Instantly share code, notes, and snippets.

View ricardocarvalhods's full-sized avatar
👨‍💻
Coding..

Ricardo Carvalho ricardocarvalhods

👨‍💻
Coding..
View GitHub Profile
@sminot
sminot / print_cloudwatch_logs_aws_batch.py
Last active June 15, 2023 16:46
Print CloudWatch logs for an AWS Batch job
#!/usr/bin/env python3
import boto3
import argparse
from datetime import datetime
parser = argparse.ArgumentParser()
parser.add_argument("job_id")
# Add the arguments
@a-hisame
a-hisame / gzip_s3_and_json_py3.py
Created July 3, 2018 10:41
To use gzip file between python application and S3 directly for Python3
#!/usr/bin/python
# -*- coding: utf-8 -*-
'''To use gzip file between python application and S3 directly for Python3.
Python 2 version - https://gist.github.com/a-hisame/f90815f4fae695ad3f16cb48a81ec06e
'''
import io
import gzip
import json
@kstohr
kstohr / fb_group_page_query.py
Last active December 20, 2021 18:33
Facebook Graph API: Search for pages and groups that match a query list
import os
import traceback
import logging
import facebook
import requests
import pandas as pd
import json
import time
import numpy as np
from pandas.io.json import json_normalize
@nkt1546789
nkt1546789 / puwrapper.py
Last active April 12, 2019 20:16
A wrapper class for PU classification on Python (proposed by Elkan and Noto, 2008).
import numpy as np
from numpy import random
from sklearn import base
class PUWrapper(object):
def __init__(self,trad_clf,n_fold=5):
self._trad_clf=trad_clf
self._n_fold=n_fold
def fit(self,X,s):
@benmarwick
benmarwick / various_speed_tests.R
Last active January 10, 2020 19:23
Speed tests of different ways to read into R large numbers of CSV files, specifically read.csv.sql, read.csv (optimised) and fread, also of parallel processing and interactive disk storage options (filehash)
# Speed tests of different ways to read in large numbers of CSV files
# specifically read.csv.sql, read.csv (optimised) and fread
library(sqldf)
setwd("~/Downloads/wordcounts")
files <- sample(list.files(".", pattern="*.csv|CSV$"), 10000)
############# read.csv.sql ###################
system.time(