Skip to content

Instantly share code, notes, and snippets.

View lorinc's full-sized avatar

Lőrinc Nyitrai lorinc

View GitHub Profile
@lorinc
lorinc / sum_of_some_calls.sql
Last active August 29, 2015 14:14
simple data aggregation problem - SQL solution
/*
problem: return the sum of call durations for those operators
who initiated at least one call per day for a given
timeslot. data: calls(mtid, date, duration)
*/
SELECT sum(duration)
FROM calls
JOIN
(SELECT MTID, count(MTID)
@lorinc
lorinc / java_app_wrapper.sh
Last active August 29, 2015 14:14
bash wrapper to (recursively) manage multiple Bukkit servers running in screens
#!/bin/bash
# todo:
# - one script to control all 3 servers
# - script to know expected screens per server
# - pull functionalities into seperate functions e.g. is_server_up()
logfolder="/home/mc/servers/logs/"
this_box=`hostname | awk 'BEGIN {FS="."} {print toupper($1)}'`
backup_folder='/home/mc/servers/backup/'
@lorinc
lorinc / lr-plot.r
Created February 2, 2015 21:18
400 lines of ETL, data cleansing, cross-validating, plotting and exploratory analysis in R using data.table and ggplot, using SFDC, Jira and accounting data
#### reading libraries ####
library(data.table)
library(ggplot2)
library(gridExtra)
#### reading sources ####
setwd("~/Desktop/consolidated")
# "SFaccountID","SFopportunityID","SFaccountName","SFcreateDate","SFendDate","SFindustry","SFproductName","SFtotalPriceCurrency","SFtotalPrice"
@lorinc
lorinc / parentesis_checker.ipynb
Last active September 29, 2016 23:32
We asked this as a Java interview question and I HAD to show them the beauty of Python ;)
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lorinc
lorinc / feature_extraction_from_images.ipynb
Last active October 17, 2016 18:59
feature extraction experiment on the Swedish Leaf Dataset
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lorinc
lorinc / nested_table_iterator.lua
Last active February 7, 2019 05:11
This is a function I wrote to support deeply nested data structures at Google in the Cloud Support BI pipeline.
-- iterator function to traverse nested tables
function iterate(tbl, parent)
parent = parent or {}
if (type(tbl)=="table") then
for key, value in pairs(tbl) do
iterate(value, table.extend(parent, key))
end
end
coroutine.yield(parent, tbl)
end
@lorinc
lorinc / scatterplot_matrix.ipynb
Last active February 7, 2019 05:23
matplotlib + pandas + scipy.stats + numpy example
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lorinc
lorinc / hierarcy-to-human-readable-ids.ipynb
Last active February 7, 2019 06:04
a data munging challenge I had to solve at UBS
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lorinc
lorinc / heart.ipynb
Created June 17, 2019 18:50
heart.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lorinc
lorinc / distance_calculation.sql
Last active August 22, 2019 17:09
Visualization of political polarization in Hungary using public parliamentary voting data using Google BigQuery. Pro bono work for atlatszo.hu, the recognized Hungarian pro-democracy data journalist group.
/*
Pairwise distance between every single members of the parliament
active in that specific term, for every single voting that happened.
Generates a 61GB dataset for the 14,452 voting events.
Next step is reducing the amount of data, using roling window aggregation
on the time dimension, and clustering on the people dimension.
*/