This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
using CSV, Feather | |
#using JLD2 | |
#using JLD#, JLSO | |
using JDF, FileIO, Blosc, StatsPlots, RCall | |
using DataFrames, WeakRefStrings # required for JLD2, JDF | |
Blosc.set_num_threads(6) | |
gen_benchmark(dirpath, largest_file, outpath, data_label; delim = ',', header=true) = begin | |
if !isdir(outpath) | |
mkpath(outpath) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
library(data.table) | |
library(disk.frame) | |
setup_disk.frame() | |
bench_disk.frame_data.table_group_by <- function(data1,n) { | |
setDT(data1) | |
a.sharded.df = as.disk.frame(data1, shardby = c("year", "month", "day")) | |
a.not_sharded.df = as.disk.frame(data1) | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
FROM python:3.7-alpine3.10 | |
RUN apk add --no-cache \ | |
build-base \ | |
cmake \ | |
bash \ | |
boost-dev \ | |
autoconf \ | |
zlib-dev \ | |
flex \ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
using JDF | |
using CSV, DataFrames | |
using Serialization:serialize,deserialize | |
using BufferedStreams | |
@time a = CSV.read("C:/Users/ZJ.DAI/Documents/git/format-wars/data/Performance_2016Q4.txt", delim = '|', header = false); | |
io = BufferedOutputStream(open("c:/data/bin.bin","w")) | |
@time serialize(io, a) | |
close(io) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
using DataFrames | |
df1 = DataFrame(a = repeat([1], 100), b = "a") | |
df2 = DataFrame(a = repeat([2], 100), b = "b") | |
B = Array{Bool, 2}(undef, 100, 2) | |
df1[B] # doesn't work | |
# Let's overload get index get index |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
x = rand(1:1_000_000, 1_000_000_000) | |
using SortingLab | |
fsort!(x) | |
function unroll_loop(x) | |
count = 0 | |
@inbounds count += x[1] != x[2] | |
@inbounds count += x[2] != x[3] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
struct Player | |
name::String | |
rating::Int | |
end | |
struct Match | |
best_of::Int | |
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
using Revise | |
includet("utils.jl") | |
function remove_player!(team) | |
if length(team) == 1 | |
team.players = [] | |
else | |
team.players = team.players[2:end] | |
end | |
team |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
using Gumbo, Cascadia, HTTP | |
using Serialization | |
urls= ["https://www.shicimingju.com/shicimark/tangshisanbaishou.html"] | |
urls = vcat(urls, ["https://www.shicimingju.com/shicimark/tangshisanbaishou_$(i)_0__0.html" for i in 2:16]) | |
function get_chars(poem::Vector{<:AbstractString})::Set{Char} | |
mapreduce(Set, union, poem) | |
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
library(shiny) | |
# Define server logic for random distribution application | |
shinyServer(function(input, output) { | |
# Reactive expression to generate the requested distribution. This is | |
# called whenever the inputs change. The renderers defined | |
# below then all use the value computed from this expression | |
data <- reactive({ | |
dist <- switch(input$dist, |