Skip to content

Instantly share code, notes, and snippets.

@pdeffebach
pdeffebach / DataFramesMetaProposal.md
Last active July 31, 2020 14:16
Proposal for new DataFramesMeta behavior

New DataFramesMeta.jl behavior

Goal

The goal of this document is to outline proposed new behavior for DataFramesMeta and it's integration with the new DataFrames.jl "piping" functions, e.g. transform, select, combine, and filter.

Limitations of current implementation

  1. The most glaring limitation of the current behavior of DataFramesMeta is
module MWEMissings
using Missings, InteractiveUtils
export TwoVectors, mapreduce_impl_bad, mapreduce_impl_good
struct TwoVectors{V}
x::V
others::V
end
@pdeffebach
pdeffebach / testperf.jl
Last active March 11, 2020 14:58
Performance of skipmissings
using BenchmarkTools, Missings
function make_x_y(N, p)
x = map(1:N) do _
rand() < p ? missing : rand()
end
y = map(1:N) do _
rand() < p ? missing : rand()
end
sx = skipmissing(x)
@pdeffebach
pdeffebach / q5.jl
Created February 29, 2020 18:32
bug in finding error
# Question 5
module Q5
using LinearAlgebra, Distributions, DataFrames
function make_data(T, N, β, σ)
T = map(1:T*N) do i
x = rand(Normal(0, sqrt(σ)))
u = rand(Normal(0, abs(x)))
y = x * β + u
x, y
@pdeffebach
pdeffebach / TestGroupedTransform.jl
Created September 30, 2018 21:00
DataFramesMeta tests
using DataFrames, DataFramesMeta, BenchmarkTools
function scalarfun(g::GroupedDataFrame)
@transform(g, t = :b[1])
end
function vectorfun(g::GroupedDataFrame)
@transform(g, t = :b .- 1.0)
end