tshort

@with macro

Here is an @with macro to evaluate DataFrame (or even Dict) columns using references.

replace_syms(x, membernames) = x
function replace_syms(e::Expr, membernames)
    if e.head != :quote
        return Expr(e.head, (isempty(e.args) ? e.args : map(x -> replace_syms(x, membernames), e.args))...)
    else

tshort

Simon's DataFrame mock-up using staged functions

module SimonDataFrame

type DataFrame{N,D}
    data::D
end

tshort

hello world

# /: comment
aK

 \ hello world
.asdf#wer 456

tshort

function str(io::IOStream, x, n::Int, indent)
    T = typeof(x)
    print(io, T, " ")
    if isa(T, CompositeKind)
        println(io)
        if n > 0
            for field in T.names
                print(io, indent, "  ", field, ": ")
                str(io, getfield(x, field), n - 1, strcat(indent, "  "))
            end

tshort

Here are versions of with and transform! for DataFrame's. They each operate on columns.

function with(df::DataFrame, ex::Expr)
    # by-column with operation on df
    
    # helper function to replace symbols in ex with a reference to the
    # appropriate column in df
    replace_symbols(x, syms::Dict) = x
    replace_symbols(e::Expr, syms::Dict) = Expr(e.head, isempty(e.args) ? e.args : map(x -> replace_symbols(x, syms), e.args), e.typ)

tshort

Here are results of Julia's benchmarks with and without bounds checking:

DataFrame  (16,3)
                         type       benchmark     time
[1,]     "no bounds checking"           "fib" 0.210762
[2,]               "standard"           "fib" 0.211954
[3,]     "no bounds checking"        "mandel" 0.572634
[4,]               "standard"        "mandel" 0.587797
[5,]     "no bounds checking"     "parse_int" 0.207615

tshort

test

# test

* bullet 1
  * bullet 1a

    indented=code();

tshort

Here is the definition:

function methodswith(io, t::Type, m::Module)
    for nm in names(m)
        try
           mt = eval(nm)
           d = mt.env.defs
           while !is(d,())
               if any(map(x -> x == t, d.sig))

tshort

Problems with a groupby transform involving sum

In this spec, the groupby transform works just fine using any of median, count, or average. When I use sum, the graph doesn't show.

tshort

Problems with a groupby transform involving sum

In this spec, the groupby transform works just fine using any of median, count, or average. When I use sum, the graph doesn't show. This version should display right.