jrevels

Comprehensive row validation requires row construction

If I am given a row value x of unknown provenance that I'd like to comprehensively validate (i.e. validate field values as well as field types) against schema s, then my only option is to construct a new row y = Legolas.row(s, x), at which point I can use y as my "s-validated" row. There is no mechanism to fully validate x "in-place".

The primary reason that this is the case is that field assignments may introduce implicit value-level constraints on a schema's input/output domain, and there is no enforceable mechanism by which schema authors are made to explicitly surface independently computable predicates for these implicit constraints.

For example, take the schema:

@schema("foo@1",

jrevels

using UUIDs, Dates, Arrow, Tables

#####
##### Signals
#####

struct Signal <: Tables.AbstractRow
    recording_uuid::UUID
    type::String
    file_uri::String

jrevels

using UUIDs, Dates, Arrow, Tables

#####
##### Signals/Annotations/Recordings
#####

struct Signal
    file_uri::String
    file_metadata::Union{Nothing,Dict{String,String}}
    channel_names::Vector{String}

jrevels

using UUIDs, Dates, Arrow, MsgPack

MsgPack.msgpack_type(::Type{Nanosecond}) = MsgPack.IntegerType()
MsgPack.from_msgpack(::Type{Nanosecond}, x::Integer) = Nanosecond(x)
MsgPack.to_msgpack(::MsgPack.IntegerType, x::Nanosecond) = x.value

MsgPack.msgpack_type(::Type{UUID}) = MsgPack.StringType()
MsgPack.from_msgpack(::Type{UUID}, x::String) = UUID(x)
MsgPack.to_msgpack(::MsgPack.StringType, x::UUID) = string(x)

jrevels

using Cassette, Test
using Cassette: @context, enabletagging, @overdub, overdub, recurse,
                hasmetadata, metadata, tag, untag
using ChainRules: frule, Zero, extern

@context DiffCtx

Cassette.metadatatype(::Type{<:DiffCtx}, ::Type{T}) where {T<:Real} = T

function D(f, x)

jrevels

Beacon Biosignals | UI/UX Engineer

(Boston, MA | Onsite Available, Remote Friendly | Full Time)

About Us:

Despite its significant potential for improving patient outcomes, brain monitoring is still not easily accessible or interpretable in clinical settings. We're going to fix that, and we'd like you to help.

We're a stealth-mode startup founded by numerical programmers, neuroscientists, ML researchers, and practicing neurologists who are committed to translating our best-of-breed clinical research from the lab into hospitals and beyond. We're well-funded, well-connected, and own a well-labeled set of brain data amassed over the past decade at some of the most prestigious medical institutions in the world. This dataset is, as far as we know, the largest of its kind in existence. We intend to put it to good use.

jrevels

Beacon Biosignals | Senior Machine Learning Engineer

(Boston, MA | Onsite Available, Remote Friendly | Full Time)

Our early-stage startup is seeking an individual to lead machine learning research/development efforts at Beacon, and along the way teach us all all how to deliver robust models to improve human health.

About Us:

Despite its significant potential for improving patient outcomes, brain monitoring is still not easily accessible or interpretable in clinical settings. We're going to fix that, and we'd like you to help.

jrevels

# NOTE: Make sure to run this in a "fresh" session. If the method you're invoking 
# has already been compiled, then the compiler might grab its old results instead
# of populating the cache we're constructing here.

mymethod = # callable value you want to check
mysig = Tuple{typeof(mymethod), #= types of arguments here =#}
p = Core.Compiler.Params(typemax(UInt))
Core.Compiler.typeinf_code(methods(mymethod).ms[end], mysig, Core.svec(), false, p)
Core.show(map(((i, x),) -> (i, x.result, x.linfo), enumerate(p.cache)))

jrevels

using ForwardDiff, ReverseDiff, BenchmarkTools

n = 100
x = rand(n)
v = rand(n)
tape = ReverseDiff.compile(ReverseDiff.GradientTape(cumprod, rand(n)))
vJ = rand(n)'

function vecjacobian_new!(vJ, v, tape, x)
    input = ReverseDiff.input_hook(tape)

jrevels

using ForwardDiff, ReverseDiff, BenchmarkTools

mutable struct MyJacobianWrapper{fType,tType} <: Function
  f::fType
  t::tType
end

function (ff::MyJacobianWrapper)(u, p)
  du1 = similar(p, size(u))
  ff.f(du1,u,p,ff.t)