Created
June 11, 2020 12:58
-
-
Save jordanlewis/3e90817323a2725e2b2c1bd27ff41813 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
proposal for syntax/method for doing type templating: | |
all functions that need to do type-specific manipulation (let’s say, equals for the purpose of this example) will get a template argument T. T will be a struct TypeInfo (naming tbd) that contains the types.T information, as well as //execgen:inline’d methods that provide each overload, as well as utility methods. | |
still working on syntax for the “go type templates” - aka, how do we specify that a template needs int and []int in the case of int64 columns, but []byte and coldata.Bytes in the case of bytes columns, so ignore that for now. | |
for example: | |
// execgen:template<typ> | |
func genericEqualityFunc(colA []_GOTYPE, colB []_GOTYPE, idx int, typ *TypeInfo) bool { | |
a := typ.UnsafeGet(colA, idx) | |
b := typ.UnsafeGet(colB, idx) | |
eq := typ.Equals(a, b) | |
return eq | |
} | |
there will be associated, inlinable methods for each TypeInfo, e.g. | |
type boolTypeInfo struct { | |
... | |
} | |
// execgen:inline | |
func (b intTypeInfo) Equals(a int, b int) bool { | |
return a == b | |
} | |
// execgen:inline | |
func (b intTypeInfo) UnsafeGet(col []int, idx int) { | |
return col[idx] | |
} | |
(edited) | |
10:21 | |
(this won’t work yet for various reasons but this is my proposal, generally) | |
rohany 10:22 PM | |
are you going to hand generate these *TypeInfo structs? | |
jordan:sparkles: 10:22 PM | |
yes, i think so | |
10:22 | |
they’re essentially hand generated already | |
rohany 10:22 PM | |
so now we're really like c++ template specialization | |
10:23 | |
I'm not clear how dispatching of *TypeInfo happens to the Specialized overloads | |
10:23 | |
Will that be an interface? | |
jordan:sparkles: 10:23 PM | |
TypeInfo itself would be an interface, but the interface wouldn’t be accessed at runtime unless someone chose to use one in a non //execgen:template method | |
10:24 | |
one cool thing btw about the //execgen:foo things so far is that a few times while playing with it, i’ve forgotten to include one or both of the annotations and everything still works :slightly_smiling_face: | |
:+1: | |
1 | |
rohany 10:24 PM | |
So how does someone use genericEqualityFunc | |
jordan:sparkles: 10:28 PM | |
well i’m still not certain about the exact mechanisms of passing in colA and colB. so that’s tbd. | |
but for the type param you’d either do genericEqualityFunc(colA, colB, intTypeInfo) (concrete template invocation) or genericEqualityFunc(colA, colB, typeInfo) where typeInfo was a type parameter that you were passed | |
the root of the chain here would eventually need to be the per-type operator structs, i think. so that’s more syntax we need to invent, but the templater would see something like this: | |
// execgen:template <typ> | |
type myOperator struct { | |
... stuff ... | |
typ TypeInfo | |
} | |
and convert that into myOperator_int for all types. then, calling genericEqualityFunc(colA, colB, o.typ) from a method on o would be a static call? i think? | |
rohany 10:29 PM | |
but for the type param you’d either do genericEqualityFunc(colA, colB, intTypeInfo) (concrete template invocation) or genericEqualityFunc(colA, colB, typeInfo) where typeInfo was a type parameter that you were passed | |
10:29 | |
ok this makes sense, and is a straight forward extension of the bool stuff we have | |
:that: | |
1 | |
10:29 | |
assuming that we can do the for each type | |
jordan:sparkles: 10:29 PM | |
yeah, that’s … also tbd but hey | |
rohany 10:30 PM | |
idt thats bad -- if we are hand writing these type structs, then there would be some way of registering them that the templater can access | |
jordan:sparkles: 10:30 PM | |
yeah. or // execgen:template <typ (types.A, types.B, types.C), b (true, false)> (edited) | |
rohany 10:30 PM | |
and convert that into myOperator_int for all types. then, calling genericEqualityFunc(colA, colB, o.typ) from a method on o would be a static call? i think? | |
10:31 | |
this seems right, but also seems like it might be hard | |
10:31 | |
i don't know if at the AST level you can do things like this (or have access to them) | |
jordan:sparkles: 10:31 PM | |
yeah…. turning o.typ into static text would be impossible i think | |
10:31 | |
well, unless you search for it during templating | |
10:31 | |
which is definitely possible - you have the receiver and the struct fields | |
rohany 10:31 PM | |
well you don't have to right -- i thought it was just going to be an opaque interface | |
10:32 | |
we can edit each of the generated structs to include the "real" type at template time | |
10:32 | |
so in the template, you'll access o.typ | |
10:32 | |
but in the generated code, we'll switch out o.typ for the real specialized one | |
10:32 | |
so then it is all static | |
jordan:sparkles: 10:32 PM | |
yes, i agree, that’s what i was trying to say as well | |
:+1: | |
1 | |
rohany 10:33 PM | |
this is pretty ambitious, but very cool | |
10:33 | |
:slightly_smiling_face: | |
jordan:sparkles: 10:33 PM | |
:smiling_imp: | |
10:33 | |
i don’t think it’s actually that bad | |
rohany 10:33 PM | |
getting the template pass through i think is the next step | |
10:33 | |
then this stuff kinda falls out | |
jordan:sparkles: 10:33 PM | |
oh i have template pass through already! :dancingpanda: | |
rohany 10:33 PM | |
oh what | |
10:33 | |
beast | |
10:34 | |
the recursive pass thing? | |
jordan:sparkles: 10:34 PM | |
yeah | |
rohany 10:34 PM | |
is that landed | |
10:34 | |
did i just not notice this in the pr | |
jordan:sparkles: 10:35 PM | |
i don’t think so. but it was easier than expected. it’s a change to monomorphization, which is now 2 passes. | |
pass 1: replace all Idents that are template formal param names with template args | |
pass 2: do the whole if/else reduction crap (btw this is not even strictly necessary as Go does good static falsification of branches but it does clean the code a lot) | |
just pass 1 by itself (as long as you do it recursively and not just in a single go) does the template pass through (edited) | |
rohany 10:36 PM | |
that makes sense -- maybe it was one of those harder to think of than write | |
:heavy_plus_sign: | |
1 | |
jordan:sparkles: 10:36 PM | |
so the output code looks like | |
if true { | |
if false { | |
... | |
else { | |
... | |
} else { | |
... | |
} | |
(edited) | |
10:36 | |
etc, probably missed braces | |
rohany 10:36 PM | |
are there any templates where we only do the type templating for a small subset of types? | |
10:37 | |
I'm not sure how progress would be made in this without creating all of the templated structs | |
10:37 | |
first | |
jordan:sparkles: 10:37 PM | |
i don’t remember, but probably | |
10:38 | |
this proposal also doesn’t yet help situations where we uniformly produce all projection/selection operators, as those have another layer of templating over operators themselves, but i think we’ll get there.. | |
rohany 10:38 PM | |
we'll get there when we get there | |
10:38 | |
this should also help all of the nested overloads we have right | |
jordan:sparkles: 10:39 PM | |
i think my (crappy) syntax proposal would help in your case tho | |
//execgen:template<a (types.Int, types.Bool)> (edited) | |
:+1: | |
1 | |
rohany 10:39 PM | |
we'll actually have int8, int16, int32, int64 type structs, rather than a nested thing underneath IntFamily | |
jordan:sparkles: 10:39 PM | |
we already have all those type structs, the nesting is just in the constructor | |
rohany 10:40 PM | |
yeam making a simple specialization syntax would help different edge cases here and there | |
10:40 | |
i meant that it would help us unnest this stuff | |
10:40 | |
more naturally | |
jordan:sparkles: 10:40 PM | |
ah yeah i agree | |
jordan:sparkles: 10:59 PM | |
i forgot that you can’t really have a true TypeInfo interface unless you added templated params to interfaces, which seems pointless. but most of the idea still holds, i think (edited) | |
rohany 11:00 PM | |
can't the TypeInfo interface just operate solely on interface{} | |
11:00 | |
so then you can use it like you have all the individual typeInfos pretemplating | |
jordan:sparkles: 11:01 PM | |
foo() int doesn’t implement foo() interface{} is the issue | |
https://play.golang.org/p/aEWpC_8qUmh | |
rohany 11:04 PM | |
wouldn't the pretemplating function return like _GOTYPE though, not int? | |
jordan:sparkles: 11:05 PM | |
_GOTYPE is an artifact of the original template method (it gets regex replaced with {{.GoType}} which is a member variable on the text/template type struct) and something i was hoping to find a way to get rid of | |
rohany 11:07 PM | |
I see | |
11:07 | |
alright, have to :brain4: | |
11:07 | |
It feels important to me that users of TypeInfo pretemplating should feel like they're looking at real go | |
11:08 | |
or at least the editor shouldn't complain | |
jordan:sparkles: 11:08 PM | |
i agree | |
11:08 | |
which does make it seems like everything should appear to be interface{} | |
rohany 11:11 PM | |
perhaps named arguments to the rescue here (somewhat?) | |
11:11 | |
template<t1, t2> | |
func x (t1 ...) (t2 ...) { | |
} | |
11:12 | |
can at least acknowlege that we're going to template over the return type or something | |
rohany 11:38 PM | |
foo() int doesn’t implement foo() interface{} is the issue | |
Can you elaborate why this is a problem / when we run into this | |
jordan:sparkles: 11:44 PM | |
type TypeInfo interface { | |
UnsafeGet(col interface{}, idx int) interface{} | |
} | |
type intInfo struct {} | |
func (i intInfo) UnsafeGet(col interface{}, idx int) interface {} { | |
return col[idx] // <-- can't do this because interface{} isn't sliceable | |
} | |
func (i bytesInfo) UnsafeGet(col interface{}, idx int) interface{} { | |
return col.Get(idx) // <--- can't do this because col isn't usable as a coldata.Bytes | |
} | |
11:45 | |
one option is that you could make a new type like | |
type AnyColSlice []interface | |
func (a AnyColSlice) Get(i int) interface{} | |
... and all methods that you might have on coldata.Bytes and other non-slice col types | |
(edited) | |
11:46 | |
and then at codegen time none of this stuff matters because the types will go away. but, dunno. the other thing that’s still missing is how to extract the text of the Go type for the calling scope. definitely still lots of missing pieces. | |
rohany 11:49 PM | |
I don't think the type specific structs need to implement the templater TypeInfo interface | |
:open_mouth: | |
1 | |
11:49 | |
the implementations would be ruined by having to ascribe to some generic interface right? | |
jordan:sparkles: 11:50 PM | |
not rly cuz they’ll get inlined, and the inlining strips type information (or it could) | |
11:51 | |
and i was thinking misty-eyed that it’d be cool to keep things working whether or not the templating actually happened - so you could choose to use a slow dynamic TypeInfo implementation at runtime if you wanted lol. but it’s pointless (edited) | |
:open_mouth: | |
1 | |
11:51 | |
that’s just a super droppable requirement, so, i’ll just drop it | |
:jordan: | |
1 | |
11:55 | |
the other thing that’s still missing is how to extract the text of the Go type for the calling scope. definitely still lots of missing pieces. | |
I think for this it might be good to copy C++ again | |
// execgen:template <typename T, typename TSlice> | |
func (col TSlice, idx int) T { | |
return col[idx] | |
} | |
but this won’t look like it compiles in goland because TSlice and T aren’t real types. without something like this though i struggle to see how you’d propagate the proper text for each type’s scalar and slice types | |
11:56 | |
(by this i mean, someone somewhere has to convert var x T into var x bool or var x int) | |
rohany 11:56 PM | |
I'm confused why you need that though -- why can't intInfo just have a method get(col []int, idx int) int | |
11:57 | |
TypeInfo can have get(col []interface, idx int), but when its swapped out for intInfo then the specialized one is used | |
jordan:sparkles: 11:57 PM | |
yeaaaah… that’s fair. i guess i was thinking optimistically that we could also have a thing that would let you just do the simple one (return col[idx]) once and overload for the more complicated ones :badpokerface: | |
11:57 | |
moving too far over skis | |
rohany 11:58 PM | |
I'm hesitant to start thinking about type replacement stuff | |
11:58 | |
we only have a syntax tree :disappointed: | |
11:59 | |
that would let you just do the simple one (return col[idx]) | |
Oh i see. This is desirable |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment