Skip to content

Instantly share code, notes, and snippets.

@rikkimax
Last active April 6, 2021 06:01
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rikkimax/826e1c4deb531e8dd993815bf914acea to your computer and use it in GitHub Desktop.
Save rikkimax/826e1c4deb531e8dd993815bf914acea to your computer and use it in GitHub Desktop.

My vision for D in the near future

As of this writing of this article D is on the verge of getting memory safety mechanisms that prevent memory escaping and use after free. Which great for D but it can surpass other languages in the C family both in terms of abstraction and also memory safety.

Step 1. An extension on the borrow checker

Walter Bright is currently in the process of developing a DIP for a borrow checker. This was spawned on by the success of Rust and its borrow checker. But unfortunately in terms of syntax this is a kind of magic syntax with different semantics being applied to a pointer even if the same syntax is being used within a function.

We I think can do better than applying different semantics to a pointer magically.

Instead of an automatic detection and determination lets introduce a new storage class called headconst. This is fairly simple in behavior that you may have already guessed. It is a memory reference that cannot be altered in anyway nor implicitly casted to anything else (as to prevent free'ing or corrupting the memory).

The effect of this new storage class is of course transitive just like const is today. To prevent any internals being linked by pointer.

As a reference it is either a slice or a pointer. The protections are from the compiler, and are not reflected within the codegen itself.

Of course since headconst is tied to the lifetime of the memory owner, it will rely on the already existing work of DIP25, DIP1000 and of course the work that Walter has already done for his borrow checker implementation.

I propose a head-const implementation over the borrow checker for a single but important reason. From my recollections I do not remember anyone asking for a borrow checker but countless of others asking for head-const in D. This solution has multiple use cases on top of memory safety that will satisfy even the most uncaring for memory safety to provide it.

An example of this can be found here: https://gist.github.com/rikkimax/4cb2cc8ddcac33c1a9bb20de432f9dea

Step 2. Named members via template parameters

This proposal adds the ability to specify types and values which are members of a template via a template parameter. It is an extension of a named arguments DIP such as Walter's and was included in my DIP 1020.

There are a few variants of syntax, but they all do the same thing. A few of these are:

alias X = X;
enum Y = Y;
@property {
    typeof(auto) SomeType;
    auto SomeValue;
}

and

template Foo(public SomeType, public int SomeValue) {
}

Which can be used like:

static assert(Foo!(SomeType: int, SomeValue: 0).SomeType == int);

Draft DIP pre template: https://gist.github.com/rikkimax/046fb4451e8cbac354ecb292f9a76798#file-template-parameters-as-properties-of-a-type-md

Signatures

Now you may be thinking why did this get a heading and not be a step? Well it is simple, this section will be devided up into two further sections.

First is type classes, this provides a static interface which can be used to compare against other types. This is the set of features most people who know signatures (type classes, Rust trait's and Swift protocols) will expect.

The second set is the dynamic bits which give signatures their runtime representation. Programmers from C and related languages will recognize a few design decisions due to having written code like it.

Step 3. Type verification

We need a way to tell the compiler directly that we want to compare another type against a specification and what defines that specification.

There will always be situations where we can't do that directly so instead we provide ways to inform the compiler what those conditions are and why it did or probably did fail. This can be done by using static assert with its message parameter for why it failed. These errors can be eaten by the compiler and fail any type checks being done implicitly. Due to being eaten, they can be propergated up should the need arise in the calling code.

For type verification purposes a signature has three items in its body of interest. First is the ability previously discussed to say some property of the implementation is wrong. The second is named members settable via template parameters but is also able to be inferred from an implementation. Lastly is method declarations.

A method declaration is the same as a method declared in a interface. It has no body unless its final.

A signature is able to inherit from other signatures too. Unlike with a class system there is no problem of a diamond inheritance. Signatures work based upon less restrictive in the interface to more restrictive in the implementation. In other words a child of another signature may make the method more specialized (i.e. adding const, child class as a return type ext.). But it may not remove existing specializations.

An example of this is:

signature InputRange {
    @property {
        typeof(auto) ElementType;
    }
    
    ElementType front();
    bool empty();
    void popFront();
}

signature ForwardRange : InputRange {
    ForwardRange!(ElementType: ElementType) save();
}

You can of course compare a signature instance and a signature declaration via the is expression with awareness of happening inside another signature of the same declaration (false if true). With : checking for the non-template-initiated type and == checking the exactness (with or without template initialization) exactness. This same method is used for comparing a signature (either the declaration or the template inititialized version) with a struct or class.

Existing template parameter syntax is extended to match existing = and : support as per the prior statement of is expression.

Step 4. Virtual representation

Some type that can construct a Signature instance:

void func(auto:Signature arg) => void func(T)(T:Signature arg)

Verify and document return type of a function:

auto:Signature func() =>

auto func() { ... }
static assert(is(ReturnType!func : Signature));

Signature as a function parameter without being template initialized:

void func(Signature value) => void func(T:Signature)(T value)

ABI:

struct SignatureInstance(ParamsExt) {
    copy constructor
    destructor

    void* ptr; // aka this
    ...fields
    ...methods
}

In other words: void*[Size].

Where field is a pointer and method is a function pointer (unless there are cases where it must be a delegate and have its own context pointer). Copy constructor and destructor are treated as if they were methods that are virtual.

A signature instance can be implicitly created when passed as a function argument or as a return value.

I.e.

Signature foo(Signature) {
    return ImplementationB();
}

...

foo(ImplementationA());

Within a signature this refers to the implementation and never the signature itself. typeof(this) is available when inferring and the expression (working upwards) should be replaced with false. So that:

signature Foo {
    static assert(!is(typeof(this)) || __traits(hasMember, typeof(this), "something"));
}

when inferring the second expression in the static assert is used and not the first.

Methods

A method that is final is described under type verification, otherwise it is virtual.

If an implementation has a method, then the signature does not need to provide a body for a method declaration. If a method body is provided and the implementation provides one, then the implementation is chosen unless override is used on the method declaration.

Step 5. Safe by default

Making safe be the default in D is a highly desirable feature that would create breakage in the short to medium term and will likely split the community in some way.

But there is a way to do it with minimal to no code breakage.

This is done by utilizing the head-const feature described in step 1.

A function is automatically @safe if it fulfills this rule: All pointer declarations (slices, classes, raw ext.) including parameters must be headconst or it must have an expression assigning it via new. If the later of the two of the rule it must never be assigned to.

To work around this you can mark a function @system or @trusted to get the old behavior.

Furthermore ref and out parameters on a function should be rewritten as headconst while keeping the same behavior from outside the function.

@rikkimax
Copy link
Author

rikkimax commented Mar 9, 2021

Signatures

Grammer

AggregateDeclaration:
+    SignatureDeclaration

+ SignatureDeclaration:
+     signature Identifier AggregateBody
+     signature Identifier TemplateParameters Constraint|opt BaseSignatureList|opt AggregateBody
+     signature Identifier TemplateParameters BaseSignatureList Constraint AggregateBody

+ BaseSignatureList:
+    : Signatures

+ Signatures:
+    Signature
+    Signature , Signatures

DeclDef:
+    @ property AliasDeclaration
+    @ property EnumDeclaration

Parameter:
+    ParameterAttributes|opt Type : Type

StorageClass:
+    auto : Type

AtAttribute:
+    @ SliceBox ! Type

NOTE: EnumDeclaration definition looks off
NOTE: Parameter and StorageClass may be wrong, but it is the most convenient place to put it in the grammer right now

Compile Time

Interface vs Implementation vs Concrete Interface

FIXME: concrete interface???

This document introduces a new type called a signature. A signature is an interface that may be used for structural typing.
The format it takes is that of a polymorphic typing of inheritance where the declarations utilize type recursion to determine validity of the symbols within the body but not for member property expressions and method bodies, for that it uses type dependecy.

Type dependency within the D programming language in the context of this specification refers to the this reference pointing to an implementation type with any user of it being templated around it.

@property alias ElementType = typeof(this).ElementType;

// becomes

alias __WrappedElementType(This) = This.ElementType;
InputRange!(ElementType: __WrappedElementType!Map);

// where InputRange is the signature, ElementType is the member property (being set via template parameter rather than being inferred), and Map is the implementation type

An implementation type will be defined as either a struct or class. An implementation instance shall be defined as a pointer to a struct or a class non-null reference.

signature IFoo {
    void func() {
        writeln(this);
    }
}

// becomes

void __WrappedIFoo_func(This)(void* this) {
    writeln(cast(This)this);
}

&__WrappedIFoo_func!MyFoo
// if MyFoo does not have the method func, it will use this template instance as a fallback instead

REFERENCE: William R. Cook, Walter Hill, and Peter S. Canning. 1989. Inheritance is not subtyping. In Proceedings of the 17th ACM SIGPLAN-SIGACT symposium on Principles of programming languages (POPL '90). Association for Computing Machinery, New York, NY, USA, 125–135. DOI:https://doi.org/10.1145/96709.96721

Type Checking

A signature may be used as part of type checking during compilation against an implementation type. This can be done inside an is expression, at a variable declaration, function or template parameter and at the return type of a function.

void atCompileTime(T : InputRange)(T arg) {}
void atCompileTimeWithType(T : InputRange!(ElementType: int))(T arg) {}

auto : InputRange atCompileTimeReturn() { ... }
auto : InputRange!(ElementType: int) atCompileTimeReturnWithType() { ... }
  1. A type checked variable declaration, parameter or return type uses the Identifier : Type syntax.
    1. When the Identifier is the auto keyword the compiler must detect this to template the outer symbol. For example void func(auto:InputRange) will become void func(__param1)(___param1:InputRange). Where the name __param1 was made up for this example.
    2. If a given Identifier does not resolve to auto it may resolve to a template parameter of the given symbol or another type.
    3. During comparison the Type to check against if it is a template instance of a signature then the checks are further extended to include checking if the signature is not only the signature itself but also of that specific template instance.
  2. A type checked is expression is similar to a type checked variable declaration, parameter or return type except it accepts a Type rather than an Identifier. Allowing for more complex operations including CTFE.

Opaque Signature

An opaque signature is hereafter defined as any given declaration, parameter or return type known at the point of a scope as either being able to become an instance of the given signature.

void forRuntime(auto : InputRange arg) {}
void forRuntimeExplicit(T)(T : InputRange arg) {}
void forRuntimeWithType(auto : InputRange!(ElementType: int) arg) {}
void forRuntimeWithTypeExplicit(T)(T : InputRange!(ElementType: int) arg) {}

void someFunctionBody() {
    auto valueA = atCompileTimeReturn();
    auto : InputRange valueB = atCompileTimeReturn();
    auto : InputRange!(ElementType: long) valueC = atCompileTimeReturn();
}
  1. If the compiler knows that a given declaration, parameter or return type is an opaque signature, it is to be treated as though it is was that signature during symbol resolution. The order in which the symbol lookup occurs:
    1. Final methods of the signature instance
    2. Implementation
    3. Non-final methods bodies from the signature instance
  2. The compiler shall not attempt to guess if a given declaration, parameter or return type is an opaque signature, instead it will be told by the user in using the Identifier : Type syntax.

Member Parameter

A member parameter is a template parameter that is declared within the body of a signature. It is exposed as a member but will be set by template initialization.

  1. Member parameters shall be defined as an alias or enum declaration that has a preceeding @property attribute. These are to be appended to the end of the template parameter sequence during parsing and must remain exposed as a member during symbol lookup.
  2. Typically member parameters will be assigned a value by inferation against an implementation type. But they may be assigned a value via a named template argument. Support for ordered template argument assignment is not required but should be preferred.
  3. Must not be guarded by a static if conditional. A version may guard it as long as it was not set inside the signature.

This Context

Within the scope of a signature body, having access to the implementation is required to properly infer more complex symbols. Hereafter this within a signature body shall be defined as refering to a reference to the implementation.

  1. Within a signature body there are two usage patterns for the this keyword.
    1. In a member property it may refer to typeof(this) and these shall be treated as the default values during inferation if the implementation type does not have the member properties name as a member.
    2. Method bodies may refer to this as to be an opaque signature to the implementation instance. Allowing for complex decisions, fallbacks and behaviors.

Slice Boxing

Signatures can only have implementation types of type struct or class, slices would enable input ranges to continue to work while migrating to a signature based design.

Explicit wrapping is an option, but a simpler alternative that the user would not need to be aware of is an opaque boxing of slices.

@SliceBox!InputRangeSliceBox
signature InputRange {}

struct InputRangeSliceBox(T) {
    T value;
    
    ...
}
  1. An attribute named SliceBox is added, that must be provided a struct that must have:
    1. A single template parameter
    2. A signel field of type provided by the template parameter
    3. Can only be applied on a signature declaration
  2. No instance of the struct actually needs to exist. A variable declaration can be treated as though it was the struct opaquely by the compiler.

Run Time

Creating An Instance

ABI

@rikkimax
Copy link
Author

rikkimax commented Mar 9, 2021

@RUSshy

The way you wrote the above would require runtime allocations, instead the below one will check if Wall or Tree is Drawable at compile time, pass the instance by ref. Meaning no memory allocations.

It will also give you nice errors saying why something is not Drawable when you attempt to pass in say an Object rather than a Wall or Tree.

class Canvas {}

signature Drawable {
    void draw(Canvas);
}

void gfx_draw_entity(ref auto:Drawable drawable) {
    drawable.draw(canvas);
}

@rikkimax
Copy link
Author

rikkimax commented Mar 9, 2021 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment