floitsch@, lrn@ 2017-03-03
Dart2js, the VM and DDC sometimes use a different evaluation order. This document tries to capture the differences, and proposes an alternative specification.
In general, Dart evaluates all its expressions from left to right. However, there is an interesting exception: the arguments to a method invocation are evaluated before the receiver function is evaluated. Concretely, given o.foo(e1, ..., eN), Dart requires o, e1, ..., eN to be evaluated before evaluating o.foo. Most of the time o.foo is a method and the evaluation doesn't matter (since evaluating to a method doesn't have any visible side-effect). However, it makes a difference when o.foo is a getter.
class A {
get getter {
print("evaluating getter");
return (x) {};
}
}
int bar() {
print("in bar");
return 499;
}
main() {
var a = new A();
a.getter(bar());
}According to the specification, this program should print:
in bar
evaluating getter
Even though the a.getter is syntactically before the call to bar(), Dart requires the argument to the call to be evaluated first.
Whenever a member doesn't exist (or the shape doesn't match), the noSuchMethod function is invoked. This method receives a filled Invocation object which contains the arguments to the non-existing member. This means that for non-existing members, arguments also need to be executed first.
This has important properties for null errors. Since null errors are mapped to noSuchMethod executions on null, the arguments to null calls are evaluated before the error is thrown:
int bar() {
print("in bar");
return 499;
}
main() {
null.foo(bar());
}A valid output for this program is:
in bar
Unhandled exception:
NoSuchMethodError: The method 'foo' was called on null.
Receiver: null
Tried calling: foo(499)
#0 Object._noSuchMethod (dart:core-patch/object_patch.dart:44)
#1 Object.noSuchMethod (dart:core-patch/object_patch.dart:47)
#2 main (null.dart:6:8)
#3 _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:261)
#4 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:148)
Because of the evaluation order and noSuchMethod we end up with different behavior for seemingly similar constructs:
// The following invocations are *usually* the same, but aren't when `o` is null, or when
// `foo` is not a member of `o`.
o.foo(e1);
o.foo.call(e1);
(o.foo)(e1);Our implementations (VM, dart2js and DDC) all behave differently, and dart2js and DDC aren't even internally consistent. Dart2js' behavior depends on optimizations, and DDC treats dynamic and non-dynamic calls (on the same member) differently.
The "Tests" section at the end of the document shows the output of our implementations. The VM is the only one implementing the spec correctly.
Dart2js is mostly correct. It sometimes fails on null.
The expression o.foo(e1) is basically compiled to o.foo$1(e1) independently if foo is a getter or a method. It's the class' responsibility to provide a call-stub that maps function calls to getter invocations, followed by an invocation of the result. For example, if foo was a getter, then dart2js would generate the following JavaScript members:
prototype.get$foo = function() { /* code for getter */ };
prototype.foo$1 = function(x) { return this.get$foo().call$1(x); };
}This approach leads to the correct evaluation order except when o is null. Since ES5, JavaScript requires a strict left-to-right evaluation, leading to an exception on o.foo$1 before e1 is evaluated.
Interestingly, dart2js, may still produce the correct behavior when it can inline the foo method. In that case, dart2js inserts an explicit null check (o.toString) before the inlined code, but evaluates the arguments to the function in the correct order:
t1 = /*e1*/;
o.toString;
// inlined code of `foo`.The o.toString is used as null check because it is extremely efficient when the object is non-null (because the property exists on all objects) and correctly throws on null:
d8> null.toString
(d8):1: TypeError: Cannot read property 'toString' of null
null.toString
^
TypeError: Cannot read property 'toString' of null
at (d8):1:5
d8> undefined.toString
(d8):1: TypeError: Cannot read property 'toString' of undefined
undefined.toString
^
TypeError: Cannot read property 'toString' of undefined
at (d8):1:10
This means that the evaluation order is optimization dependent and cannot (easily) be predicted by users.
DDC implements the wrong evaluation order (falling back to JavaScript evaluation order). It accidentally implements the correct order when doing dynamic calls.
DDC mostly maps method and getter calls to JavaScript method and getter calls. This means that it inherits JavaScript's left-to-right evaluation order. However, when doing dynamic calls, DDC evaluates the arguments before looking up the member (potentially invoking a getter):
// The dynamic send provides the already evaluated argument to the `dsend` function.
dart.dsend(o, 'foo', e1);
The specification's evaluation order has been written with implementation complexity in mind. The switch to a left-to-right order was negatively impacting the V8 developers. The Dart specification took this into account.
In a dynamic world it is easier to sequentially push the arguments onto the stack and then, at the end, evaluate the call target. With a left-to-right evaluation the call target must be kept alive while computing the arguments.
There are two developments that make the specification choice less interesting:
- Dart is getting more static. In most cases a method call is known not to go through getters or through
noSuchMethod. - (Non-)nullability will remove null-checks from method calls. An expression of the form
o.foo()is guaranteed not to have a null-error, sinceomust not benull.
This means that most method calls (of the form o.foo(e1)) are known to target a method (and not a getter), and that o cannot be null. In these cases the evaluation order doesn't have any effect (since the target resolution can't have any side-effects). As a result, I/we propose to switch to a left-to-right evaluation order in Dart.
This would have other nice side-effects. Without noSuchMethod the following invocations would be guaranteed to have the same semantics:
o.foo(e1)
(o.foo)(e1)
o.foo.call(e1)It would be nice, if noSuchMethod would also satisfy the previous equivalences. This section speculates (brainstorms) ways to get there.
I can see at least two ways to make this possible:
noSuchMethodtriggers for the member access separately (before it is known if it is a getter, setter, or method access).noSuchMethodonly works for known members, and produces an error otherwise.
In this scenario noSuchMethod only triggers for the member access. When triggered by a method call it would not receive the arguments, but would need to return a structure that then receives the arguments.
The structure could be something like:
/// When `noSuchMethod` is invoked an instance of the following structure
/// must be returned.
abstract class NoSuchMethodHandler {
factory NoSuchMethodHandler.method(Function f) { ... }
factory NoSuchMethodHandler.property(dynamic getter(), void setter(dynamic value));
factory NoSuchMethodHandler.proxy(
dynamic f(List<dynamic> positional, Map<Symbol, dynamic> named),
dynamic getter(),
void setter(dynamic value));
}
// Example usage:
class A {
noSuchMethod(Symbol member) {
switch (member) {
case #foo: return new NoSuchMethodHandler.method(print);
case #bar: return new NoSuchMethodHandler.property(() => 499, print);
default:
return new NoSuchMethodHandler.proxy(...);
}
}
}
main() {
new A().foo(); // \n
new A().foo("str"); // "str".
print(new A().bar); // 499
new A().bar = 42; // 42
new A().gee(1, 2); // ...
new A().gee = 499; // ...
var x = new A().gee; // ...Alternatively (but similar concept):
// Return either [FunctionHandler] or [PropertyHandler] from `noSuchMethod`.
abstract class FunctionHandler {
dynamic call(List positional, Map<Symbol, dynamic> named);
}
abstract class PropertyHandler {
get value;
set value(newValue);
}By providing factory constructors for functions and fields it would be easier to implement noSuchMethod for these common constructors. Currently, it is a common mistake to implement a method handler for a symbol, but not to provide a tear-off variant for the same symbol.
Another advantage of this approach is, that we could change the semantics so that implementations could permanently install these handlers after the first call. The VM could make invocations to them much faster (and potentially inline them).
For this approach, classes that use noSuchMethod would only be able to deal with members that the class supports. That is, members that it inherits (through implements, extends or with) or that it declares as abstract in its body.
For example,
class A {
void foo();
int gee;
}
class MockA implements A {
dynamic noSuchMethod(Invocation invocation) {
// Can only receive members of A.
}
}The idea would be that the compiler introduced forwarding members on its own:
// Conceptually, because of the `noSuchMethod`, the following class would be
// generated.
class MockA implements A {
void foo() => noSuchMethod(new _Invocation.method(#foo, []));
int get gee => noSuchMethod(new _Invocation.getter(#gee));
void set gee(int value) {
noSuchMethod(new _Invocation.setter(const Symbol("gee="), value));
}
dynamic noSuchMethod(Invocation invocation) {
// Original code.
}
}Clearly, the user could implement these methods by herself, but this code has the advantage that it can skip implementing lots of functions (if they are known not to be used), and the class stays resistant to new members (again, if the new members are known not to be used).
Probably we would want to disallow implementing clashing interfaces, since it's otherwise hard to generate a unique synthetic member. The following example (using the current noSuchMethod approach) would thus not be supported:
abstract class Interface1 { foo(x, y); }
abstract class Interface2 { foo({named}); }
class A implements Interface1, Interface2 {
noSuchMethod(Invocation invocation) {
if (invocation.positionalArguments.isEmpty) {
print(invocation.namedArguments[#named]);
} else {
print(invocation.positionalArguments.join(', '));
}
}
}
main() {
new A().foo(1, 2);
new A().foo(named: 3);
}Note that only supporting known interfaces is not as big of a restriction as one would think. Strong mode guarantees that all typed variables actually implement their known type. This is accomplished through is checks which look at the implemented interfaces...
Also note that the example is not warning free (and would probably not be error-free in strong mode):
Analyzing nsm.dart...
[warning] Inconsistent declarations of 'foo' are inherited from (dynamic, dynamic) → dynamic, ({named: dynamic}) → dynamic at nsm.dart:3:7 (inconsistent_method_inheritance).
[warning] 2 required argument(s) expected, but 0 found at nsm.dart:15:14 (not_enough_required_arguments).
[warning] The named parameter 'named' isn't defined at nsm.dart:15:15 (undefined_named_parameter).
3 warnings found.
If a method tear-off acts as if it has a noSuchMethod that forwards invalid #call invocations to the receiver, then calling a method will be equivalent to tearing it off and calling the tear-off.
class Foo {
noSuchMethod(...) { print("Gotcha!"); }
foo(int x. int y) { ... }
}
main() {
var foo = new Foo();
var tearOff = foo.foo;
tearOff("wrong argument"); // prints "Gotcha!";
}
// What the tear-off should act like:
class Foo$foo {
Foo _self;
noSuchMethod(invocation) {
if (invocation.memberName == #call) {
return _self.noSuchMethod(new _Invocation.method(#foo,
invocation.positionalArguments, invocation.namedArguments);
}
return super.noSuchMethod(invocation);
}
call(int x, int y) => _self.foo(x, y);
operator==(Object other) => other is Foo$foo && identical(_self, other._self);
}Lasse wrote a small summary of what is currently in the spec. This section is a copy/paste from his mail (with minor formatting changes):
TL;DR: We always evaluate arguments before trying to call a function, and before realizing that the receiver is null. We do call getters before evaluating arguments when calling a getter, except when the getter call is written as somethinggetter(args) - examples: instance.instanceGetter(args) (includes [this.]instanceGetter), Class.staticGetter(args) and prefix.libraryGetter(args). In those cases, we evaluate the getter after the arguments. The distinction is syntactical, not logical - calling a library or static getter as a single identifier (x(args)) is transformed into x.call(args) which evaluates the getter first.
There are three cases: unqualified invocation (x(args), 16.14.3), function expression invocation (e(args), 16.14.4), and ordinary method invocation (e.methodName(args), 16.17.1).
[[ and super invocations, but let's not complicate things :) ]]
The unqualified invocation depends on what x resolves to:
- local function, library (aka. top-level) function, library or static getter, or local variable
=> treat as function expression invocation, which is not a method extraction =>
e.call(args)as an ordinary method invocation. - static method of surrounding class
C=>C.x(args)(which is a member extraction, so it's a ordinary method invocation) - otherwise if in non-static context =>
this.x(args)(also directly an ordinary method invocation) - otherwise it's an error (static context and no declaration or a prefix name).
The function expression invocation, e(args), looks at the type of expression.
- If it's
e1.name(args)(eis a member extraction, 16.18), treat it directly as an ordinary method invocation. - Otherwise treat it as the ordinary method invocation
e.call(args).
So it all comes down to ordinary method invocation.
localFunction(args)=>localFunction.call(args)libraryFunction(args)=>libraryFunction.call(args)libraryGetter(args)=>libraryGetter.call(args)staticGetter(args)=>staticGetter.call(args)localVariable(args)=>localVariable.call(args)staticMethod(args)=>C.staticMethod(args)instanceUndefined(args)=>this.instanceUndefined(args)e.name(args)=>e.name(args)eOther(args)=>eOther.call(args)
So, everything is on the form e.name(args). These are all consistent in behavior:
- Evaluate
eto a valueo - Evaluate
argsto values vs - look up name on
o- either finds a getter, a method or nothing.- if a method (and its signature matches) call it! This is where "calling" actually happens.
- if getter, evaluate
Function.apply(o, vs-as-list-and-map)instead - which is basicallyo.call(vs), except that's not syntax. - otherwise call
o.noSuchMethod(invocation-of-name-and-args). (lookup is fancy and handles prefix objects and static lookup on Type objects).
So, we always evaluate arguments before trying to call, and before realizing that the receiver is null, if a getter returns null.
Methods are never null, at least.
Now, for fun:
class Nyah {
final int n;
final value;
Nyah(this.n, this.value);
get call => n >= 0 ? new Nyah(n - 1, value) : () => value;
}
main() {
print(new Nyah(5, 42)());
}The following program tests a few evaluation-order cases
typedef void F(int x);
/*
All the double-returns in functions are to prevent dart2js from
inlining these functions.
*/
dynamic get funGetter {
print("eval funGetter");
return (arg1) { print("funGetter"); };
return null;
}
F get typedFunGetter {
print("eval typedFunGetter");
return (arg1) { print("typedFunGetter"); };
return null;
}
class A {
void memberFun(int x) {
print("memberFun");
return;
return;
}
F get memberGetter {
print("eval memberGetter");
return (arg1) { print("memberGetter"); };
return null;
}
}
int bar([str]) {
print("bar $str");
return 499;
return 499;
}
confuse(x) {
return x;
return x;
}
main() {
A a = new A();
confuse(false);
A aNull = confuse(true) ? null : new A();
dynamic dynNull = confuse(true) ? null : new A();
print("funGetter-----");
funGetter(bar(1));
print("typedFunGetter-----");
typedFunGetter(bar(2));
print("memberFun-----");
a.memberFun(bar(3));
print("memberGetter-----");
a.memberGetter(bar(4));
try {
print("null.memberFun-----");
aNull.memberFun(bar(5));
} catch (e) {}
try {
print("null.memberGetter-----");
aNull.memberGetter(bar(6));
} catch (e) {}
try {
print("dyn.memberFun-----");
dynNull.memberFun(bar(7));
} catch (e) {}
try {
print("dyn.memberGetter-----");
dynNull.memberGetter(bar(8));
} catch (e) {}
}VM:
funGetter-----
eval funGetter
bar 1
funGetter
typedFunGetter-----
eval typedFunGetter
bar 2
typedFunGetter
memberFun-----
bar 3
memberFun
memberGetter-----
bar 4
eval memberGetter
memberGetter
null.memberFun-----
bar 5
null.memberGetter-----
bar 6
dyn.memberFun-----
bar 7
dyn.memberGetter-----
bar 8
dart2js:
funGetter-----
eval funGetter
bar 1
funGetter
typedFunGetter-----
eval typedFunGetter
bar 2
typedFunGetter
memberFun-----
bar 3
memberFun
memberGetter-----
bar 4
eval memberGetter
memberGetter
null.memberFun-----
null.memberGetter-----
dyn.memberFun-----
dyn.memberGetter-----
dart2js (without the inlining obstacles):
funGetter-----
eval funGetter
bar 1
funGetter
typedFunGetter-----
eval typedFunGetter
bar 2
typedFunGetter
memberFun-----
bar 3
memberFun
memberGetter-----
bar 4
eval memberGetter
memberGetter
null.memberFun-----
bar 5
null.memberGetter-----
bar 6
dyn.memberFun-----
bar 7
dyn.memberGetter-----
bar 8
DDC:
funGetter-----
eval funGetter
bar 1
funGetter
typedFunGetter-----
eval typedFunGetter
bar 2
typedFunGetter
memberFun-----
bar 3
memberFun
memberGetter-----
eval memberGetter
bar 4
memberGetter
null.memberFun-----
null.memberGetter-----
dyn.memberFun-----
bar 7
dyn.memberGetter-----
bar 8