Analysis of current noSuchMethod situation

no-such-method-analysis.js

// by Dmitry Soshnikov <dmitry.soshnikov@gmail.com>
// on "noSuchMethod" hook via proxies
// written on December 15, 2011

// Below is tl;dr
// but if you want really never-ending tl;dr, try this
// original old thread from 2010 on es-discuss:
// https://mail.mozilla.org/pipermail/es-discuss/2010-October/011929.html

// Small disclaimer and note:

    // While I was writing this small article, I more and more started to
    // agree on importance of the "extracted funargs" in this case.

// =======================================================================================
// 1. What is noSuchMethod?
// ---------------------------------------------------------------------------------------

// Why noSuchMethod? What is noSuchMethod?

// noSuchMethod is just a *reaction*. The reaction on the *event* of
// missed method on the object (including its prototype chain)

// Desugarred noSuchMethod can look like this:

try {
  foo.bar.apply(args);
} catch (e) {
  handleNoSuchMethodEvent(foo, "bar", args);
}

// or like this:

if (typeof foo.baz != "function") {
  handleNoSuchMethodEvent(foo, "baz", args);
} else {
  foo.bar.apply(args);
}

// etc.

// Obviously, for a better code-reuse, in order not to repeat
// every time this try-handleNoSuchMethodEvent, if-handleNoSuchMethodEvent, ...
// a *special hook* can be triggered *behind the scene*.

// Note once again the main thing -- it's just a *hook* for *the event*!

var foo = {
  x: 100,
  __noSuchMethod__: function (method, args) {
    console.log(method, args);
  }
}

foo.bar(1, 2, 3);

// can desugar to the same:

if (typeof foo.bar != "function") {
  if (typeof foo.__noSuchMethod__ == "function") {
    foo.__noSuchMethod__("bar", [1, 2, 3]);
  }
} else {
  foo.bar(1, 2, 3);
}

// =======================================================================================
// 3. Missing methods "should" be "funargs".
// ---------------------------------------------------------------------------------------

// In case you already forgot it -- noSuchMethod is just the *reaction*
// on the *event* of *missing a method* on the object.

// IT'S NOT ABOUT THE METHOD! There is no any method!

// So the proposal to handle it *as it would be a method*,
// seems already as just a broken logic!

// The main argument of Tom and Brendan is that a user needs
// the ability to apply these missed methods as functional objects ("funargs").

// It means, the user may *extract* the method to a variable (a casual optimization
// practice in everyday JS programming) and later to *apply* this "funarg":

// In other words, this:

foo.bar(1, 2, 3);

// SHOULD be equivalent to the:

var bar = foo.bar;
bar.apply(foo, [1, 2, 3]);

// Right. It should. But only for *existing methods*.

// Why? Because in case of noSuchMethod we work with just the *fact* of missing a method.
// NOT WITH THE METHOD ITSELF.

// Of course, "invoke-only" methods (we called them also as "phantoms"), but not "funargs"
// CAN bring disadvantage and even *confusion* for the programmer.

// For example, if a programmer sees the following JS code:

db.findByName("Brendan");

// then she has the complete right to think about the "findByName" method as about the
// real method on the "db" object. Moreover, it's not even possible to say looking only
// on this line that the method is "virtual".

// (by the way, I took this example from "Ruby on Rails" framework where these virtual methods
// such as "find_by_<fieldName>" are widely spread and useful)

// So, yes, I agree in here with Tom and Brendan, that *potentially* the user
// CAN and has the complete right to transform this code into:

db.findByName.apply(db2, ["Brendan"]);

// or even (again for the optimization) into the:

var findByName = db.findByName;
findByName.apply(db2, ["Brendan"]);

// =======================================================================================
// 3. And what about implementation?
// ---------------------------------------------------------------------------------------

// Proxies are used to imitate noSuchMethod.

// For that Tom proposes (and Brendan just supports it) to make
// all *non-existing* properties as existing!

// It's achieved with just returning a function for every missing property and this
// function when is called, already calls noSuchMethod hook:

// (Note, I used "direct proxies" format; see the strawman
// http://wiki.ecmascript.org/doku.php?id=harmony:direct_proxies)

var foo = Proxy({}, {
  get: function (target, property) {
    if (property in target) {
      return target[property];
    }
    return function () {
      var args = [].slice.call(arguments, 0);
      return noSuchMethod(target, property, args);
    };
  }
});

// Having "noSuchMethod" handler it works

function noSuchMethod(object, property, args) {
  console.log("noSuchMethod", property, args);
}

foo.bar(1, 2, 3); // noSuchMethod: "bar" [1, 2, 3]

// =======================================================================================
// 4. Broken invariants.
// ---------------------------------------------------------------------------------------

// However this logical part becomes absolutely broken if to consider the implementation's
// *consequences*.

// Having fixed one "broken invariant" (with "invoke-only-phantoms" which now are "correctly"
// normal JS "funargs"), the implementation just broke *many other logical invariants*

// ---------------------------------------------------------------------------------------
// - 4.1 Every "non-existing" property is a strange function?

// Do you remember code like this?

if (!Array.isArray) {
  Array.isArray = function () {
    // custom implementation
  };
}

// Now imagine this "Array", OR IN ESSENCE ANY OBJECT IN THE LIB CODE, is the proxy with
// the noSuchMethod implementation from above.

// First of all if-branch isn't executed, since the "Array.isArray" exists. OK, seems it's a
// modern engine and it has native support of "Array.isArray", let's use it then.

Array.isArray(foo); // noSuchMethod: "isArray" [foo]

// Wait a second ...

Array.whatA ..
Array.hey
Array.whatTheHeckIsGoingOnHere

// Ah? All of them DO EXIST and are some *strange functions*... WHAT is that at all?

// ---------------------------------------------------------------------------------------
// - 4.2 A property is nevery equal to itself

// Just stop right there! What's going on in below:

Array.isArray === Array.isArray; // false!

// What?! What's going on in this language at all?

// Yes, of course even a *simple getter* may cause *side effects* and the equality will be false,
// so we may forgive this broken invariant.

// Moreover, if to *cache* the missed method, the equality will start to work:

var cache = {};
var foo = Proxy({}, {
  get: function (target, property) {
    if (property in target) {
      return target[property];
    }
    if (cache[property]) {
      return cache[property];
    }
    return (cache[property] = function () {
      var args = [].slice.call(arguments, 0);
      return noSuchMethod(target, property, args);
    });
  }
});

// But notice, then you have to *invalidate the cache* in the delete trap!
// OTOH, what are you trying to delete? Haven't you already understood?
// Properties are non-deletable!

// ---------------------------------------------------------------------------------------
// - 4.3 Properties are non-deletable

// Come on, give it a try

delete foo.bar;

if (foo.bar) {
  console.log("Trollface ;D"); // Ah?
  console.log(foo.bar); // I'm still a function! ;D
}

// However, this invariant may become not so broken since we can't delete
// e.g. *non-configurable* properties as well. So we may treat these virtual
// missed properties as non-configurable and return `false` on `delete` operation.
// We also should adjust `getOwnPropertyDescriptor` hook.

// The implementation becomes the following:

// import default forwarding handler
module Reflect from "@reflect";

// activators cache
var cache = {};

// gets the activator of noSuchMethod
// and chaches it for the next calls
function getActivator(target, property) {
  if (cache[property]) {
    return cache[property];
  }
  return (cache[property] = function () {
    var args = [].slice.call(arguments, 0);
    return noSuchMethod(target, property, args);
  });
}

var foo = Proxy({}, {
  get: function (target, property) {
    if (property in target) {
      return target[property];
    }
    return getActivator(target, property);
  },
  delete: function (target, property) {
    if (!target.hasOwnProperty(property)) {
      return false;
    }
    // else default behavior
    return Reflect.delete(target, property);
  },
  getOwnPropertyDescriptor: function (target, property) {
    if (!target.hasOwnProperty(property)) {

      // TODO: handle Object.defineProperty invariant;
      // Potentially, a user can change some attribues via Object.defineProperty,
      // which creates the property on the object if it doesn't exist yet, but for now
      // just return descriptor unconditionally.
      // Though, if our virtual property is non-configurable, logically it means
      // user can't redefine the property.

      // TODO: fix the issue "cannot report a non-configurable descriptor for
      // non-existent property" in DirectProxies

      return {
        // adjust value to our missed methods
        value: getActivator(target, property),
        // it's always non-configurable to support `delete' invariant
        configurable: false,
        writable: true,
        enumerable: true,
      };
    }
    // else default behavior
    return Reflect.getOwnPropertyDescriptor(target, property);
  }
});

// (By the way, for the optimizations, we may even use single *activator* function, which
// executes "noSuchMethod", however in this case foo.bar === foo.qux. My old implementation:
// https://github.com/DmitrySoshnikov/es-laboratory/blob/master/examples/noSuchMethod.js)

// ---------------------------------------------------------------------------------------
// - 4.4 Broken "in" operator:

// OK, let's assume that I don't have missed properties at all and all of them are functions

// but...

if (foo.bar) {
  // exists
}

// ... what's happened here again?

"bar" in foo; // false! Torllface again ;D

// Right, "has" trap should be adjusted as well to return always `true'!

var foo = Proxy({}, {
  get: function (target, property) {
    if (property in target) {
      return target[property];
    }
    return getActivator(target, property);
  },
  has: function (target, property) {
    return true; // ;D
  },
  delete: function (target, property) {
    // handle delete
  },
  getOwnPropertyDescriptor: function (target, property) {
    // handle getOwnPropertyDescriptor
  }
});

"bar" in foo; // true :)

// (This is nearly the current Tom's propsal:
// https://mail.mozilla.org/pipermail/es-discuss/2011-December/018834.html

// =======================================================================================
// 5. OK, OK, so what do you propose then?
// ---------------------------------------------------------------------------------------

// That's said, I completely agree that a user CAN and HAS THE COMPLETE RIGHT to call a function
// as an "extracted funarg". Since, once again, it's *not possible to distinquish* a real
// method from the virtual one by just looking on the line of a code.

// Moreover, if to consider the same "Ruby on Rails" and its examples with "find_by_<fieldName>"
// virtual methods, then they are virtual *only until the fist catch*. Once they are caught
// by the "method_missing" (Ruby's analog of "noSuchMethod"), the *real method is created* on the
// objects class (prototype) and later calls are already handled directly!

// In terms of JS, if we do the same and will create methods *lazily*, it means at first call
// we can't extract the function as a "funarg", and in later we already can!

// So unfortunately, we can't support both cases -- to have missed methods as funargs and
// at the same time to keep all the invariants logical and correct. We have to get the compromise.

// And the compromise is to have noSuchMethod *in addition* to the scheme above.

// This will give us the ability to react in both manners. As described above in Tom's proposal,
// as well as *just to handle the fact* of a missing method.

// Thus, some properties which we want to have as "funargs" we return from `get' (from this viewpoint
// it doesn't differs much from a *simple getter* which returns a function!) and other just
// handle in the "noSuchMethod":

var foo = Proxy({}, {
  get: function (target, property) {
    if (property in target) {
      return target[property];
    }
    // special methods we want to have as "funargs"
    if (property.startsWith("findBy")) {
      if (cache[property]) {
        return cache[property];
      }
      return (cache[property] = function () {
        var args = [].slice.call(arguments, 0);
        return noSuchMethod(target, property, args);
      })
    }
  },
  has: function (target, property) {
    if (property.startsWith("findBy")) {
      return true; // ;D
    }
    return Reflect.has(target, property);
  },
  delete: function (target, property) {
    // handle delete
  },
  getOwnPropertyDescriptor: function (target, property) {
    // handle getOwnPropertyDescriptor
  }
  noSuchMethod: function (method, args) {
    console.log("Virtual handler", method, args);
  }
});

// How much difference and sense will it make?
// Do we need it?

// The question is still open. We may either implement it or not.
// It's just the question of the design decision.

// P.S.:

// Once again: currently the importance of the "extracted funargs" in this case
// seems for me bigger. So it's still really hard to answer the question completely,
// whether we need "invoke-only-phantoms" handler a.k.a "noSuchMethod" or should we
// use it via "no-missing-properties-and-broken-delete-but-all-are-funargs" variant?

(In case this would be implemented) why noSuchMethod instead of methodMissing?

Because Mozilla already implemented noSuchMethod.

Except it's non-standard and shouldn't be really used.

@paulmillr

Yes, NV already replied from where the roots came. And regarding on that it's non-standard, the talk is exactly about to standardize it for the proxy handlers.

Crazy, immediate, not well though of solution to this may be new typeof "phantom" which will ToBoolean to false, ToNumber to NaN and is callable. Of course, it is better to prune those that I know beforehand for sure that they will not / will be there and return null / them directly. But for "I don't know, you should invoke" a phantom wrapper around a function could work. Maybe.
EDIT: Well, it cannot be "phantom" since it is callable... or maybe it can, since it is not object... well, not thought of, as I said.

Good writeup, Dmitry.

One comment on point 4.1: I do not see the fact that |proxy.missingMethod| returns a function rather than undefined as a "broken invariant". Consider your db object and the findBy... methods. If I have code like:

if (!db.findByName) {
db.findByName = ...;
}

Then I would not want to override db's findByName method if db.findByName("...") will work just fine.
This pattern is normally used to fill in missing methods, but if a proxy defines a "noSuchMethod"-like behavior, then, to its clients, it has no more missing methods, so it needs no monkey-patching.

@DmitryShosnikov, it would probably be better not to use the term 'funarg' here, since it has a long history of meaning something totally different, relating to passing functions to other functions or returning functions from functions.

@herby

typeof "phantom" which will ToBoolean to false, ToNumber to NaN

Interesting idea, but this will make this "creatures" as something existing anyway. And in "noSuchMethod is a hook" position, it's really just a hook to handle the situation of missing method invocation. It doesn't state anything about the method itself, and doesn't assume the method exists, but vice-versa.

@tvcutsem

This pattern is normally used to fill in missing methods, but if a proxy defines a "noSuchMethod"-like behavior, then, to its clients, it has no more missing methods, so it needs no monkey-patching.

Yes, it's true, but only in this scheme when we treat this missing methods as something existing. In case of "noSuchMethod is just a hook" case, methods do not exist.

Anyway, that's said, a programmer has the complete right to rewrite the single line of a code in the manner she wants. And since in JS she can rewrite it with using apply: foo.bar(1) -> foo.bar.apply(foo, [1]), etc when sees the invocation, then not having the method as "real method" (even if it's missed), may cause the confusion for her.

I mean, this is the only argument -- "the code should be predictable for the programmer" which may justify somehow broken invariants. But at the same time, the code won't be predicable in respect of those broken invariants (with delete e.g., etc).

@samth

it would probably be better not to use the term 'funarg' here, since it has a long history of meaning something totally different, relating to passing functions to other functions or returning functions from functions.

Thanks, yeah, I'm aware about what the term does mean (and actually this is me who brought it in today talks).

Speaking on history, a "funarg" (a "functional argument") is the term which relates to the research of static (lexical) scope and inventions of closures. And today "funarg" is just obsolete term for a "closure". In the article on closure, describing closures from the theoretical viewpoint I used this term to underline the historical roots.

Regarding this topic, we just used "funarg" in the previous thread on es-discuss. Brendan just used it to describe the fact that functions are first-class and can be extracted and saved to variables or extracted and directly passed to/returned from fun-ctions as arg-uments.

Hey Dmitry,
perhaps it's just me but I don't understand in your final proposal, why not to use the else branch of the get-handler for missing methods?

get: function (target, property) {
  if (property in target) {
    return target[property];
  }
  // special methods we want to have as "funargs"
  if (property.startsWith("findBy")) {
    if (cache[property]) {
      return cache[property];
    }
    return (cache[property] = function () {
      var args = [].slice.call(arguments, 0);
      return noSuchMethod(target, property, args);
    })
  }

  // property really is missing:
  return function () {
    var args = [].slice.call(arguments, 0);
    return noSuchMethod(target, property, args);
  };
}

Thank you in advance,

Jonathan

EDIT: I think I now got it: You only want to react on the missing-event, not create methods itself. Then again, I don't understand why the caller should try to invoke a not-existing method. Or to put it the other way around, why the problem should not be handled on calling-side. Coffeescript introduced the question-mark-operator to allow a convenient check for existance.

obj.myMethod() if obj.myMethod? 

An alternative could be to provide such an operator for inline-checking of method-existence desugaring to something like:

if(typeof obj.myMethod === 'function')
  obj.myMethod();

@b-studios

why not to use the else branch of the get-handler for missing methods?

It's just a style of control-flow exit. Since every if-branch contains its own return there is no big need in else. Viewing the code top-down, we may analyze it by returns -- if there is a return, potentially we may skip further code analysis below.

Then again, I don't understand why the caller should try to invoke a not-existing method. Or to put it the other way around, why the problem should not be handled on calling-side.

Fair enough, though the main idea of this system signal of missing a method (noSuchMethod, method_missing, doesNotUnderstand, __call, etc -- it has different names in different languages), is to handle it nevertheless behind the scene as it would be the case like the method does exist.

A good example is again "Ruby on Rails" and its find_by_<fieldName> virtual methods. You of course may handle it on the callee-side with manual checks (optimized as with CoffeeScrip, or just using if), but it brings much of "syntactic noise" if to make these checks every time explicitly.

Of course "explicit is better than implicit" ideology (from Python) is good, but we also should remember, that "too much explicit" is often equals to "syntactically noisy".

obj.myMethod() if obj.myMethod?

Yes, and this already another topic. Such checks can be useful, and I proposed this existing operator for ECMAScript. Unfortunately, no result yet, since there are issues with parser ambiguities.

@DmitryShosnikov

Sorry for my ambiguous writing. I am aware of the control-flow exit. What I initially wanted to express is the following:

As far as I understood there are three types of methods:

1. Those which are already defined in target property in target
2. Those we want to create lazily as funargs property.startsWith("findBy")
3. Those which really don't exist, throwing exception noSuchMethod

I think adding a handler to catch the third case is pretty reasonable.
But the virtual methods that are added this way (e.g. in a framwork) may become confusing for a programmer, who has to work with this API. He may not be able to tell the difference between virtual methods and lazy funargs: "Ok, sometimes methods are created - sometimes not"

To prevent such complexity I suggested the explicit coffescript-like calling alternative.

I think the source for this problems is that the concept of message-passing do not completely apply to JavaScript. DoesNotUnderstand works perfectly for languages, which don't differ between property-access
and method-invocation (e.g. I'm thinking of Smalltalk and Ruby) In those languages one signal noSuchMethod is enough to handle all cases of missing attributes, methods and so on. (I hope I got this one right)

What I want to say is: With JavaScript being not only founded on message-passing maybe it
is not possible to find a transparent solution to solve this problem. And in that case I personally
would go with simplicity and stay with the proxies only. But that is only my personal liking.

Kind regards

Jonathan

@b-studios

Ah, I got it.

Yes, this is all about "message passing" and Smalltalk's #doesNotUnderstand and Ruby's method_missing are about it. In JS actually, technically there are only properties, and these properties are associated with some values.

In other words, in Ruby foo.bar is already calling a method (sending a message), but in JS it's accessing a property.

So yes, having just properties via proxies may simplify all this. The only thing I want is to reduce broken invariants (such as "everything is a function", broken delete, etc).

Though, e.g. broken delete may be not so broken, since the same situation we have with non-configurable properties.