Private Symbol presentation response

private-response.md

• Maybe some people see a one-or-the-other choice, but I see a spectrum of possibilities. And, I think each choice this proposal contains was well-thought-through and justified. We're not talking about "private symbols" as a first-class thing, but rather some subtle semantic changes for reading, writing and defining private fields and methods. The idea to use WeakMap-analogous semantics was partly motivated by a sort of impossible-to-refute logic about not introducing additional things when WeakMap exists, but let's step back from that a little bit and think about what would be best for JavaScript programmers with this as its own feature. In this presentation, I summarize the semantic choices for private fields and methods, which might be different if we adopted private symbol semantics, and the motivations in the committee (not necessarily all goals I share personally) for the semantics being the way they are. If we're revising the semantics, let's agree that we have consensus on these modifications to design goals.
  • Search up the prototype chain on read of a private field
    • This is what might provide memory savings for private methods in V8's hidden class representation, so that the existing prototype chain can be used, rather than needing a separate representation of the list of private brands in the hidden classes (since these can get out of sync). And other JS engines have a broadly analogous hidden class representation to V8's. (More notes at #implementation)
    • Observable impact: [[GetPrototypeOf]] is called--observable if a Proxy is hit in the prototype chain
    • Design goals this meets:
      • Observability: If you refactor code between, e.g., private methods and lexically scoped functions, there is no observable impact ever
      • Defensibility: There should be no way to make a change to the instance which messes with the operation of internal private method calls, whether that's causing an error in case which previously didn't have an error, or removing an error from a previous error path. The ability to dynamically mutate the prototype (either of the instance directly, or higher up in the prototype chain) enables these changes in both directions.
      • (+ analogy to WeakMaps)
    • Note, the above goals are about observability and defensibility with respect to a particular instance; to enable techniques to make the class hierarchy more defensible/less observable for instantiation (where the current state is a consequence of how ES6 classes work, but doesn't necessarily indicate that future things should be not-defensible-by-default), see #invariants below
  • Proxy transparency: private fields/methods on the proxy vs on the target
    • Note: Third option (throw an error when adding private on Proxy) not available, as this would give a way to test if an object is a Proxy, contradicting a Proxy design goal
    • Design goals this meets:
      • It is possible to basically fully encapsulate the target using a Proxy; you can make a Proxy that leaks nothing unintended (maybe not so important if "everybody" uses shadow targets, but then what's the point of the target anyway?)
      • Private field semantics are analogous to internal slot semantics, which don't forward to Proxy targets
      • (+ analogy to WeakMaps)
  • Checking [[IsExtensible]] before adding private fields or methods (not sure if you're proposing this)
    • Note, adding private fields to already-"born" objects diverges from internal slots, but this seems to be the only way to let subclasses declare private fields and work coherently with ES6's dynamic prototype chain semantics without doing two passes up and down the prototype chain on instantiation
    • Design goals this meets:
      • Defensibility: A consumer of an instance can call Object.preventExtensions on it, preventing the "normal" operation of the class (which might add more private fields or methods later, even if that's an anti-pattern)
      • (Observability not really an issue, as the constructor would have to choose to return a Proxy that does the observation)
      • (+ analogy to WeakMaps)
  • Errors vs undefined for reading missing private field (not sure if you're proposing this)
    • Design goal this meets:
      • Make it easier to catch programming errors
      • Provide a brand checking mechanism analogous to how built-in internal slots are used
    • (No analogy to WeakMaps)
  • Errors on adding private fields outside of declarations (not sure if you're proposing this; also I don't know whether you'd propose that these writes go up the prototype chain)
    • Design goal this meets:
      • Encourage a relatively fixed shape, which is more optimizable (even if there are map changes during instantiation, it should be helpful to have those mostly settled down by the time instantiation is done).
      • Define private fields up-front, in a way that's analogous to internal slots (analogy only holds for base classes, strictly speaking)
      • Be analogous for brand checks for reads to private fields
    • (No analogy to WeakMaps)
  • (Note, as far as reification goes: the PrivateName draft meets "integrity" goals that Symbol does not, since it was deemed especially important that PrivateName have this integrity. But I'm also happy to leave this aside, for the decorators proposal.)
• Note, it's possible that we have an issue of misunderstanding, if even TC39 delegates are getting confused about what invariants there are. Here are the invariants I'm aware of: (#invariants)
  • If an instance has the private methods, that means that it started to run that particular (sub)class constructor, no more, no less
  • This property can't be spoofed through Proxy wrapping prototype chain manipulation.
  • You can build higher-level guarantees on top of this, e.g., if you freeze the constructor, and the constructor calls super(), you can show that the super constructor was also executed. But, this is a very conditional guarantee because of decisions we made in ES6 classes.
  • To address the documentation issue, Neil Kakkar is working on MDN documentation for class fields and private methods (and the educators group is working on documentation for many proposals); we'll try to clarify (but are currently working through the basics). Probably we should also update the explainer (do before meeting!)
  • Documentation and education is the only comprehensive way to prevent this sort of "confused deputy problem"--it's always possible that someone could develop incorrect beliefs about how the language works, no matter what we decide. We can just do our best to choose something which, subjectively, is relatively intuitive and useful.
  • To the concrete issue of the original prototype chain not being taken, see the class.initialize proposal and Object.setImmutablePrototype proposal (both forthcoming)
• Implementation thoughts (#implementation) -- not clear how much time to spend on this point in committee
  • Design goals:
    • Private fields and methods should be (at least) as efficient to read/write/call as their public equivalents
    • There should not be per-method, per-instance space overhead for private methods (e.g., as a naive private field representation for private methods would have)
    • Implementations should not be extremely, prohibitively complex; ideally, machinery for public properties can be reused
    • These goals should hold in various cases, to avoid time/space cliffs:
      • In the interpreter/baseline compiler as well as in the optimizing compiler
      • In warmed ICs as well as the first time code runs at all
      • In objects with a shared, stable hidden class, as well as objects which degrade into hashtable-mode
  • If brand checks prove expensive to implement for private methods, and we're comfortable with it, let's consider just removing the checks, as originally proposed (link to spec change, and rebase it). This makes more sense to me than adding all these things that can be observed and manipulated from outside of the class.
  • However, we're working on the implementation in V8 and JSC, and working through any implementation difficulties. We're also in touch with the SpiderMonkey team, which has a pretty analogous, optimistic plan about how to handle this implementation.

• Maybe some people see a one-or-the-other choice, but I see a spectrum of possibilities. And, I think each choice this proposal contains was well-thought-through and justified. We're not talking about "private symbols" as a first-class thing, but rather some subtle semantic changes for reading, writing and defining private fields and methods.

The idea to use WeakMap-analogous semantics was partly motivated by a sort of impossible-to-refute logic about not introducing additional things when WeakMap exists,

It's possible to refuse it, because WeakMap doesn't provide full set of features that Symbol.private has. Don't you agree with it?

but let's step back from that a little bit and think about what would be best for JavaScript programmers with this as its own feature. In this presentation, I summarize the semantic choices for private fields and methods, which might be different if we adopted private symbol semantics, and the motivations in the committee (not necessarily all goals I share personally) for the semantics being the way they are. If we're revising the semantics, let's agree that we have consensus on these modifications to design goals.

I'm not sure that I understand the goal of this list. Could you please clarify it somehow?

• Search up the prototype chain on read of a private field
  • This is what might provide memory savings for private methods in V8's hidden class representation, so that the existing prototype chain can be used, rather than needing a separate representation of the list of private brands in the hidden classes (since these can get out of sync). And other JS engines have a broadly analogous hidden class representation to V8's. (More notes at #implementation)

Should it be considered as advantage of Symbol.private approach comparing to WeakMap-like?

• Observable impact: [[GetPrototypeOf]] is called--observable if a Proxy is hit in the prototype chain

It seems like it already works like this. Are there are any proposals that violate this stuff?

• Design goals this meets:
  • Observability: If you refactor code between, e.g., private methods and lexically scoped functions, there is no observable impact ever

Seems like Symbol.private approach serves this purpose better than WeakMap-like. Consider following code:

const A = (function () {
    function closureHiddenFn() {}

    return class ClassA {
        publicMethod() {
            closureHiddenFn();
        }
    }
})();
const a = new A();
const proxiedA = Proxy(a, {});
proxiedA.publicMethod(); // just works

rewritten to

class A {
    #privateMethod = () => {};
    publicMethod() {
        this.#privateMethod;
    }
};
const a = new A();
const proxiedA = Proxy(a, {});
// if `#privateMethod` is `Symbol.private` such code just works
// but if it has `WeakMap`-like semantic (e.g. existing proposal) it throws
proxiedA.publicMethod();

• Defensibility: There should be no way to make a change to the instance which messes with the operation of internal private method calls, whether that's causing an error in case which previously didn't have an error, or removing an error from a previous error path. The ability to dynamically mutate the prototype (either of the instance directly, or higher up in the prototype chain) enables these changes in both directions.

It seems that both Symbol.private and WeakMap-like approaches satisfy this statement/requirement.

• (+ analogy to WeakMaps)

Could you please clarify meaning of this quote?

• Note, the above goals are about observability and defensibility with respect to a particular instance; to enable techniques to make the class hierarchy more defensible/less observable for instantiation (where the current state is a consequence of how ES6 classes work, but doesn't necessarily indicate that future things should be not-defensible-by-default), see #invariants below

• Proxy transparency: private fields/methods on the proxy vs on the target
  • Note: Third option (throw an error when adding private on Proxy) not available, as this would give a way to test if an object is a Proxy, contradicting a Proxy design goal
  • Design goals this meets:
    • It is possible to basically fully encapsulate the target using a Proxy; you can make a Proxy that leaks nothing unintended (maybe not so important if "everybody" uses shadow targets, but then what's the point of the target anyway?)

Both Symbol.private and WeakMap-like approaches gives us an opportunity to build proper Membrane.

• Private field semantics are analogous to internal slot semantics, which don't forward to Proxy targets

Why it should be like so? What problem is solved by such restriction? Do, you understand, that such restriction creates other problems?

• (+ analogy to WeakMaps)

Could you please clarify meaning of this quote?

• Checking [[IsExtensible]] before adding private fields or methods (not sure if you're proposing this)
  • Note, adding private fields to already-"born" objects diverges from internal slots, but this seems to be the only way to let subclasses declare private fields and work coherently with ES6's dynamic prototype chain semantics without doing two passes up and down the prototype chain on instantiation
  • Design goals this meets:
    • Defensibility: A consumer of an instance can call Object.preventExtensions on it, preventing the "normal" operation of the class (which might add more private fields or methods later, even if that's an anti-pattern)

Do you mean, that [[IsExtensible]] shouldn't be called before adding private property, so it could appear on frozen object later?

• (Observability not really an issue, as the constructor would have to choose to return a Proxy that does the observation)

• (+ analogy to WeakMaps)

Could you please clarify meaning of this quote?

• Errors vs undefined for reading missing private field (not sure if you're proposing this)
  • Design goal this meets:
    • Make it easier to catch programming errors

Throwing an Error instead of returning undefined IMPLICITLY adds brand-checks to ALL private properties accesses and prevents only one tiny set of typos (I can't even call them bugs), like:

class A {
    #field = 1;
    method() {
        return this.#fild;
    }
}

Do you understand that existing tooling (e.g. TypeScript, Flow and linters) prevents such typos much better (also adding a whole bench of other usefulness) than ES will ever be able to (unless it introduces type checking system)?

• Provide a brand checking mechanism analogous to how built-in internal slots are used

Why do you want to add IMPLICIT brand-check which already causes issues for proxies, when there is very convenient way of EXPLICIT brand-check with Symbol.private approach, that will be used only in cases where it's really necessary? And obviously Symbol.private could be used for built-in internal slots with brand-check added if it's needed.

• (No analogy to WeakMaps)

What analogies to WeakMaps you want to add everywhere in the list?

• Errors on adding private fields outside of declarations (not sure if you're proposing this; also I don't know whether you'd propose that these writes go up the prototype chain)
  • Design goal this meets:
    • Encourage a relatively fixed shape, which is more optimizable (even if there are map changes during instantiation, it should be helpful to have those mostly settled down by the time instantiation is done).
    • Define private fields up-front, in a way that's analogous to internal slots (analogy only holds for base classes, strictly speaking)
    • Be analogous for brand checks for reads to private fields
  • (No analogy to WeakMaps)

Why do you prefer very small optimization over the huge set of really useful patterns that can replace a lot of WeakMap usages with something that much more optimizated in terms of memory and GC, has much more ergonomic syntax and proxy-safe?

• (Note, as far as reification goes: the PrivateName draft meets "integrity" goals that Symbol does not, since it was deemed especially important that PrivateName have this integrity. But I'm also happy to leave this aside, for the decorators proposal.)

What does integrity mean here?

• Note, it's possible that we have an issue of misunderstanding, if even TC39 delegates are getting confused about what invariants there are. Here are the invariants I'm aware of: (#invariants)
  • If an instance has the private methods, that means that it started to run that particular (sub)class constructor, no more, no less

So implicit brand-check again.

• This property can't be spoofed through Proxy wrapping prototype chain manipulation.

True for both Symbol.private and WeakMap-like approaches.

• You can build higher-level guarantees on top of this, e.g., if you freeze the constructor, and the constructor calls super(), you can show that the super constructor was also executed. But, this is a very conditional guarantee because of decisions we made in ES6 classes.

How does it reate to class-fields at all?

• To address the documentation issue, Neil Kakkar is working on MDN documentation for class fields and private methods (and the educators group is working on documentation for many proposals); we'll try to clarify (but are currently working through the basics). Probably we should also update the explainer (do before meeting!)
• Documentation and education is the only comprehensive way to prevent this sort of "confused deputy problem"--it's always possible that someone could develop incorrect beliefs about how the language works, no matter what we decide. We can just do our best to choose something which, subjectively, is relatively intuitive and useful.
• To the concrete issue of the original prototype chain not being taken, see the class.initialize proposal and Object.setImmutablePrototype proposal (both forthcoming)

Ok, it seems to be irrelevant to my topic of interest (Symbol.private vs WeakMap-like semantic), or did I misunderstood something?

• Implementation thoughts (#implementation) -- not clear how much time to spend on this point in committee
  • Design goals:
    • Private fields and methods should be (at least) as efficient to read/write/call as their public equivalents
    • There should not be per-method, per-instance space overhead for private methods (e.g., as a naive private field representation for private methods would have)
    • Implementations should not be extremely, prohibitively complex; ideally, machinery for public properties can be reused
    • These goals should hold in various cases, to avoid time/space cliffs:
      • In the interpreter/baseline compiler as well as in the optimizing compiler
      • In warmed ICs as well as the first time code runs at all
      • In objects with a shared, stable hidden class, as well as objects which degrade into hashtable-mode

As far as I understand both approaches satisfy this goals, because nobody raised such issues for existing proposal and Symbol.private will just easily reuse majority (if not all) of optimization that already there for public properties.

• If brand checks prove expensive to implement for private methods, and we're comfortable with it, let's consider just removing the checks, as originally proposed (link to spec change, and rebase it). This makes more sense to me than adding all these things that can be observed and manipulated from outside of the class.

Does it mean that brand-check actually isn't such an important requirement, if you are to move it? In this case, why not to tunnel privates trough Proxy? In the end why not Symbol.private if brand-check the only big difference between them?

• However, we're working on the implementation in V8 and JSC, and working through any implementation difficulties. We're also in touch with the SpiderMonkey team, which has a pretty analogous, optimistic plan about how to handle this implementation.

That is great, but don't we get into situation where implementation leads design and not the opposite (which is more correct)?