Currently, Test262 tests are required during Stage 3, where the proposal becomes implementation ready.
There is a suggestion for this TC39 meeting to move the Test262 tests as an entrance criteria for Stage 3.
The idea is good, with some trade offs and considerations.
It's good to have Test262 tests available for implementation as soon as a proposal reaches Stage 3. This gives a better base for implementing the proposed features in, perhaps, a more consistent manner.
Authoring tests helps finding spec issues (bugs, typos, web reality). This might give a better quality for proposals reaching stage 3.
This implies more compromise from proposal champions.
Stage 2 is too unstable for tests. This is pretty sensible for feature proposals that require big test footprints.
E.g. the Decorators require a fairly big amount of tests and the proposal has historically been changing throughout time. This would inevitably imply in some Stage 2.5 status, where TC39 agrees with the proposed spec, wants to go ahead with it, but needs tests to be ready before saying it's ready for implementation.
The cost of authoring and reviewing tests is high, WRT invested time. Large modifications to tests is costly.
Test262 tests being ready is different from tests landing into Test262's main (HEAD) branch. The proposed process change seems good enough to assume tests will be ready within the proposal (preferred) or as live Pull Requests to Test262.
We can provide better tooling within proposal templates to have a test262 runner ready.
Landing Stage 2 tests to the mainstream branch seems like a bad contract for implementations. Many tests won't be passing because implementations are not ready yet.
Test262 process lacks official and complete coverage tooling* and it seems unclear for most people what is necessary to cover. It is complex to translate to a person what needs to be covered other than just reading the specs and mapping existing tests.
The entrance criteria will need to ensure Test262 are sufficient, and the subjective aspect might miss several important edge cases. If this happens, it gives little value to having the tests. A good coverage needs to be enforced.
* - engine262 test reports are helpful to track coverage for built-in APIs