Ruv
(the latest at the top)
- 2026-06-04 (to be updated)
Narrow down ambiguous conditions.
Clarify meaning of terms in the typical usage examples.
Improve wording in some places.
Remove ambiguous conditions for
postponeand[compile]. Add more test cases. - 2025-10-03 Correct some normative statements; add ambiguous conditions; make better wording. Add more rationale. Correct some typos.
- 2025-09-30 Add some rationale; correct some typos and grammar mistakes; minor rewording; add consequences.
- 2025-09-29 Better wording in some places; corrections; update in ambiguous conditions.
- 2025-09-28 Huge update; incorporate proposals [249], [212] (partially), [163], [122] (partially); add tests. According to my comment on 2024-09-24, this was planned.
- 2022-09-19 explicitly allow a short formula, describe what it means, better wording, fix some typos
- 2022-08-13 Initial version
(the latest at the top)
-
2022-08-12 [249] Revert rewording the term "execution token" (proposal, retracted on 2025-09-28).
-
2021-09-08 [157] Reword the term "execution token" (proposal, accepted on 2021-09-13).
-
2021-09-08 [212] Tick and undefined execution semantics - 2 (proposal, considered on 2024-09-26).
-
2020-10-29 [163] Tick and undefined execution semantics (proposal, retracted on 2025-09-12)
-
2020-09-03 [157] Reword the term "execution token" (proposal, replaced on 2021-09-08; an attempt to solve the problem by changing the term)
-
2020-02-20 [129]
NAME>INTERPRETwording (proposal, in progress; indication of a problem) -
2019-10-08 [122] Clarify FIND, more classic approach (proposal, in progress)
By the definition of the term "execution token" in Forth-94 and Forth-2012, it's a value that identifies execution semantics. Can such a value identify other behavior, e.g., some interpretation semantics or compilation semantics? It's unclear at the first glance.
Another problem is that, following unfortunate change to the term "execution token" in the very quickly accepted proposal [157], the standard does not formally state that an execution token identifies anything at all.
Yet another problem is that it is unclear what behavior are identified by the execution token of a word whose execution semantics or interpretation semantics are not specified by the standard.
There are five words for wich the standard defines interpretation semantics
but not execution semantics:
s", s\", to, is, action-of.
Obtaining execution token (and the "tick" operation) for the last three of them is prohibited,
while the behavior identified by the execution tokens of the first two
is unclear in edge cases (it is neither explicitly defined nor undefined in compilation state).
Actually, an execution token can identify (and does identify) other semantics too, but only if they are equivalent to the execution semantics that this token also identifies.
:noname postpone if ; ( xt )- xt, which is left on the stack, identifies the execution semantics of this unnamed Forth definition.
- The execution semantics of this definition
are equivalent
to the compilation semantics for
if(this follows from the standard). - Then, this xt also identifies the compilation semantics for
if(identifies in terms of behavior, not object code).
: foo postpone if ;- xt of
fooidentifies the execution semantics offoo. - The execution semantics of
fooare equivalent to the compilation semantics forif(this follows from the standard). - Then, this xt also identifies the compilation semantics for
if.
Note that the Forth system may provide
system-dependent execution semantics for if
that are not equivalent to the execution semantics of foo.
Thus, for any execution token there exists at least one Forth definition (named or unnamed) the execution semantics of which are identified by this execution token. So, in any case, an execution token always identifies some execution semantics, but accidentally (or intentionally) these semantics can be equivalent to some interpretation semantics, or some compilation semantics, and then it identifies them too. It's unnecessary that they connected to the same Forth definition. Also, consequently, it's impossible that an execution token identifies some compilation semantics, or some interpretation semantics, but doesn't identify the equivalent execution semantics.
Note that there are cases where the semantics cannot be identified by an execution token in a Forth system, because the implementation of the system does not have or cannot have a Forth definition with equivalent execution semantics.
Examples of semantics that cannot be identified by any execution token:
- in some systems
(in which the phrase
postpone literal postpone executeis equivalent tocompile,), the formally specified execution semantics of>r; - in some systems
(where FVM does not have access to the underlying return stack,
e.g. WAForth),
the initiation semantics of
:noname; - typically, the run-time semantics of
if(which are appended to the current definition by the compilation semantics ofif);
Of course, the standard allows such implementations and disallows programs to obtain an execution token of the corresponding semantics, or even does not provide a way to obtain it.
To solve the initial problem, we can:
- formally and explicitly state the basics described above,
- specify what particular semantics are identified by the execution token of a word (in which cases they are defined by the standard and in which by the implementation, and to what extent),
- update the definition of the "execution token" term to say what these tokens identify.
Clearly defining execution tokens for edge cases
eliminates ambiguity
when obtaining them for the words to, is, and action-of.
Furthermore, the ambiguous conditions
on applying postpone and [compile] to these words
can be removed.
These conditions were inherited from the word to in Forth-94,
where they were introduced to provide a room
for both "parsing" and "non-parsing" implementations.
In reality "non-parsing" implementations were impossible
because a child of value is allowed to be immediate.
In Forth-2012, there is also no room for implementation variance of postpone
due to name>compile.
A close topic was also discussed in comp.lang.forth in 2024,
Subject: "single-xt approach in the standard".
- Message-ID:
<2024Sep23.185202@mips.complang.tuwien.ac.at>- Date: 2024-09-23 16:52:02 +00:00
- Message-ID:
<vd0e11$3itr4$1@dont-email.me>- Date: 2024-09-25 07:27:28 +00:00
-
"The execution semantics identified by xt are equivalent to the interpretation semantics of
BAZ"- This seems pretty clear.
- Note that this means that the execution semantics identified by xt are independent of STATE.
-
"xt identifies the interpretation semantics for the word
BAZ"- This means that the execution token xt identifies the execution semantics which are equivalent to the interpretation semantics for the word
BAZ. - Also, this means that the execution semantics identified by xt are independent of STATE.
- At the same time, the execution semantics of the word
BAZmay differ from the execution semantics identified by this xt.
- This means that the execution token xt identifies the execution semantics which are equivalent to the interpretation semantics for the word
-
"xt identifies the compilation semantics for the word
FOO"- This means that the execution token xt identifies the execution semantics which are equivalent to the compilation semantics for the word
FOO. - Also, this means that the execution semantics identified by xt are independent of STATE.
- At the same time, the execution semantics of the word
FOOmay differ from the execution semantics identified by this xt.
- This means that the execution token xt identifies the execution semantics which are equivalent to the compilation semantics for the word
-
"xt of the word
BAR"- This means that xt identifies the execution semantics of the word
BAR. - Whether this xt also identifies
the compilation semantics,
or the interpretation semantics,
or both of them, or neither,
for the word
BARdepends on the wordBAR(on how it is defined or specified). - Regardless of how
BARis defined, executing xt in interpretation state performs the interpretation semantics forBAR.
- This means that xt identifies the execution semantics of the word
Actually, the standard contains only one place
where the "execution token" term is used ambiguously
in a normative part —
the glossary entry for FIND.
The problem is that it says that FIND returns
the execution token of the found Forth definition
("its execution token", so it should identify
the execution semantics of that definition),
but:
FINDmay return two different execution tokens (one while compiling and another while not compiling) for the same input string (and, formally, the same Forth definition), which may identify different semantics; then, formally, at least one of them does not identify the execution semantics of the found definition (contrary to the semantic description).
In the glossary entry for
NAME>INTERPRET,
the language is just slightly non normative,
since it uses the form "xt represents" instead of the form "xt identifies".
These glossary entries also have some other problems, so they should be corrected anyway; my other proposals on this matter are in progress.
In the section 2.1 Definitions of terms, change (as of 2021-09-13):
execution token: A value that can be passed to
EXECUTE(6.1.1370)
into
execution token: A value that identifies the execution semantics of a definition.
Rationale
- This is necessary to correctly describe the "execution token" data type.
In the section 3.1.3.5 Execution tokens, make the following changes.
For any valid execution token in the system, there is at least one Forth definition (named or unnamed) whose execution semantics are identified by that execution token.
The execution semantics identified by an execution token may be equivalent to the interpretation semantics, compilation semantics, or other semantics for some named Forth definition (or definitions). In such cases, the execution token also identifies those interpretation, compilation, or other semantics.
The system does not need to identify every specified semantics by any execution token.
The execution token of a Forth definition, if available, identifies the execution semantics that are either specified by this standard for that definition or are implementation dependent (if allowed).
If the interpretation semantics for a Forth definition are defined by this standard, the execution token of that definition shall be available.
Rationale
- We use the clause "valid" in "any valid execution token"
because an execution token may become invalid
after using words like
forgetand children ofmarker. - The "if available" clause is used since
find(in interpretation state) andname>interpretmay return zero for some existing (but not user-defined) words; this effectively means that the execution token is not available for those words, andsearch-wordlistshould also return zero for them. - The last paragraph guarantees that any word that is allowed to be Ticked has an execution token.
See also: A.3.1.3.5 Execution tokens.
In the section 3.4.3.1 Execution semantics, make the following changes.
The execution semantics of
eachForth definition are specified in an "Execution:" section of its glossary entry. When a definition has only one specified behavior, the label is omitted.
into
The execution semantics of a Forth definition are specified in an "Execution:" section of its glossary entry. When a definition has only one specified behavior, the label is omitted.
Rationale
- For some words, the execution semantics are not specified by the standard.
If the execution semantics for a Forth definition are specified by this standard and the glossary entry of that definition does not have an "Interpretation:" section, the execution token of that definition identifies the specified execution semantics. Otherwise, the execution token of that definition, if available, identifies the implementation dependent execution semantics.
Rationale
- We have to rely on the absence of an "Interpretation:" section
to refer to ordinary and immediate words
until words like
>rwill be defined using a "Run-Time:" section instead of an "Execution:" section in their glossary entries (commented on 2019-06-21, 2020-08-30). - We use the term "implementation dependent" rather than "implementation defined" because this allows implementors to avoid documenting the provided execution semantics of standard words whose execution semantics are not specified by the standard.
The execution semantics identified by the execution token of a Forth definition, when they are performed in interpretation state, shall perform the interpretation semantics of that definition.
Rationale
- This guarantees that, while interpreting,
' foo executeis equivalent tofoo, even when the execution token (xt) offooidentifies implementation dependent execution semantics.
An ambiguous condition exists if the execution semantics of a Forth definition are not specified by this standard and its execution token is executed in compilation state under conditions other than those specified in this standard.
Rationale
- That behavior in compilation state might depend on the implementation.
Namely, if such a word (e.g.
s") is implemented using the single-xt approach (all classic Forth systems), this performs the compilation semantics for the word, but if such a word (e.g.s") is implemented using the dual-xt approach, this typically performs the interpretation semantics for the word. - An example of conditions under which
execution in compilation state
of the execution token of such a word (e.g.
s") is not ambiguous: iffindin compilation state returns( xt1 1 )for a word, and, coincidentally, xt1 is the execution token of the word.
An ambiguous condition exists if the interpretation semantics of a Forth definition are undefined by this standard and its execution token is executed under conditions other than those specified in this standard.
Rationale
- The execution semantics of the definition
:noname s" if" forth-worldlist search-wordlist if execute then ;is ambiguous, regardless of whether they are performed while interpreting or compiling, because they execute (at least on some systems) the execution token of the wordif, whose interpretation semantics are undefined by this standard. - In the same time, the execution semantics of the definition
:noname c" if" find 1 = if execute then ;in compilation state is not ambiguous even iffindin this case returns the same execution token as['] if(which identifies the implementation dependent execution semantics ofif).
In the section 4.1.2 Ambiguous conditions,
replace the phrase:
attempting to obtain the execution token, (e.g., with 6.1.0070
', 6.1.1550FIND, etc. of a definition with undefined interpretation semantics;
with the phrase:
attempting to obtain, using the words 6.1.0070
'or 6.1.2510['], the execution token of a definition with undefined interpretation semantics;
Rationale
- "
find" and "etc." are excluded from the list, becausefind(in interpretation state),search-wordlist, andname>interpretreturn either the execution token of the word or zero, and there is no ambiguous condition in this regard.
In the glossary entries
6.2.2295 TO,
6.2.1725 IS,
6.2.0698 ACTION-OF,
remove the sentence that starts with:
An ambiguous condition exists if any of
POSTPONE,[COMPILE],'or[']are applied
Editorial note: in the first case the form “are applied” is used, in the second and third cases the form “is applied” is used.
Rationale
- Historically, there were
two approaches
to implement
to: "parsing" and "non-parsing". Forth-94 formally specified the "parsing" approach. It also declared an ambiguous condition, perhaps, as a concession to implementations that adhered to the "non-parsing" approach. Note that no similar exception was made for the words", since it always do parsing of the parse area. In Forth-2012, more exceptions for the wordtowere added without obvious ground, and also for the wordsisandaction-of. - It is now specified that the execution tokens of these words identify implementation dependent execution semantics that, when performed in interpretation state, shall perform the specified interpretation semantics for these words.
- Also, since these words shall parse the parse area for their immediate argument,
there is no any reason
for an ambiguous condition
when applying
postponeor[compile]to these words (just as there is no such reason when applying them tos"). - Note that
[compile]is obsolescent.
This change specifies all the conditions under wich
an xt returned by search-wordlist,
' (Tick), and ['] (Bracket Tick)
can (or cannot) be used by a standard program.
This change removes prohibition on Ticking the words to, is, action-of,
and specifies that executing of the returned xt in interpretation state
shall perform the interpretation semantics for the word.
Note that executing can be performed directly by execute,
or indirectly by executing of the definition
in which this xt is compiled using compile,.
Also, this change removes prohibition on applying postpone and [compile]
to the words to, is, action-of.
If the system throws an error on postponing these words, or on ticking these words, or does not provide a correct xt for them, it should be updated to be compliant.
Most Forth systems in which these words are parsing words (as the standard requires) already comply with this change.
According to Anton Ertl's testing of five Forth systems
(comment [r885])
in 2022,
iForth 5.0.27 and VfxForth 5.11 did not comply with this change
with respect to the execution tokens of the word to.
In VfxForth 5.43,
the words is and action-of do parsing
(as specified in the standard), but the word to does not.
Another issue is that the execution tokens of these three words,
as well as the word s" (ticking of which is allowed),
are not compiled correctly by compile,.
This change does not affect the existing standard programs. However, it allows the use of certain new techniques (previously disallowed) in standard programs.
This change answers to the following requests for clarification:
- Ambiguous condition in case of undefined execution semantics
- Eliminating ambiguous conditions for Tick (partially)
See:
This proposal at
forth-standard.org:https://forth-standard.org/proposals/clarification-for-execution-token?hideDiff#reply-1572