Prompt for a novel arithmetic operation, following Zhou et al. 2022. See comments.

prompt.txt

Problem: 821`763=
Explanation:
The first number is 821, FN=[8,2,1]. The second number is 763, SN=[7,6,3]. Since FN [8,2,1] has 3 digits, SN [7,6,3] has 3 digits, thus the maximum number of digits is 3. In each subsequent step, we remove one number from the beginning of FN and one from the beginning of SN.
Length of FN is 3. FN=[8,2,1]. Length of SN is 3. SN=[7,6,3]. FN[3]=8. SN[3]=7. C[3]=0.  Since 8*7=56, 56%10=6, 56//10=5. Length of A is 1. Thus A=[6]. Since 5 is odd, we change the value of C. Since C[3]=0, C[2]=1. 
Length of FN is 2. FN=[2,1]. Length of SN is 2. SN=[6,3]. FN[2]=2. SN[2]=6. C[2]=1. Since 2*6=12, 12%10=2, 12//10=1. Length of A is 2. Thus A=[6,2]. Since 1 is odd, we change the value of C. Since C[2]=1, C[1]=0.
Length of FN is 1. FN=[1]. Length of SN is 1. SN=[3]. FN[1]=1. SN[1]=3. C[1]=0. Since 1*3=4, 4%10=4, 4//10=0. Length of A is 3. Thus A=[6,2,4]. Since 0 is even, we do not change  the value of C. Since C[1]=0, C[0]=0.
There are no more digits and C[0]=0. Thus the process is complete. Since there are no more operators, the problem is complete. The final Answer is [6,2,4].

Problem: 9098`291=
Explanation:
The first number is 9098, FN=[9,0,9,8]. The second number is 291, SN=[2,9,1]. Since FN [9,0,9,8] has 4 digits, SN [2,9,1] has 3 digits, thus the maximum number of digits is 4. In each subsequent step, we remove one number from the beginning of FN and one from the beginning of SN.
Length of FN is 4. FN=[9,0,9,8]. Length of SN is 3. SN=[2,9,1]. FN[4]=9. SN[4]=2. C[4]=0. Since 9*2=18, 18%10=8, 18//10=1. Length of A is 1. Thus A=[8]. Since 1 is odd, we change the value of C. Since C[4]=0, C[3]=1.
Length of FN is 3. FN=[0,9,8]. Length of SN is 2. SN=[9,1]. FN[3]=0. SN[3]=9. C[3]=1. Since 9*0=0, 0%10=0, 0//10=0. Length of A is 2. Thus A=[8,0]. Since 0 is even, we do not change the value of C. Since C[3]=1, C[2]=1.
Length of FN is 2. FN=[9,8]. Length of SN is 1. SN=[1]. FN[2]=9. SN[2]=1. C[2]=1. Since 9*1=9, 9%10=9, 9//10=0. Length of A is 3. Thus A=[8,0,9]. Since 0 is even, we do not change  the value of C. Since C[2]=1, C[1]=1.
Length of FN is 1. FN=[8]. Length of SN is 0. SN=[]. FN[1]=8. SN is empty. C[1]=1. Since 8*0=0, 0%10=0, 0//10=0. Length of A is 4. Thus A=[8,0,9,0]. Since 0 is even, we do not change  the value of C. Since C[1]=1, C[0]=1.
There are no more digits but C[0]=1. Length of A is 5. Thus A=[8,0,9,0,1]. 
There are no more digits and the process is complete. Since there are no more operators, the problem is complete. The final Answer is [8,0,9,0,1].

Problem: 743`128=
Explanation:
The first number is 743, FN=[7,4,3]. The second number is 128, SN=[1,2,8]. Since FN [7,4,3] has 3 digits, SN [1,2,8] has 3 digits, thus the maximum number of digits is 3. In each subsequent step, we remove one number from the beginning of FN and one from the beginning of SN.
Length of FN is 3. FN=[7,4,3]. Length of SN is 3. SN=[1,2,8]. FN[3]=7. SN[3]=1. C[3]=0.  Since 7*1=7, 7%10=7, 7//10=0. Length of A is 1. Thus A=[7]. Since 0 is even, we do not change  the value of C. Since C[3]=0, C[2]=0. 
Length of FN is 2. FN=[4,3]. Length of SN is 2. SN=[2,8]. FN[2]=4. SN[2]=2. C[2]=0. Since 4*2=8, 8%10=8, 8//10=0. Length of A is 2. Thus A=[7,8]. Since 0 is even, we do not change  the value of C. Since C[2]=0, C[1]=0.
Length of FN is 1. FN=[3]. Length of SN is 1. SN=[8]. FN[1]=3. SN[1]=8. C[1]=0. Since 8*3=24, 24%10=4, 24//10=2. Length of A is 3. Thus A=[7,8,4]. Since 2 is even, we do not change  the value of C. Since C[1]=0, C[0]=0.
There are no more digits and C[0]=0. Thus the process is complete. Since there are no more operators, the problem is complete. The final Answer is [7,8,4].

Problem: 862`7645=
Explanation:
The first number is 862, FN=[8,6,3]. The second number is 7645, SN=[7,6,4,5]. Since FN=[8,6,3] has 3 digits, SN=[7,6,4,5] has 4 digits, thus the maximum number of digits is 4. In each subsequent step, we remove one number from the beginning of FN and one from the beginning of SN.
Length of FN is 3. FN=[8,6,3]. Length of SN is 4. SN=[7,6,4,5]. FN[4]=8. SN[4]=7. C[4]=0. Since 8*7=56, 56%10=6, 56//10=5. Length of A is 1. Thus A=[6]. Since 5 is odd, we change the value of C. Since C[4]=0, C[3]=1.
Length of FN is 2. FN=[6,3]. Length of SN is 3. SN=[6,4,5]. FN[3]=6. SN[3]=6. C[3]=1. Since 6*6=36, 36%10=6, 36//10=3. Length of A is 2. Thus A=[6,6]. Since 3 is odd, we change the value of C. Since C[3]=1, C[2]=0.
Length of FN is 1. FN=[3]. Length of SN is 2. SN=[4,5]. FN[2]=3. SN[2]=4. C[2]=0. Since 3*4=12, 12%10=2, 12//10=1. Length of A is 3. Thus A=[6,6,2]. Since 1 is odd, we change the value of C. Since C[2]=0, C[1]=1.
Length of FN is 0. FN=[]. Length of SN is 1. SN=[5]. FN is empty. SN[1]=5. C[1]=1. Since 0*5=0, 0%10=0, 0//10=0. Length of A is 4. Thus A=[6,6,2,0]. Since 0 is even, we do not change  the value of C. Since C[1]=1, C[0]=1.
There are no more digits but C[0]=1. Length of A is 5. Thus A=[6,6,2,0,1]. 
There are no more digits and the process is complete. Since there are no more operators, the problem is complete. The final Answer is [6,6,2,0,1].

Problem: 17203`891457=

This is essentially an additional control for Teaching Algorithmic Reasoning via In-context Learning by Hattie Zhou (@hlml) et al.

This paper showed that large language models (LLMs) can learn to perform multi-step arithmetic operations, in-context and in a way that generalizes to arbitrary-size numbers, simply by explaining them carefully how to do it.

One caveat is that this paper used the common arithmetic operations (addition, subtraction, multiplication). Because there are many web pages explaining how to perform these operations, this left open the possibility that these results were at least in part influenced by some form of contamination.

This prompt describes a totally novel (and deliberately bizarre) arithmetic operation: pointwise multiplication modulo 10 with binary evenness-based carry, left-to-right Given two numbers, the result is computed as follows:

1. Take the last (left-most) digit of each number and multiply them together
2. The first digit of the result is the output digit for this position
3. If the second digit is even, leave the binary carry unchanged. If it is odd, switch the carry (make it 0 if it is 1, or 1 if it is 0)
4. Iterate, from left to right
5. If one of the numbers runs out of digits, assume the corresponding digit for that number is 0.
6. After all digits have been processed, if the carry is 1 add a 1 to the output.

Like other arithmetic operations, performing this operation on arbitrary numbers requires learning a complex, open-ended, stateful algorithm. Unlike other arithmetic operations, it cannot possibly be subject to contamination, since this operation is entirely novel and thus hasn't been described before.

Yet, based on this prompt alone, LLMs can learn to perform this novel operation, including on numbers larger than anything seen in the prompt. This confirms the paper's original results: LLMs do not merely apply stored patterns from their training, but can identify novel complex (algorithmic) patterns in their input, and then generalize them to novel, unseen inputs.

Implications concerning "LLM intelligence" are left to the reader.

The prompt is adapted from Zhou et al.'s original addition prompt. Confirmed to work on GPT 3.5 (complete-mode) with text-davinci-003 engine.