Skip to content

Instantly share code, notes, and snippets.

@0xbadfca11
Last active June 20, 2022 09:13
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save 0xbadfca11/f6647086ae789e1c5b8a8ee6b597f6b7 to your computer and use it in GitHub Desktop.
Save 0xbadfca11/f6647086ae789e1c5b8a8ee6b597f6b7 to your computer and use it in GitHub Desktop.
_fto132proc bug
#include <fenv.h>
#include <stdint.h>
#include <stdio.h>
extern "C" int64_t fto132proc(/* Passing by ST(0) */);
int main()
{
double f = UINT32_MAX + 0.9;
uint32_t h, l;
//printf("%x\n", fegetround());
//fesetround(f > 0 ? FE_UPWARD : FE_DOWNWARD);
#ifdef _MSC_VER
__asm
{
fld f
call fto132proc
mov h, edx
mov l, eax
}
#else
asm(
"fldl %2 \n\t"
"call fto132proc \n\t"
"movl %%edx, %0 \n\t"
"movl %%eax, %1 \n\t"
: "=d" (h), "=a" (l)
: "m"(f)
: "st"
);
#endif
printf("%lld\n", h * 1ULL << 32 | l);
printf("%llx\n", h * 1ULL << 32 | l);
}

A bug in Intel® 64 and IA-32 architectures optimization reference manual.

A buggy example code is in 3.8.4.1 Rounding Mode.

add ecx,7fffffffh ; If diff<0 then decrement integer
adc eax,0 ; INC EAX (add CARRY flag)
ret
add ecx, 7fffffffh ; If diff<0 then decrement integer
sbb eax, 0 ; DEC EAX (subtract CARRY flag)
ret

In 32-bit x86 ABI, EDX and EAX registers are used to represent 64-bit integers.
EDX used for upper 32-bit value, EAX used for lower 32-bit value.
For example, the value 0x1234567890 is EDX = 0x12, EAX = 0x34567890.
If carry or borrow occurred in EAX, it must be reflected in EDX.
But the above code forgets it.
Therefore subtracting 1 from 0x100000000(EDX = 1, EAX = 0) will be 0x1FFFFFFFF(EDX = 1, EAX = 0xFFFFFFFF).
Adding 1 to -1 (EDX = 0xFFFFFFFF, EAX = 0xFFFFFFFF) will be -4294967296 (EDX = 0xFFFFFFFF, EAX = 0).

_ftol2 of VC ++ refers to Intel code.
But VC++'s _ftol2 is fixed bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment