descriptor_plan.md

Plan: Align LFortran Descriptor with Fortran Standard / Flang

Problem

LFortran's array descriptor (CFI_cdesc_t) diverges from the Fortran standard and from both GFortran and Flang in several ways. This causes ABI incompatibility with any C code compiled against a standard-compliant ISO_Fortran_binding.h.

Current LFortran descriptor (48 + rank×24 bytes):

{ ptr, i64, i32, i8, i16, i8, i64, i8, [15 x {i64,i64,i64}] }

Offset  Size  Field
  0       8   base_addr (void*)
  8       8   elem_len (int64_t)
 16       4   version (int32_t)
 20       1   rank (int8_t)           ← signed
 21       1   pad
 22       2   type (int16_t)          ← 2 bytes
 24       1   attribute (int8_t)      ← signed
 25       7   pad
 32       8   offset (int64_t)        ← non-standard, LFortran-only
 40       1   is_allocated (uint8_t)  ← non-standard, LFortran-only
 41       7   pad
 48    rank×24  dim[] = { sm, lower_bound, extent }   ← wrong order

Total header: 48 bytes. Total for rank-15: 48 + 360 = 408 bytes.

Target descriptor (24 + rank×24 bytes, matches Flang):

{ ptr, i64, i32, u8, i8, u8, u8, [rank x {i64,i64,i64}] }

Offset  Size  Field
  0       8   base_addr (void*)
  8       8   elem_len (size_t)
 16       4   version (int)
 20       1   rank (uint8_t)          ← unsigned
 21       1   type (int8_t)           ← 1 byte, signed (for -1)
 22       1   attribute (uint8_t)     ← unsigned
 23       1   extra (uint8_t)         ← reserved/future use
 24    rank×24  dim[] = { lower_bound, extent, sm }   ← standard order

Total header: 24 bytes. Total for rank-15: 24 + 360 = 384 bytes.

Answers to open questions

Q: For type codes (item 2), should we encode like GFortran?

GFortran uses int16_t with the scheme intrinsic_type + (kind << 8), e.g.:

• CFI_type_int32_t = 1 + (4 << 8) = 0x0401
• CFI_type_double = 3 + (8 << 8) = 0x0803

This is self-describing (you can extract type category and kind from the code) but requires 2 bytes and produces different numeric values than Flang.

Flang uses signed char (1 byte) with flat sequential codes (1, 2, 3, ..., 49). Simpler, smaller, but the code is just an opaque ID.

Recommendation: Match Flang's 1-byte approach. The type code is only used for comparisons and dispatch — the self-describing property of GFortran's scheme isn't needed at runtime. Using 1 byte keeps the header at 24 bytes with no padding. If we used 2 bytes, the header would grow to 28+ bytes due to alignment, or we'd need to rearrange fields.

Q: For Flang's extra field — what is it, what does it store?

Flang's extra byte sits at offset 23 (the last byte of the 24-byte header, which would otherwise be padding). It encodes two things in its bits:

• Bit 0 (_CFI_ADDENDUM_FLAG = 1): Whether a "DescriptorAddendum" follows the dim[] array.
• Bits 1–3 (_CFI_ALLOCATOR_IDX_MASK = 0b00001110, shifted by 1): An allocator index identifying which memory allocator manages the descriptor's data.

When the addendum flag is set, the following data appears in memory after dim[rank]:

struct DescriptorAddendum {
    const DerivedType *derivedType_;   // 8 bytes: RTTI pointer
    int64_t len_[];                    // 8 bytes each: LEN parameter values
};

This is used for:

• Polymorphic objects (class(T)): derivedType_ points to runtime type info, enabling SELECT TYPE, SAME_TYPE_AS, type-bound dispatch.
• Parameterized derived types: len_[] stores deferred LEN parameter values at runtime.

The total descriptor size in Flang is: 24 + rank×24 + (addendum ? 8 + nLenParams×8 : 0)

Recommendation: For now, keep the extra byte as a reserved zero byte for future compatibility. We can use it later for polymorphic support or custom allocators.

Changes (in recommended implementation order)

1. Fix dim order (standard-mandated)

Change CFI_dim_t from {sm, lower_bound, extent} to the standard-required {lower_bound, extent, sm}.

Files to change:

• src/libasr/runtime/ISO_Fortran_binding.h — reorder struct members
• src/libasr/codegen/llvm_array_utils.h and .cpp — the LLVM IR struct type for dim, all GEP index constants for stride/lower_bound/extent
• src/libasr/codegen/asr_to_llvm.cpp — every place that reads/writes dim fields via GEP indices
• src/libasr/codegen/llvm_utils.cpp — dim field access if any

The dim GEP indices will change:

• Current: dim[i][0]=sm, dim[i][1]=lower_bound, dim[i][2]=extent
• New: dim[i][0]=lower_bound, dim[i][1]=extent, dim[i][2]=sm

2. Remove is_allocated field

Replace all uses of is_allocated with base_addr != NULL checks, matching the standard semantics.

Files to change:

• src/libasr/codegen/llvm_array_utils.h and .cpp — remove is_allocated from the descriptor struct type, update all field index constants (everything after is_allocated shifts)
• src/libasr/codegen/asr_to_llvm.cpp — replace all reads/writes of is_allocated with base_addr == null / base_addr != null comparisons
• src/libasr/runtime/ISO_Fortran_binding.h — remove the field

3. Remove offset field

Fold offset computation into base_addr adjustment or compute it from dim[i].lower_bound × dim[i].sm.

Files to change:

• src/libasr/codegen/llvm_array_utils.h and .cpp — remove offset from descriptor struct type, update field index constants
• src/libasr/codegen/asr_to_llvm.cpp — replace offset field reads/writes with computed values or base_addr adjustments
• src/libasr/runtime/ISO_Fortran_binding.h — remove the field

This is the hardest change because offset is deeply used in array indexing throughout codegen. Need to audit every use of the offset GEP to understand what it's computing and replace it.

4. Use unsigned types for rank and attribute

Change int8_t to uint8_t for rank and attribute in the C header and corresponding LLVM IR types.

Files to change:

• src/libasr/runtime/ISO_Fortran_binding.h — change typedefs
• src/libasr/codegen/llvm_array_utils.cpp — verify IR type (i8 is i8 regardless of signedness, so this may be a no-op in LLVM IR)

5. Shrink type from int16_t to int8_t

Change the type field from 2 bytes to 1 byte and adopt Flang-compatible type codes.

Files to change:

• src/libasr/runtime/ISO_Fortran_binding.h — change typedef, update all CFI_type_* macro values to match Flang's codes
• src/libasr/codegen/llvm_array_utils.cpp — change i16 to i8 in struct type
• src/libasr/codegen/asr_to_llvm.cpp — update type code assignments
• Any code that maps ASR types to CFI type codes

6. Add extra reserved byte

Add a uint8_t extra field at offset 23 (after attribute), initialized to 0. This fills the padding byte and reserves it for future use (polymorphic addendum flag, allocator index).

Files to change:

• src/libasr/runtime/ISO_Fortran_binding.h — add field
• src/libasr/codegen/llvm_array_utils.cpp — add i8 to struct type
• src/libasr/codegen/asr_to_llvm.cpp — initialize to 0 where descriptors are created

7. Change dim from fixed [15] to flexible / rank-sized

Currently LFortran always allocates dim[15]. This wastes space for low-rank arrays (most arrays are rank 1–3). Consider switching to rank-sized allocation or flexible array member.

Note: This is a larger change that affects descriptor allocation everywhere and can be deferred. The fixed [15] approach works correctly, just wastes memory.

Verification

After each change:

• Rebuild: ninja -j8
• Reference tests: ./run_tests.py
• Integration tests: cd integration_tests && ./run_tests.py -b llvm && ./run_tests.py -b llvm -f
• The bindc_iso_fb_04 test is the key validation — it exercises the C descriptor layout from C code.

Notes

• Changes 1–6 should each be a separate commit.
• Change 7 (flexible dim) can be deferred to a future PR.
• The final descriptor layout will match Flang's header byte-for-byte (24-byte header + rank×24 dim array), enabling future cross-compiler interop.
• All tests/reference/ files containing LLVM IR with %array type will need updating after struct changes.