Skip to content

Instantly share code, notes, and snippets.

@paleolimbot
Last active April 3, 2024 01:21
Show Gist options
  • Save paleolimbot/8183f50a1c1425db4a744903ba3f905b to your computer and use it in GitHub Desktop.
Save paleolimbot/8183f50a1c1425db4a744903ba3f905b to your computer and use it in GitHub Desktop.

From the root arrow checkout of https://github.com/paleolimbot/arrow plus git switch archery-nanoarrow-integration:

Make sure archery is installed:

pip install -e "dev/archery[all]"

From the csharp/ subdirectory run

dotnet build

From the arrow checkout, clone + build nanoarrow where the default Archery options look for the integration test binary.

git clone https://github.com/paleolimbot/arrow-nanoarrow.git nanoarrow
cd nanoarrow && git switch integration-fixes
mkdir cdata && cd cdata && cmake .. -DNANOARROW_BUILD_INTEGRATION_TESTS=ON && cmake --build .
cd ../..

Run the integration test for exactly one file that is failing

python -m "archery.cli" integration \
    --with-nanoarrow=true \
    --with-csharp=true \
    --run-c-data \
    --match=primitive_zerolength
"""
From the root arrow checkout of https://github.com/paleolimbot/arrow
plus `git switch archery-nanoarrow-integration`:
Make sure archery is installed:
```
pip install -e "dev/archery[all]"
```
From the csharp/ subdirectory run
```
dotnet build
```
From the arrow checkout, clone + build nanoarrow where the default Archery
options look for the integration test binary.
```
git clone https://github.com/paleolimbot/arrow-nanoarrow.git nanoarrow
cd nanoarrow && git switch integration-fixes
mkdir cdata && cd cdata && cmake .. -DNANOARROW_BUILD_INTEGRATION_TESTS=ON && cmake --build .
cd ../..
```
Run the integration test for exactly one file that is failing
python -m "archery.cli" integration \
--with-nanoarrow=true \
--with-csharp=true \
--run-c-data \
--match=primitive_zerolength
"""
import nanoarrow as na
from archery.integration import cdata
from archery.integration.tester_cpp import CppTester
from archery.integration.tester_csharp import CSharpTester
from archery.integration.tester_nanoarrow import NanoarrowTester
with open("primitive-empty.json", "w") as f:
f.write("""{
"schema": {
"fields": [
{"name": "binary_nullable", "type": {"name": "binary"}, "nullable": true, "children": []}
]
},
"batches": [
{
"count": 0,
"columns": [
{"name": "binary_nullable", "count": 0, "VALIDITY": [], "OFFSET": [0], "DATA": []}
]
}
]
}""")
tester_na = NanoarrowTester()
tester_cs = CSharpTester()
exporter = tester_na.make_c_data_exporter()
importer = tester_cs.make_c_data_importer()
json_path = "primitive-empty.json"
ffi = cdata.ffi()
c_array_ptr = ffi.new("struct ArrowArray*")
with cdata.check_memory_released(exporter, importer):
exporter.export_batch_from_json(json_path, 0, c_array_ptr)
importer.import_batch_and_compare_to_json(json_path, 0, c_array_ptr)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment