Instructions on getting through HDF compilation for Android

HDF5 Android How-to

How to build HDF5 for Android


References:
-----------

[1] http://developer.android.com/tools/sdk/ndk/index.html

[2] http://developer.android.com/sdk/index.html

[3] 
http://linux4hippos.blogspot.com/2013/05/cross-compile-hdf5-for-intel-xeon-phi.html

[4] http://danielpocock.com/building-existing-autotools-c-projects-on-android

Build Environment:
------------------

Ubuntu 12.04 LTS (64-bit)

Android NDK R9b x86_64 from [1]

HDF5 1.8.11

Steps:
------

0. Install the NDK in $ANDROID_NDK .

1. Create a standalone toolchain (GCC w/ the right sysroot) for Android 4.3 (API 
18) on ARM

$ANDROID_NDK/build/tools/make-standalone-toolchain.sh --platform=android-18

This creates a file called 
/tmp/ndk-<username>/arm-linux-androideabi-4.6.tar.bz2.
Unzip and set your environment variables. E.g.,

export PATH=/home/<username>/arm-linux-androideabi-4.6/bin:$PATH
export AR=arm-linux-androideabi-ar
export AS=arm-linux-androideabi-as
export CC=arm-linux-androideabi-gcc
export CPP=arm-linux-androideabi-cpp
export CXX=arm-linux-androideabi-g++
export LD=arm-linux-androideabi-ld
export NM=arm-linux-androideabi-nm
export RANLIB=arm-linux-androideabi-ranlib

2. Without some rather ugly hacks, configure is going nowhere. Following [3] a
few changes are necessary to get us off the ground:

a.) Edit test/Makefile.in and change the definition of TEST_SCRIPT from

TEST_SCRIPT = testerror.sh testlibinfo.sh testcheck_version.sh \
testlinks_env.sh $(am__append_1)

to

TEST_SCRIPT =

b.) Edit configure and remove the return statement 'as_fn_exit $as_status' from
the definition of as_fn_error.

See subtask HDFFV-8560.

3. ./configure -host=arm-linux-androideabi

See the attached config.log for the output. I couldn't figure out how to 
convince
configure to build the shared libraries. See subtask HDFFV-8561.

4. 'make' runs for a while and fails when compiling H5make_libsettings.c
See subtask HDFFV-8562.

We can fix that brute force by removing '#define H5_HAVE_GETPWUID 1' from 
H5pubconf.h

5. Running 'make' again tells us that we can't execute ARM binaries on our 
machine:

CC H5make_libsettings.o
CCLD H5make_libsettings
LD_LIBRARY_PATH="$LD_LIBRARY_PATH`echo | \
sed -e 's/-L/:/g' -e 's/ //g'`" \
./H5make_libsettings > H5lib_settings.c || \
(test $HDF5_Make_Ignore && echo "*** Error ignored") || \
(rm -f H5lib_settings.c ; exit 1)
/bin/bash: line 1: ./H5make_libsettings: cannot execute binary file
make[2]: *** [H5lib_settings.c] Error 1
make[2]: Leaving directory `/home/gerd/Android/HDF5/hdf5-1.8.11/src'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/gerd/Android/HDF5/hdf5-1.8.11/src'
make: *** [all-recursive] Error 1

See subtask HDFFV-8563.

5. Time to fire up the Android phone emulator! This is part of the Android SDK. 
[2]
Create a new Android Virtual Device (AVD).
Give it an name, e.g., 'HDF5emu'.
Device: Nexus S (4.0", 480x800: hdpi)
Target: Android 4.3 (API Level 18)
CPU/ABI: ARM (armeabi-v7a)
... (no changes of defaults)
Internal Storage: 1 GiB
SD Card: 256 MiB
Emulation Options: check 'Snapshot'

Click 'Ok' and start the new AVD. (click 'Start')

Make sure that the file system you are running the tests in is mounted 
read/write:

adb shell
su
mount -o rw,remount rootfs /
chmod 777 /mnt/sdcard
exit

6. Copy H5make_libsettings onto to AVD via the Android Debug Bridge:

Create a directory: adb shell mkdir /mnt/sdcard/hdf5
Copy the file: adb push src/H5make_libsettings /mnt/sdcard/hdf5
Copy the file: adb push src/libhdf5.settings /mnt/sdcard/hdf5
Make the file executable: adb shell chmod 777 
/mnt/sdcard/hdf5/H5make_libsettings
Run it: adb shell "cd /mnt/sdcard/hdf5 ; ./H5make_libsettings"

See subtask HDFFV-8564.

See the attachment H5lib_settings.c for the output.

Type 'make'.

7. 'make' cruises along until H5detect

CCLD H5detect
LD_LIBRARY_PATH="$LD_LIBRARY_PATH`echo | \
sed -e 's/-L/:/g' -e 's/ //g'`" \
./H5detect > H5Tinit.c || \
(test $HDF5_Make_Ignore && echo "*** Error ignored") || \
(rm -f H5Tinit.c ; exit 1)
/bin/bash: line 1: ./H5detect: cannot execute binary file
make[2]: *** [H5Tinit.c] Error 1
make[2]: Leaving directory `/home/gerd/Android/HDF5/hdf5-1.8.11/src'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/gerd/Android/HDF5/hdf5-1.8.11/src'
make: *** [all-recursive] Error 1

Same fix as before: copy H5detect to AVD, run, and capture the output in 
H5Tinit.c (see attachment)

8. 'make' finishes building the library and starts making tests. It fails in
     compiling tcheck_version because of a global name conflict. See subtask
     HDFFV-8565.

     Android defines two global functions 'major' and 'minor'. For now, just
     rename the global variables in tcheck_version and move on!

9. 'make' continues and starts building the tools until it hits h5ls. See
     subtask HDFFV-8566.

    Brute force fix: delete H5_HAVE_IOCTL from H5pubconf.h .

10. H5make_libsettings is invoked a second time and fails. See subtask HDFFV-8563.

11. H5detect is invoked a second time and fails. See subtask HDFFV-8563.

12. After that tests and tools build fine to the end.

13. Copy all the tests and HDF5 files onto the AVD and run them. All pass, as
     far as I recall.

14. 'make install' works.

Hello, I have a small question for you. I am working on the Android Application, and due to limitation of heap size on that OS, 'nujan' (pure java library) library is quite outdated and not allow to stream data into the file. and I want to use a JHDF library for that, can I compile the native hdf5 library using Android Studio on windows? Or I should use Linux only to do that? Thanks a million for the tutorial above.