AndroidCI.md

#Android and CI and Gradle (A How-To)

There are tech stacks in this world that make it dead simple to integrate a CI build system.
The Android platform is not one of them.

Although Gradle is getting better, it's still a bit non-deterministic, and some of the fixes you'll need will start to feel more like black magic than any sort of programming.

But fear not! It can be done!

Before we embark on our journey, you'll need a few things to run locally:

1. A (working) Gradle build
2. Automated tests (JUnit, Espresso, etc.)

If you don't have Gradle set up for your build system, it is highly recommend that you move your projects over. Android Studio has a built-in migration tool, and the Android Dev Tools website has an excellent guide on how to migrate over to the Gradle build system, whether you're on Maven, Ant, or some unholy combination of all three.

A very general example of a build.gradle file follows:

//build.gradle in /app
apply plugin: 'com.android.application'

buildscript {
	repositories {
		mavenCentral()
	}
	dependencies {
        classpath 'com.android.tools.build:gradle:0.13.2'
    }
}
//See note below
task wrapper(type: Wrapper) {
    gradleVersion = '2.1'
}

android {
    compileSdkVersion 19
    buildToolsVersion "20.0.0"
    
    defaultConfig {
        applicationId "com.example.originate"
        minSdkVersion 14
        targetSdkVersion 19
        versionCode 1
        versionName "1.0"

        testApplicationId "com.example.originate.tests"
        testInstrumentationRunner "android.test.InstrumentationTestRunner"
    }
  
    buildTypes {
        debug {
            debuggable true
        }
        release {
            debuggable false
        }
    }
}

dependencies {
    compile project(':libProject')
    compile com.android.support:support-v4:21.0.+
}

(NOTE: the Gradle Wrapper task isn't strictly necessary, but a highly recommended way of ensuring you always know what version of Gradle you're using - both for futureproofing and for regressions)

Check out the Android Developers website for some good explanations and samples.

##Choose your weapon Personally, I'm a fan of CircleCI. They sport a clean, easy-to-use interface, support more languages than you could possibly care about. Plus, they are free for open source Github projects!
(Other options include TravisCI, Jenkins, and Bamboo)

In this guide, we'll be using CircleCI, but these instructions should translate readily to TravisCI.

##Configure all the things! In order to use CircleCI to build/test your Android library, there's some configuration necessary. Below are some snippets of some of the basis configurations you might want/do. About half of this comes from the CircleCI docs and half of it comes from my blood, sweat, and tears.

At the end of this section, I'll include a complete circle.yml file.

###Machine First, the code:

machine:
	environment:
		ANDROID_HOME: /home/ubuntu/android
    java:
        version: oraclejdk6

1. The setting of the ANDROID_HOME environment variable is necessary for the Android SDKs to function properly. It'll also be useful for booting up the emulator in later steps.
2. Although setting the JDK version isn't strictly necessary, it's nice to ensure that it doesn't change behind-the-scenes and possibly surprise-bork your build.

###Dependencies + Caching

dependencies:
	cache_directories:
		- ~/.android
        - ~/android
    override:
	    - (source scripts/environmentSetup.sh && getAndroidSDK)

1. By default, CircleCI will cache nothing. You might think this a non-issue right now, but you'll reconsider when each build takes 10+ minutes to inform you that you dropped a semicolon in your log statement.
   By caching ~/.android and ~/android, you can shave precious minutes off of your build time.
2. Android provides us with a nifty command-line utility called...android (inventive!). We can use this in little Bash script that we'll write in just a second. For now, just know that scripts/environmentSetup.sh can be whatever you want, as can the Bash function getAndroidSDK.

####Bash Scripts - a Jaunt into the CLI Gradle is good at a lot of things, but it isn't yet a complete build system. Sometimes, you just need some good ol'fashioned bash scripting.

In this round, we'll download Android API 19 (Android 4.4 Jelly Bean) and create a hardware-accelerated Android AVD (Android Virtual Device - aka "emulator) image.

Note: If android commands confuse/scare you, check out the android documentation.

#!/bin/bash

# Fix the CircleCI path
function getAndroidSDK(){
  export PATH="$ANDROID_HOME/platform-tools:$ANDROID_HOME/tools:$PATH"

  DEPS="$ANDROID_HOME/installed-dependencies"

  if [ ! -e $DEPS ]; then
    cp -r /usr/local/android-sdk-linux $ANDROID_HOME &&
    echo y | android update sdk -u -a -t android-19 &&
    echo y | android update sdk -u -a -t platform-tools &&
    echo y | android update sdk -u -a -t build-tools-20.0.0 &&
    echo y | android update sdk -u -a -t sys-img-x86-android-19 &&
   #echo y | android update sdk -u -a -t addon-google_apis-google-19 && 
    echo no | android create avd -n testAVD -f -t android-19 --abi default/x86 &&
    touch $DEPS
  fi
}

1. The export PATH line is to ensure we have access to all of the Android CLI tools we'll need later in the script.
2. The DEPS=... is used in the if/then block to determine if CircleCI has already provided us with cached dependencies. If so, there's no to download anything!
3. Note that we're explicitly requesting the x86 version of the Android 19 emulator image (sys-img-x86-android-19). The emulator is notoriously slow, and we should use the hardware-accelerated version if at all possible.
4. We create the AVD with the line android create avd ..., with a target of Android 19 and a name of testAVD.
5. If you don't need the Google APIs (e.g., Maps, Play Store, etc.), you don't need to download addon-google_apis-google-19 - hence why it's commented out!

CAVEAT - Because of the way this caching works, if you ever change which version of Android you compile/run against, you need to click the "Rebuild & Clear Cache" button in CircleCI. If you don't, you'll never actually start compiling against the new SDK. You have been warned.

###You shall not pass! (until your tests have run) This section will vary greatly depending on your testing setup, so, moreso than with the rest of this post, YMMV.
This section is assuming you're using a plain vanilla Android JUnit test suite.

test:
    pre:
	    - $ANDROID_HOME/tools/emulator -avd testAVD -no-skin -no-audio -no-window:
			background: true
		- (./gradlew assembleDebug):
			timeout: 1200
		- (./gradlew assembleDebugTest):
			timeout: 1200
		- (source scripts/environmentSetup.sh && waitForAVD)
    override:
		- (./gradlew connectedAndroidTest)

1. The $ANDROID_HOME/tools/emulator starts a "headless" emulator - namely the one we just created.
   a. Running the emulator from the terminal is a blocking command. That's why we are setting the background: true attribute on the emulator command. Without this, we would have to wait anywhere between 2-7 minutes for the emulator to start and THEN build the APK, etc. This way, we kick off the emulator and can get back to building.
2. The two subsequent ./gradlew commands use the Gradle wrapper (gradle +wrapper) to build the code from your /app and androidTest directories, respectively.
3. See below for environmentSetup.sh Part II. Essentially, after building both the app and the test suite, we cannot continue without the emulator being ready. And so we wait.
4. Once the emulator is up and running, we run gradlew connectedAndroidTest, which, as its name suggests, run the tests on the connected Android device. If you're using Espresso or other test libraries, those commands would go here.
   4a. The CircleCI Android docs say that the "standard" way to run your tests is through ADB - ignore them. Gradle is the future and it elides all of those thorny problems that ADB tests have.

####Bash Round 2 As mentioned above, after Gradle has finished building your app and test suite, you'll kind of need the emulator to...y'know...run your tests.

This script relies on the currently-booting AVD's init.svc.bootanim property, which essentially tells us whether the boot animation has finished. Sometimes, it seems like it'll go on forever...

^{will the madness never stop?!}

This snippet can go in the same file as your previous bash script - in that case, you only need one #!/bin/bash - at the top of your file.

#!/bin/bash

function waitAVD {
    (
    local bootanim=""
    export PATH=$(dirname $(dirname $(which android)))/platform-tools:$PATH
    until [[ "$bootanim" =~ "stopped" ]]; do
        sleep 5
        bootanim=$(adb -e shell getprop init.svc.bootanim 2>&1)
        echo "emulator status=$bootanim"
    done
    )
}

Note: This script was adapted from this busy-wait script.

###Results By default, CircleCI will be fairly vague regarding your tests' successes and/or failures. You'll have to go hunting through the very chatty verbose Gradle loggings in order to determine exactly which tests failed. Fortunately, there's a better way - thanks to Gradle!

When you run gradlew connectedAndroidTests, Gradle, when finished, will create a folder called /build/outputs/reports/**testFolderName**/connected in whichever folder you have a build.gradle script in.
So, for example, if your repo was in ~/username/awesomerepo, with a local library in awesome_repo/lib and an app in /awesome_repo/app, the Gradle test artifacts should be in /awesome_repo/app/build/outputs/reports/**testFolderName**/connected.

In this directory, you'll find a little website that Gradle has generated, showing you which test packages and specific tests passed/failed.

If you like, you can tell CircleCI to grab this by placing the following at the top of your circle.yml file:

general: 
    artifacts:
        -/home/ubuntu/**repo_name**/build/outputs/reports/**testFolderName**/connected

You can then peruse your overwhelming success under the Artifacts tab for your CircleCI build - just click on index.html. It should pull up something like this:

##Security, Signing, and Keystores The astute among you will notice that I haven't gone much into the process of signing an Android app. This is mainly for the reason that people trying to set up APK signing fall into 2 categories - Simple and Enterprise.

Enterprise: If you're programming Android for a company, you probably have some protocol regarding where your keystores/passwords can and cannot live - so a general guide such as this won't be much help for you. Simple: If you're not Enterprise and you're not currently wearing a tinfoil hat, your security protocol is probably a little more lax.

In either case, Google and StackOverflow are your friends.

My final word of advice is that CircleCI can encrypt things like keystore passphrases - stuff you might consider passing in plain-text in your buildscript files. Check out CircleCI's Environment Variables doc.

##Finally, Go into your CircleCI settings, add a hook for your Github repo, and then do a git push origin branchName. If the Gradle Gods have smiled upon you, Circle should detect your config files and start building and testing!

Depending on your test suite, tests can take as little as a few minutes or as much as a half-hour to run. Try not to slack off in the meanwhile, but rejoice in having some solid continuous integration!

Stay tuned for a future blog post about using CircleCI to automagically deploy to MavenCentral!

##Flipping to the back of the book...

Below is the full circle.yml as well as environmentSetup.sh for your viewing/copying pleasure:

# Build configuration file for Circle CI
# needs to be named `circle.yml` and should be in the top level dir of the repo

general: 
    artifacts:
        -/home/ubuntu/**repo_name**/build/outputs/reports/**testFolderName**/connected
        
machine:
	environment:
		ANDROID_HOME: /home/ubuntu/android
    java:
        version: oraclejdk6

dependencies:
	cache_directories:
		- ~/.android
        - ~/android
    override:
	    - (echo "Downloading Android SDK v19 now!")
	    - (source scripts/environmentSetup.sh && getAndroidSDK)

test:
    pre:
	    - $ANDROID_HOME/tools/emulator -avd testAVD -no-skin -no-audio -no-window:
			background: true
		- (./gradlew assembleDebug):
			timeout: 1200
		- (./gradlew assembleDebugTest):
			timeout: 1200
		- (source scripts/environmentSetup.sh && waitForAVD)
    override:
	    - (echo "Running JUnit tests!")
		- (./gradlew connectedAndroidTest)

And the accompanying shell scripts:

#!/bin/bash

# Fix the CircleCI path
function getAndroidSDK(){
  export PATH="$ANDROID_HOME/platform-tools:$ANDROID_HOME/tools:$PATH"

  DEPS="$ANDROID_HOME/installed-dependencies"

  if [ ! -e $DEPS ]; then
    cp -r /usr/local/android-sdk-linux $ANDROID_HOME &&
    echo y | android update sdk -u -a -t android-19 &&
    echo y | android update sdk -u -a -t platform-tools &&
    echo y | android update sdk -u -a -t build-tools-20.0.0 &&
    echo y | android update sdk -u -a -t sys-img-x86-android-19 &&
    #echo y | android update sdk -u -a -t addon-google_apis-google-18 && 
    echo no | android create avd -n testAVD -f -t android-19 --abi default/x86 &&
    touch $DEPS
  fi
}

function waitForAVD {
    (
    local bootanim=""
    export PATH=$(dirname $(dirname $(which android)))/platform-tools:$PATH
    until [[ "$bootanim" =~ "stopped" ]]; do
        sleep 5
        bootanim=$(adb -e shell getprop init.svc.bootanim 2>&1)
        echo "emulator status=$bootanim"
    done
    )
}

##References

• Android Dev Tools - Migrating to Gradle
• Android-Gradle Plugin User Guide
• Android Developers Gradle Project walkthrough
• android CLI tool
• Gradle Wrapper docs
• CircleCI Android Docs
  • CircleCI background processes
  • circle.yml dissected
• StackOverflow.com