Skip to content

Instantly share code, notes, and snippets.

@jamesonwilliams
Last active January 29, 2021 07:06
Show Gist options
  • Save jamesonwilliams/e5a78e09cbe6e81a5c0b1ccc7f1ff599 to your computer and use it in GitHub Desktop.
Save jamesonwilliams/e5a78e09cbe6e81a5c0b1ccc7f1ff599 to your computer and use it in GitHub Desktop.
Using AWS Java SDK V2 from Android

Using the V2 AWS Java SDK from Android

The V2 AWS Java SDK can be used from Android, with a few caveats.

Note: currently, this is limited to API 26 and higher (60% of production devices, as of 06JUL2020.)

Other Note: this document is not production guidance for application builders. These are some of my personal notes after some study and expirementation.

Use Android Gradle Plugin 4+

The release of Android Gradle Plugin 4 has made the use of the V2 Java SDK realistic on Android.

Starting with AGP 4, Java 8+ APIs (like java.util.Optional) are desugared at build time. Jake Warton discusses this topic in more depth, here.

Android's documentation on Java 8+ API desugaring support is here.

If you were to use an earlier version of the Android Gradle Plugin, Java 8 APIs like Optional would require a minSdk of 24 at runtime. API 24 is too aggressive of a minSdk for most production applications. Android Studio 4 has lifted this limitation, by solving the problem at build-time.

To enable core library desugaring, add a compile option:

android {
    compileOptions {
        // Add this line.
        coreLibraryDesugaringEnabled true
    }
}

And specify a version of the desugar_jdk_libs to use:

dependencies {
    // Add this line.
    coreLibraryDesugaring 'com.android.tools:desugar_jdk_libs:1.0.9'
}

At present, R8 desugaring does not support ThreadLocal.withInitial(...), which is used by the SDK. I've requested that Google support it, and/or Amazon to stop using it. This currently limits the SDK to use on API 26 or higher.

Don't package some META-INF

You'll encounter build-time errors like this:

More than one file was found with OS independent path 'META-INF/INDEX.LIST'

This is tracked in Issue #1940.

A workaround is to add packagingOptions to your module-level build.gradle, to ignore some META-INF files that are dragged in by the SDK:

android {
    packagingOptions {
        exclude 'META-INF/INDEX.LIST'
        exclude 'META-INF/io.netty.versions.properties'
        exclude 'META-INF/DEPENDENCIES'
    }
}

Use Java 1.8 Compatibility

You must use Java 1.8 source and target compatibility.

The V2 Java SDK uses Java 8 features throughout: e.g. java.util.Optional as mentioned earlier.

android {
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
    
    // If consuming the SDK from Kotlin, add this, too.
    kotlinOptions {
        jvmTarget = '1.8'
    }
}

HTTP Runtimes

In short, the V2 SDK uses the Apache HTTP Client in its default HTTP runtime. This does not work "out of the box" on Android.

However, the V2 SDK allows you to swap HTTP client runtimes, by providing your own SdkHttpClient implementaiton. The V1 Java SDK did not provide this capability.

The V2 SDK ships with an UrlConnectionHttpClient, and the V2 SDK team recommends it for use on Android.

It is also possible to implement an SdkHttpClient which uses OkHttp. A proof-of-concept is provided, below.

The simplest solution is to use the UrlConnectionHttpClient that ships with the V2 SDK.

This is the solution suggested by the V2 Java SDK team in Issue #1180.

To use it in place of the Apache runtime, you need to add it as a dependency in your module-level build.gradle:

dependencies {
    implementation 'software.amazon.awssdk:iot:2.13.49'
    // Add this line.
    implementation 'software.amazon.awssdk:url-connection-client:2.13.49'
}

And then include it when you build a service client:

val credentials =
    AwsSessionCredentials.create(accessKey, secretKey, sessionToken)
val iot = IotClient.builder()
    .region(Region.US_EAST_1)
    .credentialsProvider(StaticCredentialsProvider.create(credentials))
    .httpClient(UrlConnectionHttpClient.create())
    .build()

OkHttp as HTTP Runtime

You can also use OkHttp as the HTTP runtime.

The V2 AWS Java SDK does not ship with an OkHttp implementation of the SdkHttpClient interface.

There is an outstanding feature request for this, in issue 851.

The implementation below has been shown to function for a simple IoT list API.

Add to module-level build.gradle:

dependencies {
    implementation 'com.squareup.okhttp3:okhttp:4.7.2'
    implementation 'org.conscrypt:conscrypt-android:2.4.0'
}

Use the OkHttp-based SdkHttpClient implementation when constructing an AWS service client:

val credentials =
    AwsSessionCredentials.create(accessKey, secretKey, sessionToken)
val iot = IotClient.builder()
    .region(Region.US_EAST_1)
    .credentialsProvider(StaticCredentialsProvider.create(credentials))
    .httpClient(SdkOkHttpClient.create()) // Note this line here.
    .build()

Create SdkOkHttpClient.java:

package v2sdk.sample;

import java.io.IOException;
import java.io.InputStream;
import java.util.Arrays;
import java.util.List;
import java.util.Map;

import okhttp3.Cache;
import okhttp3.Call;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;
import okhttp3.ResponseBody;

import software.amazon.awssdk.http.AbortableInputStream;
import software.amazon.awssdk.http.ExecutableHttpRequest;
import software.amazon.awssdk.http.HttpExecuteRequest;
import software.amazon.awssdk.http.HttpExecuteResponse;
import software.amazon.awssdk.http.SdkHttpClient;
import software.amazon.awssdk.http.SdkHttpRequest;
import software.amazon.awssdk.http.SdkHttpResponse;

final class SdkOkHttpClient implements SdkHttpClient {
    private static final List<String> IGNORED_HEADERS = Arrays.asList("Content-Type", "Host");

    private final OkHttpClient okHttpClient;

    private SdkOkHttpClient(OkHttpClient okHttpClient) {
        this.okHttpClient = okHttpClient;
    }

    static SdkOkHttpClient create() {
        return new SdkOkHttpClient(new OkHttpClient());
    }

    @Override
    public ExecutableHttpRequest prepareRequest(HttpExecuteRequest request) {
        return HttpRequest.create(okHttpClient, request);
    }

    @Override
    public String clientName() {
        return SdkOkHttpClient.class.getSimpleName();
    }

    @Override
    public void close() {
        okHttpClient.dispatcher().executorService().shutdown();
        okHttpClient.connectionPool().evictAll();
        Cache cache = okHttpClient.cache();
        if (cache == null) {
            return;
        }
        try {
            cache.close();
        } catch (IOException failureToClose) {
            // Sigh.
        }
    }

    private static final class HttpRequest implements ExecutableHttpRequest {
        private final OkHttpClient okHttpClient;
        private final HttpExecuteRequest request;

        private HttpRequest(OkHttpClient okHttpClient, HttpExecuteRequest request) {
            this.okHttpClient = okHttpClient;
            this.request = request;
        }

        static HttpRequest create(OkHttpClient okHttpClient, HttpExecuteRequest request) {
            return new HttpRequest(okHttpClient, request);
        }

        @Override
        public HttpExecuteResponse call() throws IOException {
            return execute(request);
        }

        @Override
        public void abort() {
        }

        private HttpExecuteResponse execute(HttpExecuteRequest request) throws IOException {
            Request okHttpRequest = toOkHttpRequest(request);
            Call okHttpCall = okHttpClient.newCall(okHttpRequest);
            Response okHttpResponse = okHttpCall.execute();
            return createResponse(okHttpResponse, okHttpCall);
        }

        private Request toOkHttpRequest(HttpExecuteRequest request) throws IOException {
            SdkHttpRequest sdkHttpRequest = request.httpRequest();
            int contentLength =
                sdkHttpRequest.firstMatchingHeader("Content-Length")
                    .map(Integer::parseInt)
                    .orElse(0);
            byte[] bytes = new byte[contentLength];

            if (request.contentStreamProvider().isPresent()) {
                InputStream requestStream = request.contentStreamProvider().get().newStream();
                requestStream.read(bytes, 0, contentLength);
            }
            Request.Builder requestBuilder = new Request.Builder()
                .url(sdkHttpRequest.getUri().toString());

            final RequestBody requestBody;
            switch (sdkHttpRequest.method()) {
                case PATCH:
                case PUT:
                case POST:
                    requestBody = RequestBody.create(bytes);
                    break;
                default:
                    requestBody = null;
            }
            requestBuilder.method(sdkHttpRequest.method().name(), requestBody);

            // Add headers.
            for (Map.Entry<String, List<String>> headers : sdkHttpRequest.headers().entrySet()) {
                for (String value : headers.getValue()) {
                    if (!IGNORED_HEADERS.contains(headers.getKey())) {
                        requestBuilder.addHeader(headers.getKey(), value);
                    }
                }
            }

            return requestBuilder.build();
        }

        private HttpExecuteResponse createResponse(Response okHttpResponse, Call okHttpCall) {
            SdkHttpResponse response = SdkHttpResponse.builder()
                .statusCode(okHttpResponse.code())
                .statusText(okHttpResponse.message())
                .headers(okHttpResponse.headers().toMultimap())
                .build();
            ResponseBody okHttpResponseBody = okHttpResponse.body();

            AbortableInputStream responseBody = (okHttpResponseBody == null) ?
                null : AbortableInputStream.create(okHttpResponseBody.byteStream(), okHttpCall::cancel);

            return HttpExecuteResponse.builder()
                .response(response)
                .responseBody(responseBody)
                .build();
        }
    }
}

There may be a way to get this to work. I haven't found it, yet.

Out-of-the-box, when simply constructing an IoT client, with no custom bells and whistles, I get this exception at runtime:

Caused by: java.lang.NoSuchFieldError: No static field INSTANCE of type Lorg/apache/http/conn/ssl/AllowAllHostnameVerifier; in class Lorg/apache/http/conn/ssl/AllowAllHostnameVerifier; or its superclasses (declaration of 'org.apache.http.conn.ssl.AllowAllHostnameVerifier' appears in /system/framework/framework.jar!classes3.dex)

In Issue #1180, the V2 Java SDK Team says:

Unfortunately, we won't be able to fix Apache http client for Android use case. Can you try with http url connection client?

The Apache HTTP Client has a lot of baggage on Android. It was included in the operating system and causes runtime conflicts, on some versions of Android. On other versions, you have to include special flags to use the OS-provided impelmentation.

@jamesonwilliams
Copy link
Author

jamesonwilliams commented Jan 29, 2021

Hey @kapil-mangtani, glad you found it useful!

AWS currently has no plans to expand Android support in the V2 Java SDK. Likewise, I don't think Google is currently prioritizing the de-sugaring rules for these various Java 8 APIs.

Realistically, you have a few other options.

  1. Don't use an AWS SDK on the Android device. Instead, call your AWS services from an AWS Lambda function, using a traditional SDK. Expose that functionality to your app via API Gateway. Call the API Gateway directly from a regular HTTP client.
  2. Use the Android SDK. It's old, and it only has support for a handful of services. However, it does run on most Android API levels that anyone would reasonably want to support.
  3. Use Amplify Android, if it covers your use case.
  4. Try using the V1 Java SDK. I haven't actually tried this, yet!

You might also want to checkout my blog post about this topic, A Brief History of AWS Mobile SDKs and How to Use Them in 2021.

We do have a fifth, more modern solution on the way. But it's still several months off, and I'm not allowed to talk about it, yet. 😃

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment