This document demonstrates the "CustomEndpointAddress" feature of Azure Event Hubs Java SDK with AMQP WebSockets transport.
The custom endpoint address refers to a developer-provided endpoint address resolvable to EventHubs or configured to route traffic to EventHubs.
The document uses an AppGateway configured to route traffic to its "Public IP" to an EventHubs, so AppGateway's "PublicIP" is the custom endpoint address.
Before going into details, an important note -
If you're using Microsoft internal Azure subscription, you won't be able to establish a connection to AppGateway "Public IP" outside corpnet due to the company's security policies. You can use an MSDN or personal Azure subscription.
The Java code to connect to port 443 of AppGateway "Public IP" (that underneath established connection to the Event Hub instance) is shown below.
-
The "Public IP" of the AppGateway is provided through
customEndpointAddress(..)setter. -
The Application Gateway natively supports Web Socket; we set the transport type as
AMQP_WEB_SOCKETSthroughtransportType(..)setter.
import com.azure.core.credential.AzureNamedKeyCredential;
import com.azure.core.amqp.AmqpTransportType;
import com.azure.messaging.eventhubs.EventHubClientBuilder;
import com.azure.messaging.eventhubs.EventHubProducerClient;AzureNamedKeyCredential credential = new AzureNamedKeyCredential(
"RootManageSharedAccessKey",
"<Shared Access Policy Primary or Secondary Key from the Portal>");EventHubProducerClient producer = new EventHubClientBuilder()
.credential("<eventubs-namespace>.servicebus.windows.net", "<eventhubs-name>", credential)
.customEndpointAddress("https://<public-ip-of-app-gateway>") // e.g. "https://20.232.196.115"
.transportType(AmqpTransportType.AMQP_WEB_SOCKETS)
.buildProducerClient();During the dev phase, developers often use a self-signed certificate for HTTPS.
When establishing an HTTPS connection to the AppGateway's "Public IP" on port 443 configured with a self-signed PFX certificate, Java expects the following -
- The Java Keystore has the CRT (client certificate) version of the self-signed PFX certificate.
- The subject alternative name (SAN) field in the PFX and CRT has the value
IP Address: <IPv4 address>where the IPv4 address is the AppGateway's "Public IP" address (e.g., 20.232.196.115).
We start by creating a dummy self-signed PFX certificate with some random DNS field (e.g., contso.com), we use this certificate while creating the Application Gateway instance.
Later, once the AppGateway instance is created and the "Public IP" is allocated, we'll replace this certificate with a new self-signed PFX certificate with the subject alternative name (SAN) as the "Public IP".
Let's use openssl tool to create the dummy Self-signed PFX certificate with contoso.com as the DNS field. Start by creating the openssl config file config1.txt with the content below -
[req]
distinguished_name = req_distinguished_name
x509_extensions = v3_req
prompt = no
[req_distinguished_name]
C = US
ST = WA
L = Redmond
O = Contoso
OU = JavaSDK
CN = DevExperience
[v3_req]
keyUsage = keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = contoso.com
use this config config1.txt and generate keypair, cert-1.pem
openssl req -x509 -nodes -keyout ./cert-1.pem -out ./cert-1.pem -days 365 -newkey rsa:2048 -config ./config1.txt
and export the keypair as PFX, cert-1.pfx.
openssl pkcs12 -export -out ./cert-1.pfx -in ./cert-1.pem -name "AppGateway Dummy Cert" -passout pass:
<pfx-password>
We'll create and configure the AppGateway; In the end the Application Gateway will have -
- A "PublicIP".
- A "listener" for the frontend port 443 with the self-signed PFX certificate cert-1.pfx for HTTPS.
- A "backend-pool" that uses FQDN (
<eventubs-namespace>.servicebus.windows.net) to refer to the Web Socket enabled (enabled by default) Event Hubs. - A "routing-rule" that associates the "listener" with the "backend-pool".
In the first step, select to create a new virtual network (with default values)
In the frontend configuration page, opt-into create a new public IP.
In the backend configuration page, create a new backend-pool and add the FQDN of the event hub namespace (in the form <eventubs-namespace>.servicebus.windows.net).
In the configuration page, select add a new "routing rule".
The rule allows
- Specifying "listener" to open frontend port 443 for the "Public IP" and set PFX certificate for HTTPS.
- Specifying "backend settings" to flow the traffic from frontend port 443 to backend port 443 of FQDN (i.e.,
<eventubs-namespace>.servicebus.windows.netin the "backend-pool").
For the "listener" - specify the port as 443, select HTTPS and upload the certificate cert-1.pfx created in the previous section.
In the "Backend targets" tab, add "new backend settings".
- Specify protocol as HTTPS, port as 443.
- Choose "Yes" for 'well-known CA certificate'.
- Choose "Yes" for 'Override with new host name'.
- Select the option 'Pick Hostname from the backend target'.
Now we can proceed and create the Application Gateway. It takes a couple of minutes to get the instance ready. Once ready, go to the "Overview" page to see the "Public IP"
The Portal gives an option to test if the backend is healthy, i.e., if the target (i.e., <eventubs-namespace>.servicebus.windows.net) can respond to HTTPS GET on port 443.
If all the configurations are correct, then the "Public IP" will be reachable at port 443 through the browser. Of course, it is expected to get the cert not secure warning as we used a self-signed PFX certificate.
At this point, if we try the Java code from the initial section, it fails with SSLHandShakeException.
EventHubProducerClient producer = new EventHubClientBuilder()
.credential("<eventubs-namespace>.servicebus.windows.net", "<eventhubs-name>", credential)
.customEndpointAddress("https://<public-ip-of-app-gateway>") // e.g. "https://20.232.196.115"
.transportType(AmqpTransportType.AMQP_WEB_SOCKETS)
.buildProducerClient();The cert-1.pfx is a self-signed certificate, i.e., a well-known CA does not issue it, so Java will not be able to validate it using its trusted KeyStore. This leads to the below connection exception -
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
While we can add the CRT (client certificate) version of the cert-1.pfx into the Java Keystore, the connection will fail with a new connection exception -
javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names matching IP address <AppGateway's Public IP Address> found
This is because we're trying to make an SSL connection to the IP address (e.g., 20.232.196.115) and the Java's cert validation phase could not find a cert in the KeyStore with this "Public IP" as subject alternative name (SAN) field.
To solve this, we need to create a self-signed PFX with SAN as AppGateway's "Public IP", update the AppGateway listener for 443 to use this PFX, then import the CRT version of the same PFX to Java KeyStore.
Let's create a self-signed certificate with the "Public IP" address of the AppGateway as Subject Alternative Name (SAN).
Create the openssl config file config2.txt with the content below. Make sure to change the placeholder <AppGateway Public IP Address> with the "Public IP" of AppGateway.
[req]
distinguished_name = req_distinguished_name
x509_extensions = v3_req
prompt = no
[req_distinguished_name]
C = US
ST = WA
L = Redmond
O = Contoso
OU = JavaSDK
CN = DevExperience
[v3_req]
keyUsage = keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
IP.1 = <AppGateway Public IP Address>
use this config config2.txt and generate keypair, cert-2.pem
openssl req -x509 -nodes -keyout ./cert-2.pem -out ./cert-2.pem -days 365 -newkey rsa:2048 -config ./config2.txt
and export the keypair as PFX, cert-2.pfx.
openssl pkcs12 -export -out ./cert-2.pfx -in ./cert-2.pem -name "AppGateway SAN Cert" -passout pass:
<pfx-password>
Go to Application Gateway in the portal, update the "listener-443" we created earlier to use the new self-signed PFX certificate (replacing cert-1.pfx with cert-2.pfx).
We can export the CRT (client certificate) from the cert-2.pfx using openssl
openssl pkcs12 -in ./cert-2.pfx -out cert-2.crt -nokeys -clcerts
then import it to the Java KeyStore.
keytool -import -file cert-2.crt -keystore "C:\Program Files\Zulu\zulu-11\lib\security\cacerts" -alias AppGatway-SelfSigned-Cert
Note: To find the absolute path to the Keystore
- Use mvn --version to print the Java Home (below the value of runtime:)
C:\code> mvn --version
Apache Maven 3.8.6 (...)
Maven home: C:\apache-maven-3.8.6
Java version: 11.0.15, vendor: Azul Systems, Inc., runtime: C:\Program Files\Zulu\zulu-11
- The Java KeyStore will at "\lib\security\cacerts" relative to the Java Home e.g. "C:\Program Files\Zulu\zulu-11\lib\security\cacerts".
At this point, the java code can connect to Event Hubs through the AppGateway Public IP address.
List<EventData> telemetryEvents = Arrays.asList(
new EventData("Roast beef".getBytes(UTF_8)),
new EventData("Cheese".getBytes(UTF_8)),
new EventData("Tofu".getBytes(UTF_8)),
new EventData("Turkey".getBytes(UTF_8)));
AzureNamedKeyCredential credential = new AzureNamedKeyCredential("RootManageSharedAccessKey",
"<Shared Access Policy Primary or Secondary Key from the Portal>");
EventHubProducerClient producer = new EventHubClientBuilder()
.credential("<eventubs-namespace>.servicebus.windows.net", "<eventhubs-name>", credential)
.customEndpointAddress("https://<public-ip-of-app-gateway>") // e.g. "https://20.232.196.115"
.transportType(AmqpTransportType.AMQP_WEB_SOCKETS)
.buildProducerClient();
EventDataBatch currentBatch = producer.createBatch();
for (EventData event : telemetryEvents) {
if (currentBatch.tryAdd(event)) {
continue;
}
producer.send(currentBatch);
currentBatch = producer.createBatch();
if (!currentBatch.tryAdd(event)) {
System.err.printf("Event is too large for an empty batch. Skipping. Max size: %s. Event: %s%n",
currentBatch.getMaxSizeInBytes(), event.getBodyAsString());
}
}
producer.send(currentBatch);After testing, you can remove the CRT from the Keystore
keytool.exe -delete -keystore "C:\Program Files\Zulu\zulu-11\lib\security\cacerts" -alias AppGatway-SelfSigned-Cert
While this document uses the "Shared Access Policy Primary or Secondary Key obtained from the Portal" for authentication, there are other ways to authenticate. For example authentication using Service Princpal is documented in the Gist EventHubsServicePrincipal.md
The Azure Private Endpoint feature allows applying granular network access control to Azure Event Hubs.
As long as the network and environment configurations are correct and the client host can resolve the Event Hubs, the Azure SDK will be able to communicate with the Event Hubs.
Refer to the below Gist for details on Private IP enabled Event Hubs, with typical public access restricted but allowed to access using Application Gateways "Public IP."
PrivateEndpointEnabledEvent Hubs.md
So far, in this and referenced Gists, we used the IPv4 (e.g., https://20.232.196.115) of AppGateway as the CustomEndpointAddress.
If you own a root domain name (e.g., anuchan.us), we can use Azure DNS zone to create a DNS record (e.g., apg-eh.anuchan.us) under the root and associate it to this IPv4 address, enabling us to use custom domain as CustomEndpointAddress.
CustomDomainCustomEndpointAddress.md
The library natively supports communication through HTTP Proxy; this Gist HttpProxyCustomEndpointAddress.md will walk you through the setup.