I started this as an experiment - what can I do with an Azure IoT Edge module? Containers are inherently moveable, so I wanted to see if I could move an Azure IoT Edge module into a pod on a Kubernetes cluster. I succeeded in taking our sample temperature sensor image and running it as a pod.
You probably wouldn't. I haven’t come up with a use case for putting a single IoT Edge module container into a cluster, but I could think of a case where you have existing module image you would like to run on a device, but that device is too small or otherwise unable to run as an IoT Edge device. This document shows how software running the module SDK can be configured to connect to an external device.
A leaf device is a downstream device that has an identity created with the Azure IoT Hub cloud service, typically running the Azure IoT Device Client SDK, using the Azure IoT Hub as a gateway. A leaf module is a term I coined for this document. It is a downstream device that has a module identity created with the Azure IoT Hub cloud service, typically running the Azure IoT Module Client SDK.
The leaf device and leaf module differ little from Azure IoT Edge's perspective. Messages from leaf devices can be routed on the edge the same as a module. There are three primary differences:
- A module's identity is tied to the IoT Edge's device id. Any data associated with the module would be recorded as events on the Azure IoT Edge device. Creating a leaf device would generally make more sense for any IoT application where a separate device identity is needed.
- No C2D messages for modules. This is not currently supported for modules on Azure IoT Edge.
- The SDK used to build the application. Devices typically use the Device Client SDK, and modules typically use the Module Client SDK.
In normal operation, the module's identity and trust bundle are provided for by the IoT Edge Security daemon. Access to the daemon is not allowed outside the system running the Edge runtime. We need to get module identity and a root of trust differently.
For Linux systems, the minimum a leaf module needs is a connection string and a valid CA certificate from the EdgeHub's certificate chain. The connection string is passed in the EdgeHubConnectionString environment variable. The certificate must be a valid file accessible to the application, either installed in the certificate store, or the path to the file must be passed in via an environment variable named EdgeModuleCACertificateFile.
I'm using a pre-built module image, so I can't install the certificate, I will give the SDK a file it can read and set the environment variable.
The IoT Edge device is set up in the same way it would be set up as a transparent gateway. I used the documentation to set up a transparent gateway to configure my device.
In my setup, I started with a basic and convenient configuration: an Ubuntu 18.04 machine as my IoT Edge device and set up minikube on it.
Here are the steps I took:
-
Created a new edge device in the Azure Portal.
-
Created an empty deployment.
A deployment is required to start the EdgeHub module so the edge device will act as a gateway to downstream devices. I chose an empty deployment (no modules except EdgeAgent and EdgeHub) for convenience.
-
On the Ubuntu machine,
-
Generated certificates for a transparent gateway. Followed the instructions from here. In the step where you create the edge device certificate, use the FQDN for the system. i.e.
./certGen.sh create_edge_device_certificate "<my Ubuntu system's FQDN>"This is required to be the same as the GateWayHostName in the module's connection string
-
Added the certificate locations to /etc/iotedge/config.yaml, as described here.
-
Updated the config.yaml file with the new device's connection string.
-
-
Created a module identity using the IoT extension for Azure CLI:
az iot hub module-identity create -n <iot-hub> -d <edge-device> -m remote-sensorThis command creates a module identity without requiring a module specification in the device. If you add a module through the Azure Portal, the module will have a specification and will be launched by the Edge runtime.
-
Got the module's connection string via the Azure CLI:
az iot hub module-identity show-connection-string -n <iot-hub> -d <edge-device> -m remote-sensorAt this point, I have a module named "remote-sensor", its connection string, and all the certificates needed to connect to edge hub. The edge device configuration is complete.
I started by installing secrets into my minikube cluster:
-
Created a namespace.
kubectl create namespace temperature-sensorI created a namespace in my cluster to isolate it from other applications. This is not required.
-
Contructed the leaf module connection string and wrote it to a file.
echo -n '<remote-sensor connection string>;GatewayHostName=<my Ubuntu system's FQDN>' connection_string.txt -
Created the connection string secret.
kubectl create secret generic -n temperature-sensor connection-string --from-file=./connection_string.txtThis secret may now be referenced by name: connection-string, with key "connection_string.txt". This will be mounted into the pod as an environment variable for the Module Client SDK to use.
The next secret required is the root CA certificate for our Edge Device. I chose to store this certificate as a generic secret, but there may be better ways in Kubernetes to handle this.
-
Changed directories to where the generated certificates were created.
-
Created the CA certificate secret using the Owner CA certificate for the device.
kubectl create secret generic -n temperature-sensor ca-cert --from-file=certs/azure-iot-test-only.root.ca.cert.pemThis secret may now be referenced by the name "ca-cert" with key "azure-iot-test-only.root.ca.cert.pem". This will be mounted as a file in the pod, and then we will set an environment variable for the Module Client SDK to locate it.
-
Created my pod.
First, I needed to create a YAML file with the pod specification, I creatively named this temperature-sensor.yaml. This is the content of that file:
apiVersion: v1
kind: Pod
metadata:
name: temperature-sensor
namespace: temperature-sensor
spec:
containers:
- name: temperature-sensor
image: mcr.microsoft.com/azureiotedge-simulated-temperature-sensor:1.0
env:
- name: EdgeHubConnectionString
valueFrom:
secretKeyRef:
name: connection-string
key: connection_string.txt
- name: EdgeModuleCACertificateFile
value: /etc/secret/azure-iot-test-only.root.ca.cert.pem
volumeMounts:
- name: ca-cert
mountPath: "/etc/secret"
readOnly: true
volumes:
- name: ca-cert
secret:
secretName: ca-certWhat's happening in this pod spec?
- It has a name of temperature-sensor and put it in the same namespace (temperature-sensor) as the module secrets
- It creates one container in this pod, same name and uses the sample simulated temperature sensor module image.
- It mounts into the container the ca-cert secret as a file in "/etc/secrets"
- It passes the container an environment variable named "EdgeHubConnectionString" set to the value of the connection-string secret.
- It passes the container another environment variable named "EdgeModuleCACertificateFile" set to the location of certificate file.
All that was left is to finally create this pod.
kubectl apply -f temperature-sensor.yaml
If everything goes correctly, the logs look like this:
$ kubectl logs -n temperature-sensor temperature-sensor
[2019-01-10 02:12:16 : Starting Module
[01/10/2019 02:12:16.960 AM] Main()
Initializing simulated temperature sensor to send 500 messages, at an interval of 5 seconds.
To change this, set the environment variable MessageCount to the number of messages that should be sent (set it to -1 to send unlimited messages).
Using transport Amqp_Tcp_Only
Successfully initialized module client.
01/10/2019 02:12:20> Sending message: 1, Body: [{"machine":{"temperature":21.219922209377366,"pressure":1.0250544289164087},"ambient":{"temperature":21.287290462193681,"humidity":26},"timeCreated":"2019-01-10T02:12:20.5461996Z"}]
01/10/2019 02:12:25> Sending message: 2, Body: [{"machine":{"temperature":21.529869530363879,"pressure":1.0603648832060115},"ambient":{"temperature":21.300895716902286,"humidity":26},"timeCreated":"2019-01-10T02:12:25.7930952Z"}]