Skip to content

Instantly share code, notes, and snippets.

@dhet

dhet/labnote-17-11-24.md

Last active Nov 25, 2017
Embed
What would you like to do?
Labnote 17-11-24: Embedded OpenDDS via Linux Containers

Embedded OpenDDS via Linux Containers

Ultimate Goal

Deploy OpenDDS programs within containers running on different Raspberry Pis connected via a LAN switch. The OpenDDS participants need to be able to communicate with each other using RTPS. The programs must be able to use sensor data from sensors connected to the Raspberry Pi's GPIO interface. One participant should make use of the Raspberry Pi's camera.

Test Environment

  • OpenDDS 3.12
  • Three Raspberry Pis v3 Model B
  • Docker 17.11.0-ce

Current Status

  • OpenDDS programs run within cross platform containers (ARM/x86), but programs within containers can't communicate with each other yet
  • OpenDDS programs run on several machines within a LAN communicating with each other (without containers)

Challenges

Containers: Raspberry Pi's Processor Architecture

Raspis are ARM systems. In order for Docker images to be compatible to ARM systems they need to be built on ARM systems. However, Raspis are slow and builds take long to complete. We wanted to run the builds on our laptops because they're faster. So we looked for ways to build cross platform Docker images. We found a way: use docker base images that utilize QEMU for compatibility. We tried several images before finding one that worked:

  • armv7/armhf-ubuntu: no x86 support
  • armv7/armhf-debian: no x86 support
  • ryankurte/docker-rpi-emu: Didn't run on Raspi for some reason, hung up when starting the container
  • resin/armv7hf-debian: Perl path problems when building. A Perl library used in the DDS build script was not available. When entering raw base image: "Command not found:" error
  • resin/rpi-raspbian: Same Parl path problems before but we got it to work by extending the dockerfile by an additional apt install command, adding the missing library.

We ended up using resin/rpi-raspbian because that was the only one that worked.

A requirement for building cross platform docker containers is to execute the following command beforehand: docker run --rm --privileged multiarch/qemu-user-static:register --reset Source

Containers: OpenDDS Uses Random Ports

OpenDDS uses random ports for service discovery and communication. Docker blocks any incoming network traffic into the container unless the used ports are explicitly bound to the host. This is a problem because a DDS participant's port are not known in advance. We found a way to specify ports from a config file but this has several issues:

  • cross container, multi host communication still doesn't work (still looking for solutions)
  • this approach defeats the purpose of location transparent services (services need knowledge of their network interface)

Due to this, we suspect that dynamic service discovery does not work with containers. Maybe this is a limiation of OpenDDS and it works for other implementations.

Open Questions

  • Docker
    • How does QEMU work?
    • How do the QEMU enabled base images work?
    • What does the command required before building do?

Milestones

  • Single host DDS communication
  • Multi host DDS communication
  • Single host / cross container DDS communication
  • Multi host / cross container DDS communication
  • Control GPIO pins from DDS client
  • Sensor input on host A causes sensor output on host B
  • Control GPIO pins from within container
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment