Skip to content

Instantly share code, notes, and snippets.

@tylerlum
Last active September 24, 2023 09:33
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tylerlum/51c6c246419a1ecc107e0b7a28514b9f to your computer and use it in GitHub Desktop.
Save tylerlum/51c6c246419a1ecc107e0b7a28514b9f to your computer and use it in GitHub Desktop.
Tyler Lum Google Summer of Code 2019 with Open Robotics

Google Summer of Code 2019 with Open Robotics

Author: Tyler Lum

Overview

This GitHub Gist outlines the open-source software work that I have done with Open Robotics through the Google Summer of Code program. This includes:

  • a high-level description of the work that I have done

  • a list of the code changes that I have made

  • a list of the software and tools that I used

  • a detailed description of my projects

Open Robotics

Figure 1: Open Robotics is an open-source organization that builds and maintains robotics software and hardware. They work with industry, academia, and government to support education, research, and product development.


Google Summer of Code

Figure 2: Google Summer of Code is a global program that connects open-source organizations with talented student developers. Students spend the summer writing open-source software and connecting with other Google Summer of Code students across the world.

Project Summary

The focus of my work has been writing software for VRX (Virtual RobotX). VRX is an autonomous boat simulator that uses Gazebo and ROS for physics, rendering, and communication. The software that I wrote adds important features for customization and control, enhances the simulator's overall realism, and improves the infrastructure for competitions that use the simulator.

Key_Teleop_2

Figure 3: VRX is an autonomous boat simulator that uses Gazebo and ROS for physics, rendering, and communication.


Rviz

Figure 4: The VRX WAM-V (Wave Adapted Modular Vessel) has many sensors, including lidar, camera, and IMU. The sensor data can be visualized using RVIZ.

Maritime RobotX is a competition hosted by RoboNation and the Office of Naval Research, in which teams develop an autonomous surface vehicle that will perform various tasks, including navigation, perception, and docking. Teams are comprised of students, faculty, industrial partners, and government partners.

Maritime RobotX is RoboNation’s most complex robotic competition to-date. VRX is a simulation environment designed to support the RobotX Challenge by allowing competitors to test and improve their software extensively on a virtual boat.

RobotX

Figure 5: RobotX is RoboNation’s most complex robotic competition to-date.

If you would like to learn more about VRX, you can check out the open-source repository and the VRX highlight video.

Software, Languages, and Tools Used

Software

  • ROS Melodic

  • Gazebo 9

  • Docker + Dockerhub

  • Docker networking

  • OGRE (Object Oriented Graphics Rendering Engine)

  • Mercurial and Git

  • Bitbucket

  • Ubuntu 18.04

Languages

  • C++

  • Python

  • Bash

  • GLSL (GL shading language)

  • Xacro

  • SDF (Spatial Data File)

  • URDF (Universal Robotic Description Format)

Tools

  • Vim

  • Terminator

1. Ocean Reflections, Refractions and Waves

Software Used: C++, OGRE, GLSL

Purpose

The ocean visuals in VRX have always been great. They used a GerstnerWave model for realistic ocean waves. However, for vision-based tasks on the water, ocean reflections are an important factor. Because many autonomous boats have camera sensors for object recognition, having realistic water reflections in the simulation will be important for testing the reliability and effectiveness of these systems. As well, ocean reflections are very visually pleasing and make the simulation feel more realistic. This visual effect was missing in VRX.

Original Ocean

Figure 6: Original Ocean Waves. They looked realistic, but were missing reflection and refraction effects.

Description

Gazebo uses Object Oriented Graphics Rendering Engine (OGRE) for its rendering. After extensive experimenting with OGRE and reading OGRE documentation, I was able to add a simple reflection texture to the water by writing a Gazebo Visual Plugin with OGRE objects.

This involved:

  1. finding the OGRE camera that is being viewed

  2. using render-to-texture to render the camera view onto a texture

  3. modifying the textures using clipping planes and reflections to get the desired reflection and refraction textures

  4. applying this texture to the ocean material

  5. mixing the textures with the current ocean water and wave color with linear interpolation

  6. continuously updating the texture

The last step was tuning the ratio between reflection, refraction, background, and water color in the GLSL shaders.

Maximum Reflection

Figure 7: Early stage reflection. Maximum reflection settings made the ocean look like a mirror.


NEW_Reflection_Integrated

Figure 8: Middle stage reflection. Reflections were integrated with the original ocean waves.

Result

After tuning the ratio between the reflection, refraction, background, and water color, the ocean reflections and refractions looked realistic and subtle. This can be seen from the comparison between real images and simulated images.

In terms of performance, rendering to texture is much more computationally efficient than ray-tracing. An exact measurement will be made soon.

NEW_Subtle_Reflection

Figure 9: Late stage reflection. By adding a fresnel effect, the amount of reflection changes based on angle. This makes the effect much more subtle and realistic.


Real Image

Figure 10: Real image of buoys, with reflections.


Sim_Reflection

Figure 11: Simulated image of buoys, with reflections.

Relevant Commits

2. Automated Evaluation for VRX Competitors

Software Used: Bash, Docker, Dockerhub

Purpose

For the VRX competition, teams will be submitting their control software and their desired sensor and thruster configurations to be used on their boats. Their system will be evaluated on multiple tasks, including perception, navigation, and docking.

The purpose of this project is to set up automated evaluation of the VRX teams, which is secure, reproducible, and easy to use. This evaluation software would be used to run the final VRX competition. Teams could use this open software themselves to evaluate their own systems. The only difference between the teams' testing and the final competition would be the different environmental conditions in the task worlds, including wave amplitude, wind speed, and lighting variation.

Description

I wrote the software for vrx-docker, which is a repository containing scripts and infrastructure that will be used for automatic evaluation of Virtual RobotX (VRX) teams' submissions.

This repository consists of two major components:

  1. The VRX server system, which runs the VRX Gazebo simulation

  2. The VRX competitor system, which runs a team's control software.

For security and reproducibility, the VRX server and the VRX competitor's systems will be run in separate, isolated environments called Docker containers.

Much of the structure was based off of ARIAC Docker, but required significant modifications to match the needs of VRX.

Result

vrx-docker is now a public repository that is able to take in configuration files and Dockerhub images as input and then output detailed information about a team's performance. This includes trial scores, ROS and Gazebo logs, output text files, Gazebo playback files, and trial videos. It has clear documentation in its README that will allow beginners to easily use the software.

vrx-docker-1

Figure 12: Prepare-all scripts to prepare all teams WAM-Vs and task worlds.


vrx-docker-2

Figure 13: Run-all scripts that show when an error has occurred. Detailed description of error stored in a txt file.


vrx-docker-3

Figure 14: Run-all scripts that run the entire competition, storing all log files and scoring.


Relevant Commits

3. WAM-V Customization

Software Used: Python, Roslaunch, URDF

Purpose

VRX has a large suite of sensors, including lidar, GPS, IMU, and camera. It also offered multiple thruster configurations, including T, X, and H. However, customization of the sensor and thruster configuration was very limited.

The purpose of this project is to allow for easy configuration of sensors and thrusters, such that users can easily set how many they want and where they will be located.

Description

To create a custom configuration:

  1. users create a yaml file that outlines their configuration

  2. users pass this in as a parameter to a roslaunch script, which runs a Python script

  3. the Python script parses the yaml files and creates a custom urdf file that can be used directly by Gazebo

Result

Users can now outline their sensor and thruster configurations by creating a yaml file like so:

wamv_camera:
    - name: left_camera
      x: 0.75
      y: 0.3
      z: 1.8
      P: ${radians(15)}
      Y: ${radians(90)}
      post_Y: ${radians(90)}
    - name: right_camera
      x: 0.75
      y: -0.3
      z: 1.8
      P: ${radians(15)}
      Y: ${radians(-90)}
      post_Y: ${radians(-90)}
wamv_imu:
    - name: imu_wamv
wamv_p3d:
    - name: p3d_wamv
lidar:
    - name: lidar_wamv
      z: 1.5

then create a custom urdf file by running:

roslaunch vrx_gazebo generate_wamv.launch thruster_yaml:=<thruster yaml> sensor_yaml:=<sensor yaml>  wamv_target:=<output file>

then use their custom configuration in the simulation by running:

roslaunch vrx_gazebo sandisland.launch urdf:=<output file>

This results in a simple, easy, and reliable way to create custom boat configurations. This is very valuable for both VRX competitors, as well as researchers using the simulation for other autonomous surface vessel projects.

Custom_Sensors_1

Figure 15: Customizable sensor configuration


Custom_Sensors_2

Figure 16: Customizable sensor configuration


Custom_Thrusters_1

Figure 17: Customizable thruster configuration


Custom_Thrusters_2

Figure 18: Customizable thruster configuration

Relevant Commits

4. Thruster Articulation

Software Used: C++, Gazebo ROS Plugin, URDF

Purpose

Many autonomous surface vessels have thrusters whose angle can be controlled on command. Thruster articulation is a thruster's ability to rotate or redirect its thrust. VRX did not have this functionality; its thrusters stayed at a fixed angle, which was limiting to many teams.

The purpose of this project is to add thruster articulation functionality so that teams can control the angle of their thrusters.

Description

To add this functionality, I:

  1. implemented a Gazebo ROS plugin that subscribed to the angle topic

  2. added a rotational joint on the engine

  3. used a PID controller to change the joint angle to match the desired angle

Result

Users can now set the angle of their thruster by publishing to the relevant rostopic, eg. /thrusters/right_thrust_angle.

The maximum angle can be set by adding <maxAngle>${pi/2}</maxAngle> to the plugin.

This functionality can be optionally disabled by adding <enableAngle>false</enableAngle> to the plugin.

Thruster_Articulation

Figure 19: VRX thruster angle can now be controlled on command

Relevant Commits

5. Limit Non-competition Topic Exposure

Software Used: Bash, Roslaunch, Gazebo, ROS, Docker Networking

Purpose

Gazebo exposes an API that allows users to do things like spawn models, move objects, and observe model and link states. This includes Gazebo transport topics, Gazebo ROS services and ROS topics, and VRX specific rostopics. While this is very useful for general use, for the purpose of competition, we want competitors to localize and navigate using their sensors and thrusters, not using this additional information.

The purpose of this project is to allow us to limit the exposure of this additional information optionally.

Description

There are three forms of topics/services we want to optionally hide. Each one is solved with a different solution.

  1. Gazebo transport topics - Through use of vrx-docker, as described in section 2, there will be separate Docker containers for the simulation and the competitor's software. By setting a GAZEBO_IP that is excluded from the Docker network subnet mask, this blocks Gazebo transport topic communication, which stops the competitor from seeing these topics.

  2. Gazebo ros topics and services - The same solution above cannot be used, as we need to allow some rostopics to be used for thruster and sensor communication. Gazebo recently exposed a new parameter called enable_ros_network, which we set to false. This hides the exact Gazebo services and topics we want to hide.

  3. VRX debugging ros topics - We set an environment variable VRX_DEBUG that decides if the debugging topic information is shown or not.

In implementing (2), the Gazebo service we used to spawn the WAM-V model was blocked. To work around this issue, I wrote a new script called spawn_wamv.bash which uses gz model to spawn the model instead.

Result

Now, we can run roslaunch vrx_gazebo sandisland.launch to run the simulation with all topics and services or run roslaunch vrx_gazebo sandisland.launch non_competition_mode:=false to hide them.

➜  vrx-docker rostopic list
/clock
/gazebo/link_states
/gazebo/model_states
/gazebo/parameter_descriptions
/gazebo/parameter_updates
/gazebo/set_link_state
/gazebo/set_model_state
/joint_states
/rosout
/rosout_agg
/thrusters/left_thrust_angle
/thrusters/left_thrust_cmd
/thrusters/right_thrust_angle
/thrusters/right_thrust_cmd
/vrx
/vrx/debug/wind/direction
/vrx/debug/wind/speed
/vrx/dock_2016_placard1/shuffle
/vrx/dock_2016_placard2/shuffle
/vrx/dock_2016_placard3/shuffle
/vrx/dock_2018_placard1/shuffle
/vrx/dock_2018_placard2/shuffle
➜  vrx-docker rosservice list
/gazebo/apply_body_wrench
/gazebo/apply_joint_effort
/gazebo/clear_body_wrenches
/gazebo/clear_joint_forces
/gazebo/delete_light
/gazebo/delete_model
/gazebo/get_joint_properties
/gazebo/get_light_properties
/gazebo/get_link_properties
/gazebo/get_link_state
/gazebo/get_loggers
/gazebo/get_model_properties
/gazebo/get_model_state
/gazebo/get_physics_properties
/gazebo/get_world_properties
/gazebo/pause_physics
/gazebo/reset_simulation
/gazebo/reset_world
/gazebo/set_joint_properties
/gazebo/set_light_properties
/gazebo/set_link_properties
/gazebo/set_link_state
/gazebo/set_logger_level
/gazebo/set_model_configuration
/gazebo/set_model_state
/gazebo/set_parameters
/gazebo/set_physics_properties
/gazebo/spawn_sdf_model
/gazebo/spawn_urdf_model
/gazebo/unpause_physics
/gazebo_gui/get_loggers
/gazebo_gui/set_logger_level
/rosout/get_loggers
/rosout/set_logger_level

Above is the topic and service list showing everything from running roslaunch vrx_gazebo sandisland.launch.


➜  vrx-docker rostopic list
/clock
/joint_states
/rosout
/rosout_agg
/thrusters/left_thrust_angle
/thrusters/left_thrust_cmd
/thrusters/right_thrust_angle
/thrusters/right_thrust_cmd
/vrx
/vrx/debug/wind/direction
/vrx/debug/wind/speed
/vrx/dock_2016_placard1/shuffle
/vrx/dock_2016_placard2/shuffle
/vrx/dock_2016_placard3/shuffle
/vrx/dock_2018_placard1/shuffle
/vrx/dock_2018_placard2/shuffle
➜  vrx-docker rosservice list
/gazebo/get_loggers
/gazebo/set_logger_level
/gazebo_gui/get_loggers
/gazebo_gui/set_logger_level
/rosout/get_loggers
/rosout/set_logger_level

Above is the limited topic and service list from running roslaunch vrx_gazebo sandisland.launch non_competition_mode:=false.

Relevant Commits

6. WAM-V Visual Enhancements

Software Used: Xacro, URDF, SDF, Mesh, Roslaunch, Gazebo

Purpose

While the WAM-V and its environment were very visually pleasing, the WAM-V sensors were left as simple white shapes like boxes or cylinders. They did not have sensor posts, so they were left floating in the air.

The purpose of this project is to add meshes and textures to the WAM-V and its sensors for aesthetic improvement. This would not leave the sensors floating, but mount them on realistic sensor posts.

Description

To make this change, I needed to:

  1. insert the meshes and textures to the repository

  2. put them inside the visual tags of the sensors

  3. align their pose and orientation correctly

  4. add sensor posts with varying heights and angles

This last step required intelligent use of coordinate frames and transformations; it is set up such that users can set the desired pose of the sensors and the sensor post would match its height and swivel around to meet the sensor at the right location.

Result

The result of this project was a significant enhancement to the aesthetics and realism of the WAM-V sensors.

CPU Cases

Figure 20: CPU Cases on the WAM-V to match the real competition weight and look


CPU Cases Collision

Figure 21: Visualization of the collision boxes of the CPU Cases


Original Lidar

Figure 22: Old visual of the sensors, which were very plain


New Lidar old Camera

Figure 23: New lidar visual along with old cameras


Lidar and Camera on Posts

Figure 24: New lidar and camera visuals

Relevant Commits

7. VRX Recording and Playback

Software Used: roslaunch, Gazebo

Purpose

After running a competition using automated evaluation, teams could get their ROS/Gazebo log data, rostopic information (in a rosbag), and scoring results. However, it would be very valuable to be able to see what happened in the competition.

The purpose of this project is to allow recording and playback of a competition trial, which could be used to understand what happened in a competition.

Description

To add this feature, I used Gazebo's recording and playback API. I needed to tune the record period to ensure that the recording looked fluid and captured enough information.

Result

Users can now record the simulation by running

roslaunch vrx_gazebo sandisland.launch extra_gazebo_args:="-r --record_period 0.01 --record_path $HOME/.gazebo"

and then running

roslaunch vrx_gazebo playback.launch log_file:=$HOME/.gazebo paused:=true

Relevant Commits

Thank You to Open Robotics and Google Summer of Code

I want to say thank you to Open Robotics and Google Summer of Code for the hard work they have put in to make this happen. Writing open-source source software has been a valuable learning experience for me, and I am excited to keep contributing to the open-source community.

Thank you to Carlos, Ian, and Brian. Your mentorship and support have made this a very special experience for me.

About Tyler

My name is Tyler Lum and I am an Engineering Physics student at the University of British Columbia. I am passionate about robotics, AI, and software development, which is what motivated me to take part in the Google Summer of Code program. I spent the Summer of 2019 working with Open Robotics to work with intelligent robotics software engineers, contribute to one of the most popular open-source robotics simulators called Gazebo, and learn more about the open-source community. If you would like to connect or learn more about this project, you can contact me at tyler.lum@alumni.ubc.ca

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment