Skip to content

Instantly share code, notes, and snippets.

@duskmoon314

duskmoon314/note-on-add-rust-to-p4runtime.md

Created June 9, 2024 07:52
Show Gist options
  • Save duskmoon314/59ca06dd5cac6ed00a45a112d1ef5ba6 to your computer and use it in GitHub Desktop.
Save duskmoon314/59ca06dd5cac6ed00a45a112d1ef5ba6 to your computer and use it in GitHub Desktop.
Download ZIP
My note on how I add generated rust code for P4Runtime
Raw
note-on-add-rust-to-p4runtime.md

How I add generated rust code for P4Runtime

Campbell He (duskmoon) 2024-06

Background

Recently, I've been working on a research project that requires interaction with bmv2 and, eventually, Tofino switches. For this purpose, I chose P4Runtime as the control plane interface. P4Runtime currently supports three languages: C++, Go, and Python. Although these languages can be used to implement the control plane, I prefer Rust due to my personal preference and some negative experiences with the languages mentioned above. Therefore, I began exploring ways to generate Rust code for P4Runtime.

I might save time on my research project using Python or Go. But I'm not going to give up on Rust that easily. :P

First Glance at Rust with gRPC

When searching for "grpc" on crates.io, I found 165 results. Among them, tonic is the most popular and is maintained by hyperium, making it a reliable choice for a first try.

Tonic is a gRPC over HTTP/2 implementation that fits my requirements. It builds on prost and leverages tonic-build to generate Rust code from .proto files. There are two ways to use it:

  • Integrate tonic-build into the build script of the Rust project.
  • Use tonic-build to generate Rust code and include it in the project.

Since most usage examples follow the first method, I decided to try it and found a similar approach in another-s347/rusty-p4-proto. However, the generated code was not viewable as it was generated in a $OUT_DIR. I prefer the generated code in the project directory for easy viewing and modification. Therefore, I decided to use the second method, which led me to duskmoon314/p4runtime-rs.

I built a simple xtask binary to generate Rust code into the src directory in this archived attempt. The generated code is viewable, and utility functions can be added alongside the generated code. But then I asked myself, "Can I contribute the generated code back to the P4Runtime project?"

Second Try with protoc-gen-prost

To contribute the generated code back, I need to understand the architecture of the P4Runtime repo. There are many directories in the repo:

p4runtime
├── bazel/              # Bazel build files
├── CI/                 # CI scripts
├── codegen/            # scripts to generated code (Highly relevant)
├── CONTRIBUTING.md
├── docs/               # Documentation
├── go/                 # Generated Go code
├── go.mod
├── go.sum
├── LICENSE
├── proto/              # P4Runtime proto files (Highly relevant)
├── py/                 # Generated Python code
├── README.md
└── tools/              # Tools for P4Runtime documentation

After reading the README.md and scripts in the codegen directory, I found that I'm better off using protoc to generate code like other languages. So, I searched and found neoeinstein/protoc-gen-prost. This repo provides several protoc plugins for generating Rust code and supports tonic. I decided to give it a try.

How other languages are generated in P4Runtime

Before adding Rust support, we need to understand how other languages are generated. The entry point is codegen/update.sh. This script generates code in three steps:

  1. Prepare the docker image.
  2. Run codegen/compile_protos.sh in the container.
  3. Copy generated code to the corresponding directory: go/, py/, etc.

The compile_protos.sh script is the key. It uses protoc to generate code:

set -o xtrace
$PROTOC $PROTOS --cpp_out "$BUILD_DIR/cpp_out" $PROTOFLAGS
$PROTOC $PROTOS --grpc_out "$BUILD_DIR/grpc_out" --plugin=protoc-gen-grpc="$GRPC_CPP_PLUGIN" $PROTOFLAGS
$PROTOC $PROTOS --python_out "$BUILD_DIR/py_out" $PROTOFLAGS --grpc_out "$BUILD_DIR/py_out" --plugin=protoc-gen-grpc="$GRPC_PY_PLUGIN"
$PROTOC $PROTOS --go_out="$BUILD_DIR/go_out" --go-grpc_out="$BUILD_DIR/go_out" $PROTOFLAGS

Thus, I need to add a similar line for Rust.

Adding Rust support

After reading the docs of protoc-gen-prost and its sibling plugins, I came up with the following lines:

$PROTOC $PROTOS $PROTOFLAGS \
    --prost_out="$BUILD_DIR/rust_out/src" \
    --prost_opt=compile_well_known_types \
    --prost_opt=extern_path=.google.protobuf=::pbjson_types \
    --prost-serde_out="$BUILD_DIR/rust_out/src" \
    --tonic_out="$BUILD_DIR/rust_out/src" \
    --tonic_opt=compile_well_known_types \
    --tonic_opt=extern_path=.google.protobuf=::pbjson_types \
    --prost-crate_out="$BUILD_DIR/rust_out" \
    --prost-crate_opt="gen_crate=rust/Cargo.toml"

Here, I will try my best to explain what the above lines do:

  • --prost_out="$BUILD_DIR/rust_out/src": Use protoc-gen-prost to generate protobuf definitions based on prost crate and put them in the rust_out/src directory.
  • --prost_opt=compile_well_known_types: Compile well-known types: google.protobuf.Any
  • --prost_opt=extern_path=.google.protobuf=::pbjson_types: Use pbjson_types crate for well-known types.
    • The above two options use pbjson_types::Any instead of a generated google.protobuf.Any. This may be useful for compatibility with other crates.
  • --prost-serde_out="$BUILD_DIR/rust_out/src": Use protoc-gen-prost-serde to generate serde serialization/deserialization code.
    • This is useful for converting protobuf messages to/from JSON and other formats.
    • For example, in P4Runtime, we can convert P4Info from the p4c generated JSON to a Rust struct.
  • --tonic_out="$BUILD_DIR/rust_out/src": Use protoc-gen-tonic to generate gRPC code (client and server) based on tonic crate.
  • --tonic_opt=compile_well_known_types: Compile well-known types: google.protobuf.Any
  • --tonic_opt=extern_path=.google.protobuf=::pbjson_types: Use pbjson_types crate for well-known types.
    • As for prost, this is for compatibility with other crates.
  • --prost-crate_out="$BUILD_DIR/rust_out": Use protoc-gen-prost-crate to generate utility things for a crate.
  • --prost-crate_opt="gen_crate=rust/Cargo.toml": Use Cargo.toml in the rust directory to generate the crate.
    • The above two options use the preconfigured Cargo.toml to generate features and lib.rs for the crate.

After adding the above lines and other necessary changes, I generated Rust code like this:

p4runtime/rust
├── Cargo.lock
├── Cargo.toml
├── LICENSE -> ../LICENSE
├── README.md -> ../README.md
└── src
 ├── google.rpc.rs                # Generated by protoc-gen-prost, contains google.rpc.Status, google.rpc.Code, etc.
 ├── google.rpc.serde.rs          # Generated by protoc-gen-prost-serde
 ├── lib.rs                       # The entry point of the crate
 ├── p4.config.v1.rs              # Generated by protoc-gen-prost, contains p4.config.v1.P4Info, etc.
 ├── p4.config.v1.serde.rs        # Generated by protoc-gen-prost-serde
 ├── p4.v1.rs                     # Generated by protoc-gen-prost, contains p4.v1.WriteRequest, p4.v1.Update, etc.
 ├── p4.v1.serde.rs               # Generated by protoc-gen-prost-serde
 └── p4.v1.tonic.rs               # Generated by protoc-gen-tonic, contains P4RuntimeClient, etc.

I don't know if the above lines are the best way to generate Rust code, but they work and suit my needs. If you have any suggestions, please feel free to let me know. :)

How to use the generated code

The above-generated code is very simple and not very convenient to use. To make it more user-friendly, I followed antoninbas/p4runtime-go-client and built a wrapper crate: duskmoon314/p4runtime-client-rs.

In this crate, I re-export the generated code as p4runtime and provide a Client struct with useful methods. For Example:

//Example taken from https://github.com/duskmoon314/p4runtime-client-rs/blob/main/examples/basic/src/main.rs
use p4runtime_client::p4runtime::p4::v1 as p4_v1;
use p4runtime_client::p4runtime::p4::config::v1 as p4_config_v1;

// Create a client
let mut p4rt_client = p4_v1::p4_runtime_client::P4RuntimeClient::connect("http://127.0.0.1:9559").await?;
let mut client = p4runtime_client::client::Client::new(
    p4rt_client,                        // The generated tonic client
    0,                                  // The device id
    p4_v1::Uint128 { high: 0, low: 1 }, // The election id
    None,                               // The role
    p4runtime_client::client::ClientOptions {
        stream_channel_buffer_size: 1024,
        ..Default::default()
    }
);

// Start master arbitration
client.run().await?;

// Load P4Info
let p4bin = include_bytes!(...);
let p4info = include_bytes!(...);
let p4info = p4_config_v1::P4Info::decode(&p4info[..])?;
client.p4info_mut().load(p4info);

// Set forwarding pipeline config
client.set_forwarding_pipeline_config(p4bin.to_vec()).await?;

// Create and insert a table entry
let table_entry = client.table().new_entry(
    "ipv4_lpm",
    vec![(
        "hdr.ipv4.dst".to_string(),
        p4_v1::field_match::FieldMatchType::Lpm(p4_v1::field_match::Lpm {
            value: vec![10, 0, 1, 2],
            prefix_len: 32,
        }),
    )],
    Some(client.table().new_action("ipv4_forward", vec![vec![8, 0, 0, 0, 1, 2], vec![1]])),
    0, // The priority
);
client.table_mut().insert_entry(table_entry).await?;

Currently, only table, counter and digest are supported. I plan to add more features in the future.

Conclusion

In this note, I shared my experience adding Rust support for P4Runtime. I hope this note can help the WG members understand what is in the p4lang/p4runtime's PR#483. I may forget to include some details, so please ask me if you have any questions. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment