Skip to content

Instantly share code, notes, and snippets.

@MathiasKoch
Last active June 20, 2023 07:19
Show Gist options
  • Save MathiasKoch/292804fe7291a25161ddd78112289e31 to your computer and use it in GitHub Desktop.
Save MathiasKoch/292804fe7291a25161ddd78112289e31 to your computer and use it in GitHub Desktop.
AWS IoT Protobuf decoding

Creating descriptor file

protoc --descriptor_set_out=issue.desc --proto_path=src/sensor --include_imports issue.proto

Running the decode locally using protoc:

echo 0a0f0a0470315f31100018a0c1d7c5fb300a0f0a0470325f3110001888c9d7c5fb300a0f0a0470315f31100018a8e8d7c5fb300a0f0a0470325f3110001890f0d7c5fb300a0f0a0470315f31100018b08fd8c5fb300a0f0a0470325f311000189897d8c5fb300a0f0a0470315f31100018b8b6d8c5fb300a0f0a0470325f31100018a0bed8c5fb3010c0ddd8c5fb30 | xxd -r -p | protoc --decode protos.input.Packet issue.proto

Expected output:

values {
  id: "p1_1"
  v: 0
  t: 1682431140000
}
values {
  id: "p2_1"
  v: 0
  t: 1682431141000
}
values {
  id: "p1_1"
  v: 0
  t: 1682431145000
}
values {
  id: "p2_1"
  v: 0
  t: 1682431146000
}
values {
  id: "p1_1"
  v: 0
  t: 1682431150000
}
values {
  id: "p2_1"
  v: 0
  t: 1682431151000
}
values {
  id: "p1_1"
  v: 0
  t: 1682431155000
}
values {
  id: "p2_1"
  v: 0
  t: 1682431156000
}
timestamp: 1682431160000

Output using SELECT VALUE decode(*, 'proto', 'MyS3Bucket', 'issue.desc', 'issue', 'Packet') FROM '+/input/+/proto'

values {
  id: "p1_1"
  v: 0
  t: "1682431140000"
}
values {
  id: "p2_1"
  v: 0
  t: "1682431141000"
}
values {
  id: "p1_1"
  v: 0
  t: "1682431145000"
}
values {
  id: "p2_1"
  v: 0
  t: "1682431146000"
}
values {
  id: "p1_1"
  v: 0
  t: "1682431150000"
}
values {
  id: "p2_1"
  v: 0
  t: "1682431151000"
}
values {
  id: "p1_1"
  v: 0
  t: "1682431155000"
}
values {
  id: "p2_1"
  v: 0
  t: "1682431156000"
}
timestamp: "1682431160000"

Same result happens when using every type of 64 bit values (uint64, int64, sint64, fixed64, sfixed64). Works as expected with 32 bit sizes, except for the obvious limitation on number sizes.

syntax = "proto3";
package protos.input;
message Sample {
string id = 1;
optional uint32 v = 2;
uint64 t = 3;
optional bool q = 4;
}
message Packet {
repeated Sample values = 1;
uint64 timestamp = 2;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment