Single Message Transform to add topic, partition, offset to sink message

curl -i -X PUT -H  "Content-Type:application/json" \
      http://localhost:8083/connectors/sink-elastic-orders-00/config \
      -d '{
          "connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector",
          "topics": "orders",
          "connection.url": "http://elasticsearch:9200",
          "type.name": "type.name=kafkaconnect",
          "key.ignore": "true",
          "schema.ignore": "false",
          "errors.tolerance":"all",
          "transforms": "addTS,InsertTopic,InsertOffset,InsertPartition",
          "transforms.addTS.type": "org.apache.kafka.connect.transforms.InsertField$Value",
          "transforms.addTS.timestamp.field": "op_ts",
          "transforms.InsertTopic.type":"org.apache.kafka.connect.transforms.InsertField$Value",
          "transforms.InsertTopic.topic.field":"source_topic",
          "transforms.InsertOffset.type":"org.apache.kafka.connect.transforms.InsertField$Value",
          "transforms.InsertOffset.offset.field":"source_offset",
          "transforms.InsertPartition.type":"org.apache.kafka.connect.transforms.InsertField$Value",
          "transforms.InsertPartition.partition.field":"source_partition"
      }'

Do we have a way to use message field value for ElasticSearch index? Means i push old historical data into Kafka then ElasticsearchSinkConnector should be able to push into right ES index "indexPrefix-{messageCreatedDate}". I am not that Timestamp transforms can be used in this case

is there a way to add the kafka record key to the value before sending it to a sink? This is not specific to elasticsearch but any sink. I can't seem to find such a transform, and it seems to me like it should be a solved problem 🤔

@cyberjar09 I found https://github.com/jzaralim/KeyToValueTransform and also discussion here: https://www.mail-archive.com/users@kafka.apache.org/msg37254.html. You can also use Kafka Streams, ksqlDB, etc.

hey thanks a lot for the guidance @rmoff 🎉
much appreciated! 😄