Skip to content

Instantly share code, notes, and snippets.

@mmedvede
Last active July 19, 2019 20:21
Show Gist options
  • Save mmedvede/7826cea9da9737311b3a05b53d6b447f to your computer and use it in GitHub Desktop.
Save mmedvede/7826cea9da9737311b3a05b53d6b447f to your computer and use it in GitHub Desktop.
# Notes on setting up parallel metric writes from m3aggregator
# Notes on setting up parallel metric writes from m3aggregator
# Setup aggregation topics
curl -vvvsSf -X POST -H 'topic-name: aggregated_metrics' localhost:7201/api/v1/topic/init -d '{ "numberOfShards": 64 }'
curl -vvvsSf -X POST -H 'topic-name: aggregated_metrics_2' localhost:7201/api/v1/topic/init -d '{ "numberOfShards": 64 }'
# coordinator that would be on aggregated_metrics topic
curl -vvvsSf -X POST localhost:7201/api/v1/services/m3coordinator/placement/init -d '{
"instances": [
{
"id": "m3coordinator01",
"zone": "embedded",
"endpoint": "m3coordinator01:7507",
"hostname": "m3coordinator01",
"port": 7507
}
]
}'
# coordinator that would be on aggregated_metrics_2 topic
curl -vvvsSf -X POST -H 'Cluster-Environment-Name: env2' localhost:7201/api/v1/services/m3coordinator/placement/init -d '{
"instances": [
{
"id": "coordinator_env2",
"zone": "embedded",
"endpoint": "m3coordinator02:7507",
"hostname": "m3coordinator02",
"port": 7507
}
]
}'
# Add coordinators in default_env to the aggregator topic
curl -vvvsSf -X POST localhost:7201/api/v1/topic -d '{
"consumerService": {
"serviceId": {
"name": "m3coordinator",
"environment": "default_env",
"zone": "embedded"
},
"consumptionType": "SHARED",
"messageTtlNanos": "600000000000"
}
}' # msgs will be discarded after 600000000000ns = 10mins
# "consumptionType": "REPLICATED",
# Add coordinators in env2 to the aggregator topic
curl -vvvsSf -X POST -H 'topic-name: aggregated_metrics_2' localhost:7201/api/v1/topic -d '{
"consumerService": {
"serviceId": {
"name": "m3coordinator",
"environment": "env2",
"zone": "embedded"
},
"consumptionType": "SHARED",
"messageTtlNanos": "600000000000"
}
}'
# And then add second dynamicBackend to m3aggregator config, changing topic:
writer:
topicName: aggregated_metrics_2
topicServiceOverride:
zone: embedded
environment: default_env
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment