Skip to content

Instantly share code, notes, and snippets.

@consideRatio
Created March 13, 2023 17:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save consideRatio/071110916cb58220657398c61c14af7c to your computer and use it in GitHub Desktop.
Save consideRatio/071110916cb58220657398c61c14af7c to your computer and use it in GitHub Desktop.
Script in development used to calculate profile_list request for memory and cpu for `n2-highmem-[4|16|64]` nodes and `r5.[|4|16]xlarge` nodes
"""
This is a basic script that prints shares of CPU/Memory for machines of
different size. The script reads a list of nodes specifications, then prints
node share choices based on how much memory is required.
The mem_guarantee specification is calculated for each node by first subtracting
4 GB of memory for each node. This is to ensure that all nodes retain ~4 GB of
memory.
"""
import sys
from ruamel.yaml import YAML
# I declared three node types all having 1:16 ratio between CPU:GB_RAM, starting
# at 4, stopping at 64, excluding the often available 96 CPU machine.
#
# I opted for mem_choices based on the following ideas:
#
# - Choose multiples of 2.
#
# - Exclude options with less than 1 GB of memory.
#
# - Exclude options that could fit more than 128 users per node, as most k8s
# clusters and nodes only allows us to schedule a bit more than 100 users per
# node.
#
# EKS
eks_nodes = [
{
"cpu": 4,
"mem": 29.937,
"mem_display": 32,
"mem_choices": [1, 2, 4, 8, 16, 32],
},
{
"cpu": 16,
"mem": 121.513,
"mem_display": 128,
"mem_choices": [1, 2, 4, 8, 16, 32, 64, 128],
},
{
"cpu": 64,
"mem": 490.130,
"mem_display": 512,
"mem_choices": [4, 8, 16, 32, 64, 128, 256, 512],
}
]
# GKE
gke_nodes = [
{
"cpu": 4,
"mem": 27.738,
"mem_display": 32,
"mem_choices": [1, 2, 4, 8, 16, 32],
},
{
"cpu": 16,
"mem": 116.549,
"mem_display": 128,
"mem_choices": [1, 2, 4, 8, 16, 32, 64, 128],
},
{
"cpu": 64,
"mem": 486.949,
"mem_display": 512,
"mem_choices": [4, 8, 16, 32, 64, 128, 256, 512],
}
]
nodes = gke_nodes
for node in nodes:
print(f"Node type: {node['cpu']} CPU, {node['mem_display']} GB")
choices = {}
for mem_choice in node["mem_choices"]:
mem_minus_one_factor = (node["mem"] - 1) / node["mem"]
mem_gibi_label_factor = node["mem"] / node["mem_display"]
mem_guarantee = round(mem_minus_one_factor * mem_gibi_label_factor * mem_choice, 3)
cpu_guarantee = round(mem_choice / 8 * 0.1, 3)
choices[f"mem_{mem_choice}"] = {
"display_name": f"~{mem_choice} GB, ~{mem_choice / 8} CPU",
"kubespawner_override": {
"mem_guarantee": f"{mem_guarantee}G",
"cpu_guarantee": cpu_guarantee,
}
}
yaml = YAML()
yaml.indent(mapping=2, offset=2, sequence=4)
choices_string = yaml.dump(choices, sys.stdout)
print("")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment