On decommissioning a node (https://www.cockroachlabs.com/docs/stable/remove-nodes.html) the instructions say to check the node before commissioning: "Open the Admin UI, click Metrics on the left, select the Replication dashboard, and hover over the Replicas per Store and Leaseholders per Store graphs" but don't say what to do with that information The example scenarios linked right before that say if there aren't enough nodes then decommissioning will hang, but don't say what metric/indicators let you know if there are enough nodes to decommission My own naive guess would be replicated ranges - the docs say by default you want 3 replications of a range, but the screenshots show replicated ranges at ~70. Does this mean there's 20+ times the required capacity in that example? As a separate question, what metrics should be used to identify if a cluster should be scaled out? Data usage near max capacity? Or will replicas drop towards some critical value?
kena @knz 20:29 these are all very good questions!
rendaw