Vanlightly/What a cost analysis might consist of

## What a cost analysis might consist of
A cost analysis can be quite a large undertaking but also extremely valuable. How far one takes it depends on the time available and the value of the analysis.

There are different levels of detail/accuracy (in order of cheapest and easiest, to most effort/hardest):
* Use back of the napkin math, based on prior experience, to cost the infrastructure needed.
* Use modeling (spreadsheets or simple programs that simulate the workloads and hardware), with some assumptions from prior experience, to cost the infrastructure needed.
* Use synthetic benchmarking to cost infrastructure needed. This requires deploying OSS components and running perf tests.

I typically go for the modeling approach if the workload is low variance in hardware demands and I have a lot of experience with the workload, else it's benchmarking. The benchmarking doesn't need to be super comprehensive, but enough to get some confidence in the sizing needed to handle some common workloads.

The way I would generally tackle this is in two parts:
1. A discussion of the general approaches, so that everyone understands the technical underpinnings of each approach. This will include both details on usage patterns (how users interact with the observability system) and technical details of how storage works, how compute works, typical workloads, typical challenges.
2. The cost analysis (perhaps a case study or two based on some common usage patterns). Ideally the analysis should be based on what the user wants to achieve. This may be tough if the two opposing approaches are different enough that the user can achieve different things. But we should start at what can be realistically done, then work backwards to cost that out. Explore dimensions such as number of tags, scale, latency requirements. Explore optimizations such as sampling (inc different sampling techniques), compression, etc. Different workloads such as adhoc vs dashboards (which could possibly be optimized).

Comparing an infrastructure cost analysis with vendor pricing it problematic as pricing strategy and infrastructure costs are different. Many start ups charge very little, making a loss as their pricing barely covers infrastructure. More mature vendors charge more, but these are sustainable businesses that make enough revenue to continually invest back into their platform and innovate over the long term. It might be interesting for example not know the infra costs of a case study using, for example, Honeycomb, but I'm not sure a vendor will want to disclose infra costs.
	A cost analysis can be quite a large undertaking but also extremely valuable. How far one takes it depends on the time available and the value of the analysis.

	There are different levels of detail/accuracy (in order of cheapest and easiest, to most effort/hardest):
	* Use back of the napkin math, based on prior experience, to cost the infrastructure needed.
	* Use modeling (spreadsheets or simple programs that simulate the workloads and hardware), with some assumptions from prior experience, to cost the infrastructure needed.
	* Use synthetic benchmarking to cost infrastructure needed. This requires deploying OSS components and running perf tests.

	I typically go for the modeling approach if the workload is low variance in hardware demands and I have a lot of experience with the workload, else it's benchmarking. The benchmarking doesn't need to be super comprehensive, but enough to get some confidence in the sizing needed to handle some common workloads.

	The way I would generally tackle this is in two parts:
	1. A discussion of the general approaches, so that everyone understands the technical underpinnings of each approach. This will include both details on usage patterns (how users interact with the observability system) and technical details of how storage works, how compute works, typical workloads, typical challenges.
	2. The cost analysis (perhaps a case study or two based on some common usage patterns). Ideally the analysis should be based on what the user wants to achieve. This may be tough if the two opposing approaches are different enough that the user can achieve different things. But we should start at what can be realistically done, then work backwards to cost that out. Explore dimensions such as number of tags, scale, latency requirements. Explore optimizations such as sampling (inc different sampling techniques), compression, etc. Different workloads such as adhoc vs dashboards (which could possibly be optimized).

	Comparing an infrastructure cost analysis with vendor pricing it problematic as pricing strategy and infrastructure costs are different. Many start ups charge very little, making a loss as their pricing barely covers infrastructure. More mature vendors charge more, but these are sustainable businesses that make enough revenue to continually invest back into their platform and innovate over the long term. It might be interesting for example not know the infra costs of a case study using, for example, Honeycomb, but I'm not sure a vendor will want to disclose infra costs.