Datadog is very nice. Here's something I wrote when asked what value we were getting from it.
I would break it down into four pieces. Datadog is
-
providing functionality
-
we need
-
in an easy-to-use manner
-
that would be difficult to build and maintain ourselves
It gathers system metrics, integrates with key software we use, and provides a standard interface to which our applications can send custom metrics.
Datadog has prebuilt integrations to pull data from almost every important service we use.
Through the integrations datadog generates a consolidated event stream that we can filter and earch as needed.
Datadog lets us build dashboards that combine metrics from many different sources. We can combine and transform metrics to make them more useful. It also provides an powerful interface for interactive exploration of metrics.
Datadog has nice stream processing capabilities for generating alerts, and it can surface them in services we use like pagerduty and slack.
We don't get nearly enough insight from cloudwatch alone, we need an on-instance tool to gather system and app metrics.
There are lots of services with operational signficiance, but many of them don't provide a good way to access their data.
We would spend dramatically longer investigating problems if we had to look at eash source of events in isolation. Many of our event sources don't even provide a way for us to view past events or to query them.
Per-service and per-instance dashboards are important for investigating problems quickly. The consolidation of data from multiple sources is again a key feature.
We need to do anaylze trends in our metrics and alert on them.
The agent is deployable via a chef cookbook datadog wrote for us. It requires minimal configuration. It knows which system and application metrics are worth gathering.
Integrating with all the data sources is literally a few clicks.
The interface makes searching and filtering events straightforward.
There are prebuilt dashbaords for lots of things we care about. Snazzy features like autocomplete and templating make building our own dashboards easy.
The guided steps and previewed outputs make creating alerts simple.
Here I described a system of collectd, custom code to pull metrics from cloudwatch, custom code to pull or receive events from various sources (airbrake, cloudtrail, chef, pagerduty, jenkins, etc) influxdb, and grafana.