Troubleshooting/Debugging distributed systems is not easy. It can be really challenging if the systems are not thoroughly monitored. Since the begining of this microservice journey, I have been researching the best way we can build our monitoring infrastructure/instrument our code. This is the the solution I am proposing we adopt going forward.
###Prometheus
Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud. Prometheus works well for recording any purely numeric time series. It fits both machine-centric monitoring as well as monitoring of highly dynamic service-oriented architectures. In a world of microservices, its support for multi-dimensional data collection and querying is a particular strength.
We will be using prometheus for all our monitoring needs except in cases where prometheus cannot handle at the moment(eg hystrix metrics). The major reason for choosing prometheus over other monitoring solutions