Autometrics Implementors Guide

README.md

This is a very rough outline of the things that need to be done to implement autometrics in another language:

1. Research auto-instrumentation libraries for the language

   1. How do they work? This is mostly to serve as a potential source of inspiration (or maybe we can just use one of those libraries with a little configuration?)

2. How does code generation work in the language? Are there macros, decorators, or some other method for generating code?

   1. Autometrics needs to be able to insert metrics-collection code at the beginning and end of any instrumented function
      1. Rust does this with an attribute macro
      2. TypeScript does this at runtime with a wrapper function
   2. Can this also modify the doc comments of the given function?
      1. In Rust, macros can add to doc comments
      2. In TypeScript, we needed a separate Language Service Plugin to do this
   3. Before worrying about actually doing anything related to metrics, just try to insert some code and comments to see what’s possible

3. Pick a metrics collection library to use

   1. Ideally it would support any of the popular libraries in that language, but just pick one to start
   2. Look for prometheus client libraries or OpenTelemetry clients

4. Collect the metrics

   1. Create a counter called function.calls.count and a histogram called function.calls.duration (some Prometheus clients may want you to use underscores instead of dots as separators)
   2. Grab the function name and use it as a label with the key function
   3. If you can easily get the name of the module the function is in, attach that as a label with the key module (this is mostly to handle the case where a code base has multiple functions with the same names in different files)
   4. At the beginning of the function, set some variable to the current time
   5. At the end of the function, increment the call counter and record the function duration
   6. Export the metrics using the exporter of the library you’ve chosen
   7. Check if the metrics are being registered correctly:

      • Your metrics output should something like this:

        # HELP target_info Target metadata
        # TYPE target_info gauge
        target_info{service_name="unknown_service:node",telemetry_sdk_language="nodejs",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.9.1"} 1
        # HELP function_calls_count_total description missing
        # TYPE function_calls_count_total counter
        function_calls_count{module="/build/index.js",function="rootHandler",result="ok"} 2 1676828080964
        # HELP function_calls_duration description missing
        # TYPE function_calls_duration histogram
        function_calls_duration_count{function="rootHandler"} 2 1676828080964
        function_calls_duration_sum{function="rootHandler"} 1 1676828080964
        function_calls_duration_bucket{function="rootHandler",le="0"} 0 1676828080964
        function_calls_duration_bucket{function="rootHandler",le="5"} 2 1676828080964
        function_calls_duration_bucket{function="rootHandler",le="10"} 2 1676828080964
        function_calls_duration_bucket{function="rootHandler",le="25"} 2 1676828080964
        function_calls_duration_bucket{function="rootHandler",le="50"} 2 1676828080964
        function_calls_duration_bucket{function="rootHandler",le="75"} 2 1676828080964
        function_calls_duration_bucket{function="rootHandler",le="100"} 2 1676828080964
        function_calls_duration_bucket{function="rootHandler",le="250"} 2 1676828080964
        function_calls_duration_bucket{function="rootHandler",le="500"} 2 1676828080964
        function_calls_duration_bucket{function="rootHandler",le="1000"} 2 1676828080964
        function_calls_duration_bucket{function="rootHandler",le="+Inf"} 2 1676828080964

5. Generate query links

   1. Generate Prometheus queries and links and insert them into the function’s doc comments
      1. This is the relevant code in Rust: https://github.com/fiberplane/autometrics-rs/blob/main/autometrics-macros/src/lib.rs#L314-L408
      2. This is the code in TypeScript: https://github.com/fiberplane/autometrics-ts/blob/db1eb17c0cbf71a138f2706a732bdb729c65a908/packages/autometrics-docs/src/index.ts#L87
   2. (if needed) Investigate a Language Service Plugin
      1. if the language doesn’t support adding doc comments from the wrapper/macro/decorator directly, another possible venue would be “jacking in” to the LSP itself – check if the LSP has the API for injecting custom markdown text (most LSP servers support rendering markdown in doc comments)

6. Examples and documentation

   1. Write a couple of examples that show how to use the library with popular API frameworks

7. Alerts & Grafana dashboards

   1. This is a stretch goal

8. Profit