Use fluentd to filter logs

filter logs.md

Use Catalog to Deploy Fluentd Server

Rancher sends the log to fluentd server, you can add filter, parser plugins configure to customize your log. Rancher support deploy fluentd server by using catalog Fluentd Aggregator

1. Browse to the project catalog that you want to deploy fluentd aggregator.

2. Click the Launch, select fluentd-aggregator, view details.

3. Configure deploy Namespace.

4. Configure deploy Service Type, the default is ClusterIP, you can use DNS or ClusterIP to access the service inside the cluster.

5. Configure output, select one or customize yours, check catalog description Output Plugins, make sure your output is supported. For more information, see Fluentd Output.

6. Configure filter, giving an example for using the filter to only include log from namespace kube-system. For more information, see Fluentd Filter.

   <filter cluster.**>
   @type grep
   <exclude>
      key $.kubernetes.namespace_name
      pattern ^test$
   </exclude>
   </filter>
   

7. Configure parser if need, for more information, see Fluentd Parser.

8. Configure formatter if need, for more information, see Fluentd Formatter.

9. After deployed the Fluentd Aggregator, go to Tools > Logging, configure Fluentd output log to your Fluentd Aggregator.

10. Complete the Fluentd Server Configuration form.

    1. From the Endpoint field, enter the cluster dns you deployed just now (i.e. fluentd-aggregator.fluentd-aggregator.svc.cluster.local:24224)

    2. Continue to configure other fields.

11. Click Test. Rancher sends a test log to Fluentd.

12. Click Save.

Result: Rancher is now configured to send logs to Fluentd Aggregator. Check you Fluentd Aggregator output events for your cluster and containers.