gpolaert

# Input for FILE1
$InputFileName /<path_to_file1>
$InputFileTag <app_name_file1>
$InputFileStateFile <unique_file_id1>
$InputFileSeverity info
$InputRunFileMonitor

gpolaert

When you are trying to understand what's going on on your app, you need to be focused on your logs. But when you are dealing with database, or java app, many of ours are faced up to multiline errors, queries or events. You can loose a couple of hours just trying to rebuild each event.

Dealing with multiline can be relativly simple with a unique file, but it's more touchy when your stack aggregate all events to a unique place like a DB, a syslog server or in Elasticsearch.

The best solution is still modifying your formatter directly from your code and log all events in a single line or a single Json object.

gpolaert

nginx.common %{ipOrHost:clientip}[\s-]*\[%{date("dd/MMM/yyyy:HH:mm:ss Z"):timestamp}\] "(?:%{word:verb} %{notSpace:request}(?: HTTP/%{numberStr:httpversion})?|%{data:rawrequest})" %{integer:response} (?>%{integer:bytes}|-)
nginx.combined %{nginx.common} "%{regex("[^\\\"]*"):referrer:nullIf("-")}" "%{regex("[^\\\"]*"):agent:nullIf("-")}".*

gpolaert

How to forward Google App Engine to Logmatic.io

Here are my notes about how to forward Google Cloud Log to another provider, Logmatic.io.

Before start, make sure you have

• A google cloud account available
• A public url to manage the subcription with the Google pub/sub API (here:
• A Logmatic write API-Key

To go furher, here is the google documentation: Cloud Pub/Sub