With Windows Containers, as with any application, you will need to consider your logging strategy. In the containers world it is generally accepted to log to STDOUT/STDERR (standard out and standard error). As you scale up your services you will need to aggregate and store these logs for analysis and debugging later on. There exist many back end systems to aggregate logs and help with analysis, in Azure you have Log Analytics and Application Insights.
There are two general strategies for configuring your containers to send logs to a backend logging system containers:
- In container logging - Your application is responsible for sending the logs directly to the backend system. This can also be [configured via sidecar].
- Host level logging - You application is configured for sending logs to a log collection agent on the system. The logging agent will then be responsible for aggregating logs and sending to backend logging system.
[image of logging strategies]
Using the in container logging can be effective for smaller systems and teams. It requires less moving parts and can be quickly configured through many logging libraries using the logging libraries syncs. As the number of services increases, as is the case in many container orchestration systems, having each service send their own logs creates a lot of overhead particularly in terms of network load. By having an agent on each host that aggregates and collects logs you can reduce the number of connections to the logging backend as well as preform advanced batching, filtering and transformations before sending the logs.
If choosing the logging agent strategy then you can use the generally excepted method of logging to STDOUT/STDERR. Most monitoring agents, such as [Log analytics monitoring agent], are configured to read container STDOUT/STDERR. Refer to your monitoring agent's documentation for correct configuration in a container environment. Each orchestrator (see Service Fabric or [AKS]https://docs.microsoft.com/en-us/azure/monitoring/monitoring-container-overview)) has documentation for best practices inside there platforms. It is also possible to use other techniques to get the logs to agent via UDP or ETW if you have requirement that is not met by using stand out and stand error.
With Windows containers there are a few additional considerations in the case where you are running Windows Services inside the container and using the Host level logging strategy. If you are running an executable as the entry point to the application then configuring your application to log to STDOUT/STDERR will be the same as with any container.
If you have an executable that is the entry point to your application then you can configure you application to to write to STDOUT. Refer to you programming language configuration to learn how to do this.
In .NET 4.x applications this can be accomplished through most logging libraries (log4net, serilog, etc.) or you can look into using ConsoleTraceListener. See more information's on [logging with .NET].
You can confirm this is working by running docker logs <container-id> on your local machine during development.
When containerizing a window service, such as IIS, the task of logging to STDOUT is requires additional configuration. As the service is a running in the background (and is not the docker [monitored application](need doc and link on how to configure service monitoring) entrypoint for the container) anything written to the console is not going to show up up in STDOUT. In addition, many older applications have other logging strategies, such as logging to a rotating log file which may not be easily changed to STDOUT do to using a custom logging component.
There are a few strategies that are available to help address logging with Window Services:
- Convert the Windows Service to an executable and make it the entry point to your container [comment]: <not sure if we should point at external tool? would be nice if we supplied a monitoring agent that was capable of this> - Use a Service Monitoring tool such as Spinner to tail a file to console out
- Configure a side car with to collect logs from the file system in the container and send to console out (or directly to you logging backend).
- Configure a volume to the host and use Monitoring Agent on Host to read files directly
Each strategy has it's pro's and con's. Some of the strategies may not be available to you such as converting you application to a executable if you are using IIS.
If it is possible to convert the Windows Service to an executable this is a simple way to accomplish the task. It allows you to configure your windows container with the executable as the entry point and is the standard way of running applications in containers.
The approach is dependant on your application and may not be option for you scenario. For example it is not possible to move your ASP.NET Application hosted in IIS to a executable.
In platforms that enable you to configure sidecars, it is possible to configure a agent (or application) to listen to your application logs and send them to STDOUT or to your logging back end directly.
When configuring the side car, consider if you can change your logging direction inside your service to log to UDP instead of a file. Many popular logging libraries can be configured to send logs to a UDP port. If this is an option then sending the logs directly to UDP instead of a file first will minimize resources on disk. If this is not possible then logging to file and configuring the side car to read the log file is an alternative.
If you configure the logging agent to send to STDOUT you can confirm this is working by running docker logs <container-id> on your local machine during development.
Note do we want to include an example? FluentD is the simpliest though there is no official windows container for fluentd. OMS agent won't work in the soultion though many other agents like datadog would.] As an example, we can use Fluentd as a side car in Kuberentes and send the logs to console out: See example at https://github.com/jsturtevant/windows-k8-playground/tree/master/logging and https://gist.github.com/jsturtevant/26b3f9b9fc87cf517d37c49759295eb6 and fluentd PR at fluent/fluentd-docker-image#130 also have examples: https://github.com/paulbouwer/windows-containers-workshop/tree/master/labs
If add a sidecar is not available for your platform you can configure the container to log directly to the host. On the host you can then configure your Monitoring Agent to watch those files directly. An example would be to run your container with
docker run -v c:/logs/app_data:c:/inetpub/wwwroot/app_data -d -p 8080:80 iislogging-example
Then you can refer to your Monitoring agent on how to configure it to watch a given directory. See using custom log sources if you are using Log Analytics.
You can configure IIS logs to send to a file by using the following command in your Dockerfile:
Set-WebConfigurationProperty -pspath 'MACHINE/WEBROOT/APPHOST' -filter "system.applicationHost/sites/sitedefaults/logFile" -name "logTargetW3C" -value "File"
You can set permissions for your application to log to a folder using the following command in your Dockerfile:
RUN icacls C:/inetpub/wwwroot/App_Data /grant IIS_IUSRS:f /T
If you are logging directly to the host you will need to consider log rotation and log clean up. Docker supports log rotation through docker deamon if you are logging to STDOUT, otherwise you are responsible for log rotation and clean up on the host for volumes mounted directly to the host.