Skip to content

Instantly share code, notes, and snippets.

@emxsys

emxsys/northern-horizon-mapserver-01.md

Last active August 22, 2023 10:59
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save emxsys/d692a2a11fa4620277d6cc4007d1e0c2 to your computer and use it in GitHub Desktop.
Save emxsys/d692a2a11fa4620277d6cc4007d1e0c2 to your computer and use it in GitHub Desktop.
Download ZIP
Northern Horizon MapServer Configuration
Raw
northern-horizon-mapserver-01.md

Northern Horizon Map Server

  • Host: DigitalOcean
  • Team: Emxsys
  • Project: Northern Horizon Servers
  • Server: mapserver.northernhorizon.org
  • IP Address: 157.245.134.29

The Northern Horizon map server is virtual machine hosted on DigitalOcean that provides web map services (WMS) and web map tile services (WMTS) to Northern Academy projects and the WorldWind Explorer. The map server is configured to provide web map clients with timely and usable responses to WMS/WMTS requests through the use of optimized source data, properly formatted response headers, and client and server-side caching techiques. The server has been secured with a firewall, log analysis tools, and a utilitly (fail2ban) to prevent brute force attacks. Finally, a SSL certificate has been assigned via the Let's Encrypt service.

Imagery and elevation source data are located in AWS compatible storage on the DigitalOcean host. This data has been optimized to reduce rendering times, bandwidth and storage costs.

Response headers to WMS requests are configured to support client-side caching and CORS. The map server also provides proxies to several external servers that do not natively provide proper headers (e.g., USGS) and injects the missing headers in the response.

The map server leverages server side caching to provide fast responses to WMS requests for common imagery and elevation data. The WMTS endpoint serves clients with rapid responses from pre-built caches of Blue Marble and Landsat map tiles.

Map Services

wms

This primary endpoint provides access to the server's imagery layers, including worldwide Blue Marble and Landset imagery.

elev

This primary endpoint provides access to the server's elevation layers, including GEBCO, SRTM, ASTER, and NED data sets.

worldwind25/wms

This endpoint provides map layers compatible with worldwind25.arc.nasa.gov layers.

worldwind26/elev

This endpoint provides elevation layers compatible with worldwind26.arc.nasa.gov elevation layers.

worldwind27/wms/virtualearth

This endpoint provides MicroSoft VirtualEarth layers compatible with worldwind27.arc.nasa.gov.

mapcache/wmts

This endpoint provides access to pre-built map tiles for Blue Marble and Landsat imagery.

opentopomap

This endpoint is a proxy to worldwide Topographic maps from OpenStreetMap. This proxy provides Cache-Control headers to support client-side caching of map tiles. This proxy is defined in /etc/apache2/apache.conf.

OpenTopoMap is a free, topographic map generated from OpenStreetMap data and SRTM elevation data. The card style is based on the official cards and relies on good legibility through high contrast and balanced signatures.

Copyright: © OpenStreetMap-Mitwirkende, SRTM | Kartendarstellung: © OpenTopoMap (CC-BY-SA)

The proxied tiles are retrieved with the following path: https://mapserver.northernhorizon.org/opentopomap/{z}/{x}/{y}.png

USGSTopo

This endpoint is a proxy to USGS Topo Base Map from the National Map. This proxy provides Cache-Control headers to support client-side caching of map tiles. This proxy is defined in /etc/apache2/apache.conf.

USGSImageryTopo

This endpoint is a proxy to USGS Imagery Topo Base Map from the National Map. This proxy provides Cache-Control headers to support client-side caching of map tiles. This proxy is defined in /etc/apache2/apache.conf.

Droplet Specifications: mapserver-01

The mapserver-01 droplet is a scalable Ubuntu 20.04 virtual machine running Apache, MapServer and MapCache.

  • IP Address: 157.245.134.29

Size

The droplet can be resized to any one of these three configurations. Prices per hour vary per size.

  • 4 GB Memory / 2 AMD vCPUs / 80 GB Disk / NYC1 - Ubuntu 20.04 (LTS) x64
  • 8 GB Memory / 4 AMD vCPUs / 80 GB Disk / NYC1 - Ubuntu 20.04 (LTS) x64
  • 16 GB Memory / 8 AMD vCPUs / 80 GB Disk / NYC1 - Ubuntu 20.04 (LTS) x64

Spaces: emxsys

The emxsys space provides AWS compatible storage for the optimized imagery and elevation data used by the MapServer to generate map tiles for WMS clients.

Endpoints

These endpoints are referenced by the Map Server to retrieve the raw imagery and elevations needed to satisfy WMS client requests.

Folders

Listed below are the folders within the emxsys space containing data used by the MapServer:

  • elevations
    • asterv2
    • gebco
    • ned10m
    • strm-cgiar
  • imagery
    • bmng-topo-bathy (Blue Marble Next Generation 2004 imagery)
    • i3 (Landsat imagery)

Access

The data are accessed by MapServer via the GDAL vsicurl virtual file system handler. vsicurl is a file system handler that allows on-the-fly random reading of files available through HTTP/FTP web protocols, without prior download of the entire file. The following ogrinfo output for a BlueMarble layer shows the location path used to access the data:

ogrinfo -al /opt/mapserver/data/bluemarble/bmng-topo-bathy-200402.shp
Output:
INFO: Open of `bmng-topo-bathy-200402.shp'
      using driver `ESRI Shapefile' successful.

Layer name: bmng-topo-bathy-200402
Metadata:
  DBF_DATE_LAST_UPDATE=2019-03-19
Geometry: Polygon
Feature Count: 8
Extent: (-180.000000, -90.000000) - (180.000000, 90.000000)
Layer SRS WKT:
(unknown)
location: String (254.0)
OGRFeature(bmng-topo-bathy-200402):0
  location (String) = /vsicurl/https://emxsys.sfo2.digitaloceanspaces.com/imagery/bmng-topo-bathy/optimized/bm200402/world.topo.bathy.200402.3x21600x21600.A1.tif
  POLYGON ((-180 90,-90 90,-90 0,-180 0,-180 90))
.
.
.

Configuration: mapserver-01

This section documents the setup of the mapserver-01 virtual machine, specifically Apache, MapServer and MapCache.

For reference and additional information see:

Initial Setup

For reference, see: Initial Server Setup with Ubuntu 20.04 | DigitalOcean

Add User(s)

Login as root via DigitalOcean Droplet console and then perform the following steps as the root user.

Add new user(s) and set password

adduser xxx

Add administrative user(s) to sudoers group

usermod -aG sudo xxx

Enable Firewall

Get the firewall's capabilities

ufw app list

Output:

 Available applications:
   OpenSSH

Turn on OpenSSH

ufw allow OpenSSH

Output:

 Rules updated
 Rules updated (v6)

Enable the firewall

ufw enable

Output:

Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup

Verify the firewall status is active

ufw status

Output:

Status: active
 
 To                         Action      From
 --                         ------      ----
 OpenSSH                    ALLOW       Anywhere                  
 OpenSSH (v6)               ALLOW       Anywhere (v6)

logout or exit

exit

Login to the server via SSH

Open up a terminal in your local Linux enviorment and start an SSH session to verify access to the server.

Login via SSH

ssh username@157.245.134.29

-or-

ssh username@mapserver.northernhorizon.org

Output:

username@mapserver-01:~$

Install Postfix mail transfer agent

Postfix is used to email log files to adminstrator(s).

sudo apt-get update
sudo apt install mailutils

sudo ufw app list
sudo ufw app info 'Postfix'
sudo ufw app info 'Postfix SMTPS'
sudo ufw app info 'Postfix Submission'
sudo ufw allow 'Postfix Submission'
sudo ufw status

To (re)configure

Configure for Internet site and set origin to mapserver.northernhorizon.org:

sudo dpkg-reconfigure postfix

Output:

        setting synchronous mail queue updates: false
	setting myorigin
	setting destinations: mapserver-01, mapserver.northernhorizon.org, localhost.localdomain, localhost
	setting relayhost: 
	setting mynetworks: 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
	setting mailbox_size_limit: 0
	setting recipient_delimiter: +
	setting inet_interfaces: all
	setting inet_protocols: ipv4
	WARNING: /etc/aliases exists, but does not have a root alias.
	
	Postfix (main.cf) is now set up with the changes above.  If you need to make 
	changes, edit /etc/postfix/main.cf (and others) as needed.  To view Postfix 
	configuration values, see postconf(1).
	
	After modifying main.cf, be sure to run 'systemctl reload postfix'.
	
	Running newaliases
	Processing triggers for libc-bin (2.31-0ubuntu9.2) ...

After configuration, setup for outgoing mail only

To setup for outgoing only, edit main.cf and change inet_interfaces from all to localhost:

sudo nano /etc/postfix/main.cf
sudo systemctl restart postfix

Install Logwatch utiltiy

Logwatch is a powerful log parser and analyzer used for security monitoring and emailing alerts.

sudo apt-get update
sudo apt-get install logwatch
sudo nano /usr/share/logwatch/default.conf/logwatch.conf

Review/edit daily report job

cat /etc/cron.daily/00logwatch

Example: Apache (http) service for last 24 hours

sudo logwatch --detail med --service http --range "since 24 hours ago for those hours" | less

Example: Mail (Postfix) service for last 24 hours

sudo logwatch --detail med --service Postfix --range "since 24 hours ago for those hours" | less

Example: All services for last hour

sudo logwatch --detail med --service all --range "since 1 hours ago for those hours" | less

Test email report

sudo logwatch --output mail

Install Fail2Ban utility

Fail2Ban is a utility designed to thwart brute force attacks on the server. The attacker's IP addresses are banned after n failed attempts to access the server.

sudo apt-get update
sudo apt-get install fail2ban
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo nano /etc/fail2ban/jail.local 
sudo systemctl restart fail2ban

Potentially useful command you can run from the DigitalOcean web console to unban your IP address:

sudo fail2ban-client set sshd unbanip xxx.xxx.xxx.xxx

Install Log File Navigator

sudo apt-get update
sudo apt-get install lnav

Example: System log

sudo lnav /var/log/syslog

Example: Apache access log

sudo lnav /var/log/apache/access.log

Example: Apache error log

sudo lnav /var/log/apache/error.log

Example: Map Server error log

sudo lnav /opt/mapserver/tmp/ms-error.log

Apache Install

Apache (apache2) is the web server from which the MapServer and MapCache services run.

Install Apache

Run the following commands to install the Apache2 web server:

sudo apt-get update
sudo apt-get install apache2
sudo apt-get install apache2-utils

Set the ServerName

Add the ServerName mapserver-01 line to the default Apache configuration file via the nano editor:

sudo nano /etc/apache2/apache2.conf

Validate and restart:

sudo apache2ctl configtest
sudo systemctl restart apache2

Adjust the firewall to allow web traffic

sudo ufw app list
sudo ufw app info "Apache Full"
sudo ufw allow in "Apache Full"

Enable Apache modules

Enable the following modules in the Apache web server:

sudo a2enmod cgid
sudo a2enmod proxy
sudo a2enmod proxy_connect
sudo a2enmod proxy_http
sudo a2enmod proxy_html
sudo a2enmod rewrite
sudo a2enmod deflate
sudo a2enmod xml2enc

sudo a2enmod cache
sudo a2enmod cache_disk
sudo a2enmod expires
sudo a2enmod headers

Use defaults for cache-disk.conf. Review configuration:

sudo nano /etc/apache2/mods-enabled/cache_disk.conf

sudo apachectl configtest
sudo systemctl restart apache2
sudo systemctl status apache2

Let's Encrypt SSL Cert

See: How To Secure Apache with Let's Encrypt on Ubuntu 20.04 | DigitalOcean

sudo apt install certbot python3-certbot-apache
sudo certbot --apache
Output
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Enter email address (used for urgent renewal and security notices) (Enter 'c' to
cancel): azhao@northernacademy.org

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
agree in order to register with the ACME server at
https://acme-v02.api.letsencrypt.org/directory
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(A)gree/(C)ancel: a

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing to share your email address with the Electronic Frontier
Foundation, a founding partner of the Let's Encrypt project and the non-profit
organization that develops Certbot? We'd like to send you email about our work
encrypting the web, EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: n

Which names would you like to activate HTTPS for?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: mapserver.northernhorizon.org
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel):
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for mapserver.northernhorizon.org
Waiting for verification...
Cleaning up challenges
Enabled Apache socache_shmcb module
Enabled Apache ssl module
Deploying Certificate to VirtualHost /etc/apache2/sites-enabled/default-ssl.conf

Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Redirecting vhost in /etc/apache2/sites-enabled/000-default.conf to ssl vhost in /etc/apache2/sites-enabled/default-ssl.conf

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations! You have successfully enabled
https://mapserver.northernhorizon.org

You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=mapserver.northernhorizon.org
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/mapserver.northernhorizon.org/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/mapserver.northernhorizon.org/privkey.pem
   Your cert will expire on 2021-12-19. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot again
   with the "certonly" option. To non-interactively renew *all* of
   your certificates, run "certbot renew"
 - Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

Apache Cache

The Apache mod_cache and mod_cache_disk (installed) are essential to the performance of the Apache/MapServer instance. The configuration and maintenence of the cache are documented here. It is important to know that changes to your MapServer may require you to clear data from the cache so that your updated data is returned instead of the now-stale cached data. You can use the htcacheclean and cleancache.sh utilities (documented below) to purge the cache of stale data.

Cache Configuration

Open your virtual host file(s) for the imagery server. For example:

sudo nano /etc/apache2/sites-enabled/000-default.conf

Add the Apache caching configuration, as follows:

To start leave the CacheQuickHandler off for complete processing of caching rules:

        CacheQuickHandler off

Setup a locking mechanism based on Apache docs:

        CacheLock on
        CacheLockPath /tmp/mod_cache-lock
        CacheLockMaxAge 5

Don't store cookies in the cache to prevent leaking of user-specific cookies

        CacheIgnoreHeaders Set-Cookie

Web WorldWind requests require CacheIgnoreCacheControl to be enabled to obtain cache hits. This tells the server to attempt to serve the resource from the cache even if the request contains no-cache header values.

        CacheIgnoreCacheControl On

Now we'll enable caching for the map services endpoints with a number of directives. CacheEnable disk defines the caching implemenation. CacheHeader on enables a reponse header that will indicate whether there was a cache hit or miss. Another directive we'll set is CacheDefaultExpire so that we can set an expiration (in seconds) if neither the Expires nor the Last-Modified headers are set on the content. Similarly, we'll set CacheMaxExpire to cap the amount of time items will be saved. We'll set the CacheLastModifiedFactor so that Apache can create an expiration date if it has a Last-Modified date, but no expiration. The factor is multiplied by the time since modification to set a reasonable expiration.

The ExpiresActive on enables expiration processing. The ExpiresDefault directive sets the default expiration time. These will set the Expires and the Cache-Control "max-age" to the correct values. When you are certain the caching is working as desired, you can extend the expiration time.

Within the <Location /wms> block, add the following cache directives:

	CacheEnable disk 
	CacheHeader on

	CacheDefaultExpire 600
	CacheMaxExpire 86400
	CacheLastModifiedFactor 0.5

	ExpiresActive on
	ExpiresDefault "access plus 1 week"

	Header merge Cache-Control public

Your edited virtual host .conf file should something look like this:

<VirtualHost *:80>
    	ServerAdmin webmaster@localhost
	.
	.
	.
	# Apache caching configuration
        CacheQuickHandler off

        CacheLock on
        CacheLockPath /tmp/mod_cache-lock
        CacheLockMaxAge 5

        CacheIgnoreHeaders Set-Cookie
        CacheIgnoreCacheControl On
	
        # MapServer /wms endpoint
        Alias /wms /usr/lib/cgi-bin/mapserv
        <Location /wms>
                CacheEnable disk /wms
                CacheHeader on

                CacheDefaultExpire 600
                CacheMaxExpire 86400
                CacheLastModifiedFactor 0.5

                ExpiresActive on
                ExpiresDefault "access plus 1 week"

                Header merge Cache-Control public

                SetHandler cgi-script
                Options ExecCGI
                SetEnv MS_MAPFILE /opt/mapserver/map/imagery.map
        </Location>	
	.
	.
	.
</VirtualHost>

Cache Maintenance

htcacheclean

htcacheclean (installed by apache2-utils) is used to manage the cache. If htcacheclean is passed one or more URLs, each URL will be deleted from the cache. If multiple variants of an URL exists, all variants would be deleted. Following are a few examples of its use.

The following command displays the contents of the cache. The -p switch specifies the cache location; the -a (or -A) dumps the contents.

sudo htcacheclean -p /var/cache/apache2/mod_cache_disk/ -a

The following command manually cleans the cache and ensure the size is not larger than 1000MB. The -l switch specifies the resulting cache size; the -v displays verbose results.

sudo htcacheclean -p /var/cache/apache2/mod_cache_disk/ -l 1000M -v

This command runs the cache cleanup in a daemon; it will clean our cache directory every 30 minutes and make sure that it will not get bigger than 1000MB.

htcacheclean -d30 -n -t -p /var/cache/apache2/mod_disk_cache -l 1000M -i

To learn more about htcacheclean, take a look at:

man htcacheclean

The apache2-utils install may have already installed the apache-htcacheclean service. Examine the status and runtime parameters of the service with systemctl.

sudo systemctl status apache-htcacheclean

To change the service's runtime parameters, edit the file ```/etc/default/apache-htcacheclean`` and change the default values. Start or stop the service with systemctl as required for your installation.

cleancache.sh

The cleancache.sh script (located in /usr/local/bin) is front-end to htcacheclean. It provides a mechanism to query and clear the cache of URLs matching a regex expression. Here are some examples:

cleancache.sh --help
Output
Usage:
     -p|--path                  Path to cache folder, default: /var/cache/apache2/mod_cache_disk
     -e|--expression            Regular expression for locating URLs to delete
     -l|--list                  Lists the urls in the cache
    -dr|--dry-run               Lists urls that would be deleted
     -h|--help                  Displays this help
     -v|--verbose               Displays verbose output
    -nc|--no-colour             Disables colour output
    -cr|--cron                  Run silently unless we encounter an error

Listing the GetCapabilites documents in the cache:

cleancache.sh -e GetCap -dr
Output
Sudo: Updating cached credentials ...
[sudo] password for xxxxx:
https://mapserver.northernhorizon.org:443/wms?service=WMS&request=GetCapabilities&vers

Clearing the GetCapabilites documents in the cache:

cleancache.sh -e GetCap
Output
Sudo: Updating cached credentials ...
Removed: https://mapserver.northernhorizon.org:443/wms?service=WMS&request=GetCapabilities&vers

Removing metatile entries from the Apache cache after seeding the MapCache with tiles:

cleancache.sh -e "localhost"

Other useful commands

These alias commands may be useful for querying the Apache disk cache:

# Apache Cache tools
alias cache-size='sudo du -sh /var/cache/apache2/mod_cache_disk/'
alias cache-list='sudo /usr/bin/htcacheclean -p /var/cache/apache2/mod_cache_disk/ -a'
alias cache-list-details='sudo /usr/bin/htcacheclean -p /var/cache/apache2/mod_cache_disk/ -A'

MapServer Install

Install GDAL

GDAL is used to prepare your data for MapServer. GDAL is also used by MapServer itself also to convert and serve imagery and elevations in the desired formats:

sudo apt-get install gdal-bin

Install MapServer

sudo apt-get install cgi-mapserver mapserver-bin

Configure Apache to run MapServer CGI Configure Apache to run MapServer on the /wms endpoint.

Enable the mod-cgid module:

sudo a2enmod cgid

Add the following content to your Apache configuration file (e.g., /etc/apache2/sites-enabled/000-default.conf). Note that the MS_MAPFILE variable below refers to an imagery.map file at /opt/mapserver/map/. We will create that in the next configuration step.

Alias /wms /usr/lib/cgi-bin/mapserv
<Location /wms>
	SetHandler cgi-script
	Options ExecCGI
	SetEnv MS_MAPFILE /opt/mapserver/map/imagery.map
</Location>

Configure MapServer to serve imagery

We're going to configure MapServer to serve RASTER data. See the MapServer Raster Data documentation for more information about what we are accomplishing with the following.

Prepare the folders used by MapServer:

sudo mkdir -p /opt/mapserver/map
sudo mkdir -p /opt/mapserver/map/layers
sudo mkdir -p /opt/mapserver/data
sudo mkdir -p /opt/mapserver/tmp
sudo mkdir -p /opt/mapserver/templates

Ensure the MapServer tmp folder can be written to by Apache:

sudo chown www-data:www-data /opt/mapserver/tmp/

Create (or copy) the imagery.map file and the individual layer files (*.lay) for Landsat and Blue Marble:

sudo touch /opt/mapserver/map/imagery.map
sudo touch /opt/mapserver/map/layers/i3.lay
sudo touch /opt/mapserver/map/layers/bm200401.lay
sudo touch /opt/mapserver/map/layers/bm200402.lay
sudo touch /opt/mapserver/map/layers/bm200403.lay
sudo touch /opt/mapserver/map/layers/bm200404.lay
sudo touch /opt/mapserver/map/layers/bm200405.lay
sudo touch /opt/mapserver/map/layers/bm200406.lay
sudo touch /opt/mapserver/map/layers/bm200407.lay
sudo touch /opt/mapserver/map/layers/bm200408.lay
sudo touch /opt/mapserver/map/layers/bm200409.lay
sudo touch /opt/mapserver/map/layers/bm200410.lay
sudo touch /opt/mapserver/map/layers/bm200411.lay
sudo touch /opt/mapserver/map/layers/bm200412.lay

Edit the .map files and layer files:

sudo nano /opt/mapserver/map/imagery.map

... See the Reference files section below for file contents

Tile Indexing

Example command for building a tile index against the AWS compatible storage in the emxsys cloud space:

aws s3 ls emxsys/elevations/ned10m/optimized --recursive --endpoint=https://sfo2.digitaloceanspaces.com | awk '{ printf "/vsicurl/https://emxsys.sfo2.digitaloceanspaces.com/%s\n",  $4}' | xargs -n1 gdaltindex ned10m-index.shp

MapCache

sudo apt-get install libapache2-mod-mapcache mapcache-tools

Edit host file and add:

# MapCache /mapcache endpoint
<IfModule mapcache_module>
        <Directory /opt/mapcache>
                Require all granted
        </Directory>
        MapCacheAlias /mapcache "/opt/mapcache/mapcache.xml"
</IfModule>

Seeding the cache:

Commands for seeding the caches:

sudo -u www-data mapcache_seed -m seed -c /opt/mapcache/mapcache.xml -t BlueMarble-200405 -g WGS84-9 -z 0,8 --nthreads 8
sudo -u www-data mapcache_seed -m seed -c /opt/mapcache/mapcache.xml -t BMNGLandsat -g WGS84-14 -z 2,4 --metasize 1,1 --nthreads 8
sudo -u www-data mapcache_seed -m seed -c /opt/mapcache/mapcache.xml -t BMNGLandsat -g WGS84-14 -z 5,7 --nthreads 8

Reference Files

Apache

Apache Server Configuration

/etc/apache2/apache.conf

Note the ProxyPass configurations.

# This is the main Apache server configuration file.  It contains the
# configuration directives that give the server its instructions.
# See http://httpd.apache.org/docs/2.4/ for detailed information about
# the directives and /usr/share/doc/apache2/README.Debian about Debian specific
# hints.
#
#
# Summary of how the Apache 2 configuration works in Debian:
# The Apache 2 web server configuration in Debian is quite different to
# upstream's suggested way to configure the web server. This is because Debian's
# default Apache2 installation attempts to make adding and removing modules,
# virtual hosts, and extra configuration directives as flexible as possible, in
# order to make automating the changes and administering the server as easy as
# possible.

# It is split into several files forming the configuration hierarchy outlined
# below, all located in the /etc/apache2/ directory:
#
#       /etc/apache2/
#       |-- apache2.conf
#       |       `--  ports.conf
#       |-- mods-enabled
#       |       |-- *.load
#       |       `-- *.conf
#       |-- conf-enabled
#       |       `-- *.conf
#       `-- sites-enabled
#               `-- *.conf
#
#
# * apache2.conf is the main configuration file (this file). It puts the pieces
#   together by including all remaining configuration files when starting up the
#   web server.
#
# * ports.conf is always included from the main configuration file. It is
#   supposed to determine listening ports for incoming connections which can be
#   customized anytime.
#
# * Configuration files in the mods-enabled/, conf-enabled/ and sites-enabled/
#   directories contain particular configuration snippets which manage modules,
#   global configuration fragments, or virtual host configurations,
#   respectively.
#
#   They are activated by symlinking available configuration files from their
#   respective *-available/ counterparts. These should be managed by using our
#   helpers a2enmod/a2dismod, a2ensite/a2dissite and a2enconf/a2disconf. See
#   their respective man pages for detailed information.
#
# * The binary is called apache2. Due to the use of environment variables, in
#   the default configuration, apache2 needs to be started/stopped with
#   /etc/init.d/apache2 or apache2ctl. Calling /usr/bin/apache2 directly will not
#   work with the default configuration.


# Global configuration
#

#
# ServerRoot: The top of the directory tree under which the server's
# configuration, error, and log files are kept.
#
# NOTE!  If you intend to place this on an NFS (or otherwise network)
# mounted filesystem then please read the Mutex documentation (available
# at <URL:http://httpd.apache.org/docs/2.4/mod/core.html#mutex>);
# you will save yourself a lot of trouble.
#
# Do NOT add a slash at the end of the directory path.
#
#ServerRoot "/etc/apache2"

#
# The accept serialization lock file MUST BE STORED ON A LOCAL DISK.
#
#Mutex file:${APACHE_LOCK_DIR} default

#
# The directory where shm and other runtime files will be stored.
#

DefaultRuntimeDir ${APACHE_RUN_DIR}

#
# PidFile: The file in which the server should record its process
# identification number when it starts.
# This needs to be set in /etc/apache2/envvars
#
PidFile ${APACHE_PID_FILE}

#
# Timeout: The number of seconds before receives and sends time out.
# Default is 300; changed to 600 to build MapCache top level tiles.
# OK to change back.
Timeout 600

#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive On

#
# MaxKeepAliveRequests: The maximum number of requests to allow
# during a persistent connection. Set to 0 to allow an unlimited amount.
# We recommend you leave this number high, for maximum performance.
#
MaxKeepAliveRequests 100

#
# KeepAliveTimeout: Number of seconds to wait for the next request from the
# same client on the same connection.
#
KeepAliveTimeout 5


# These need to be set in /etc/apache2/envvars
User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}

#
# HostnameLookups: Log the names of clients or just their IP addresses
# e.g., www.apache.org (on) or 204.62.129.132 (off).
# The default is off because it'd be overall better for the net if people
# had to knowingly turn this feature on, since enabling it means that
# each client request will result in AT LEAST one lookup request to the
# nameserver.
#
HostnameLookups Off

# ErrorLog: The location of the error log file.
# If you do not specify an ErrorLog directive within a <VirtualHost>
# container, error messages relating to that virtual host will be
# logged here.  If you *do* define an error logfile for a <VirtualHost>
# container, that host's errors will be logged there and not here.
#
ErrorLog ${APACHE_LOG_DIR}/error.log

#
# LogLevel: Control the severity of messages logged to the error_log.
# Available values: trace8, ..., trace1, debug, info, notice, warn,
# error, crit, alert, emerg.
# It is also possible to configure the log level for particular modules, e.g.
# "LogLevel info ssl:warn"
#
LogLevel warn

# Include module configuration:
IncludeOptional mods-enabled/*.load
IncludeOptional mods-enabled/*.conf

# Include list of ports to listen on
Include ports.conf


# Sets the default security model of the Apache2 HTTPD server. It does
# not allow access to the root filesystem outside of /usr/share and /var/www.
# The former is used by web applications packaged in Debian,
# the latter may be used for local directories served by the web server. If
# your system is serving content from a sub-directory in /srv you must allow
# access here, or in any related virtual host.
<Directory />
        Options FollowSymLinks
        AllowOverride None
        Require all denied
</Directory>

<Directory /usr/share>
        AllowOverride None
        Require all granted
</Directory>

<Directory /var/www/>
        Options Indexes FollowSymLinks
        AllowOverride None
        Require all granted
</Directory>

#<Directory /srv/>
#       Options Indexes FollowSymLinks
#       AllowOverride None
#       Require all granted
#</Directory>




# AccessFileName: The name of the file to look for in each directory
# for additional configuration directives.  See also the AllowOverride
# directive.
#
AccessFileName .htaccess

#
# The following lines prevent .htaccess and .htpasswd files from being
# viewed by Web clients.
#
<FilesMatch "^\.ht">
        Require all denied
</FilesMatch>


#
# The following directives define some format nicknames for use with
# a CustomLog directive.
#
# These deviate from the Common Log Format definitions in that they use %O
# (the actual bytes sent including headers) instead of %b (the size of the
# requested file), because the latter makes it impossible to detect partial
# requests.
#
# Note that the use of %{X-Forwarded-For}i instead of %h is not recommended.
# Use mod_remoteip instead.
#
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent

# Include of directories ignores editors' and dpkg's backup files,
# see README.Debian for details.

# Include generic snippets of statements
IncludeOptional conf-enabled/*.conf

# Include the virtual host configurations:
IncludeOptional sites-enabled/*.conf

# Set the global server name
ServerName mapserver-01



# Proxies to remote servers

SSLProxyEngine On
ProxyRequests Off

# OpenTopoMap WMTS
# Add a Cache-Control header to support client-side caching.
<LocationMatch "/opentopomap/">
        ProxyPass https://a.tile.opentopomap.org//
        Header set Cache-Control "max-age=2419200, public"
        Header set Access-Control-Allow-Origin "*"
</LocationMatch>

# USGS Topos and NAIP (dynamic) add 4 week expiration
<LocationMatch "/USGSTopo/">
        ProxyPass https://basemap.nationalmap.gov:443/arcgis/services/USGSTopo/
        Header set Cache-Control "max-age=2419200, public"
</LocationMatch>

<LocationMatch "/USGSImageryTopo/">
        ProxyPass https://basemap.nationalmap.gov:443/arcgis/services/USGSImageryTopo/
        Header set Cache-Control "max-age=2419200, public"
</LocationMatch>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

Virtual Hosts

/etc/apache2/sites-enabled/default-ssl.conf

<IfModule mod_ssl.c>
        <VirtualHost _default_:443>

                ServerName mapserver.northernhorizon.org
                ServerAdmin azhao@northernacademy.org

                DocumentRoot /var/www/html

                # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
                # error, crit, alert, emerg.
                # It is also possible to configure the loglevel for particular
                # modules, e.g.
                #LogLevel info ssl:warn

                ErrorLog ${APACHE_LOG_DIR}/error.log
                CustomLog ${APACHE_LOG_DIR}/access.log combined

                #   SSL Engine Switch:
                #   Enable/Disable SSL for this virtual host.
                SSLEngine on

                #   SSL Engine Options:
                #   Set various options for the SSL engine.
                #   o FakeBasicAuth:
                #        Translate the client X.509 into a Basic Authorisation.  This means that
                #        the standard Auth/DBMAuth methods can be used for access control.  The
                #        user name is the `one line' version of the client's X.509 certificate.
                #        Note that no password is obtained from the user. Every entry in the user
                #        file needs this password: `xxj31ZMTZzkVA'.
                #   o ExportCertData:
                #        This exports two additional environment variables: SSL_CLIENT_CERT and
                #        SSL_SERVER_CERT. These contain the PEM-encoded certificates of the
                #        server (always existing) and the client (only existing when client
                #        authentication is used). This can be used to import the certificates
                #        into CGI scripts.
                #   o StdEnvVars:
                #        This exports the standard SSL/TLS related `SSL_*' environment variables.
                #        Per default this exportation is switched off for performance reasons,
                #        because the extraction step is an expensive operation and is usually
                #        useless for serving static content. So one usually enables the
                #        exportation for CGI and SSI requests only.
                #   o OptRenegotiate:
                #        This enables optimized SSL connection renegotiation handling when SSL
                #        directives are used in per-directory context.
                #SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
                <FilesMatch "\.(cgi|shtml|phtml|php)$">
                                SSLOptions +StdEnvVars
                </FilesMatch>
                <Directory /usr/lib/cgi-bin>
                                SSLOptions +StdEnvVars
                </Directory>

                # Let's Encyrpt certificates:
                SSLCertificateFile      /etc/letsencrypt/live/mapserver.northernhorizon.org/fullchain.pem
                SSLCertificateKeyFile /etc/letsencrypt/live/mapserver.northernhorizon.org/privkey.pem
                Include /etc/letsencrypt/options-ssl-apache.conf

		# BDS - MapServer configurations begin here.

		# Enable CORS support
		Header set Access-Control-Allow-Origin "*"

		# Apache caching setup required for caching MapServer tiles
		#  Leave the CacheQuickHandler off for complete processing of caching rules:
		CacheQuickHandler off
		#  Setup a locking mechanism based on Apache docs:
		CacheLock on
		CacheLockPath /tmp/mod_cache-lock
		CacheLockMaxAge 5
		#  Don't store cookies in the cache to prevent leaking of user-specific cookies:
		CacheIgnoreHeaders Set-Cookie
		#  Web WorldWind requests require CacheIgnoreCacheControl to be enabled to
		#  obtain cache hits. This tells the server to attempt to serve the resource
		#  from the cache even if the request contains no-cache header values:
		CacheIgnoreCacheControl On


		# MapServer requires CGI
		ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
		<Directory "/usr/lib/cgi-bin">
			AllowOverride None
			Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
			Order allow,deny
			Allow from all
		</Directory>

		# MapServer /wms endpoint
		Alias /wms /usr/lib/cgi-bin/mapserv
		<Location /wms>
			CacheEnable disk
			CacheHeader on

			CacheDefaultExpire 600
			CacheMaxExpire 86400
			CacheLastModifiedFactor 0.5

			ExpiresActive on
			ExpiresDefault "access plus 1 week"

			Header merge Cache-Control public

			SetHandler cgi-script
			Options ExecCGI
			SetEnv MS_MAPFILE /opt/mapserver/map/imagery.map
		</Location>

		# MapServer /worldwind25/wms endpoint
		Alias /worldwind25/wms /usr/lib/cgi-bin/mapserv
		<Location /worldwind25/wms>
			CacheEnable disk
			CacheHeader on

			CacheDefaultExpire 600
			CacheMaxExpire 86400
			CacheLastModifiedFactor 0.5

			ExpiresActive on
			ExpiresDefault "access plus 1 week"

			Header merge Cache-Control public
			# Header set Cache-Control "max-age=290304000, public"

			SetHandler cgi-script
			Options ExecCGI
			SetEnv MS_MAPFILE /opt/mapserver/map/worldwind25.map
		</Location>

		# MapServer /worldwind26/elev endpoint
		Alias /worldwind26/elev /usr/lib/cgi-bin/mapserv
		<Location /worldwind26/elev>
			CacheEnable disk
			CacheHeader on

			CacheDefaultExpire 600
			CacheMaxExpire 86400
			CacheLastModifiedFactor 0.5

			ExpiresActive on
			ExpiresDefault "access plus 1 week"

			Header merge Cache-Control public
			# Header set Cache-Control "max-age=290304000, public"

			SetHandler cgi-script
			Options ExecCGI
			SetEnv MS_MAPFILE /opt/mapserver/map/worldwind26.map
		</Location>

		# MapServer /elev endpoint
		Alias /elev /usr/lib/cgi-bin/mapserv
		<Location /elev>
			CacheEnable disk
			CacheHeader on

			CacheDefaultExpire 600
			CacheMaxExpire 86400
			CacheLastModifiedFactor 0.5

			ExpiresActive on
			ExpiresDefault "access plus 1 week"

			Header merge Cache-Control public
			# Header set Cache-Control "max-age=290304000, public"

			SetHandler cgi-script
			Options ExecCGI
			SetEnv MS_MAPFILE /opt/mapserver/map/elevations.map
		</Location>


		# MapCache /mapcache endpoint
		<IfModule mapcache_module>
			<Directory /opt/mapcache>
				Require all granted
			</Directory>
			MapCacheAlias /mapcache "/opt/mapcache/mapcache.xml"
		</IfModule>

        </VirtualHost>
</IfModule>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

/etc/apache2/sites-enabled/000-default.conf

<VirtualHost *:80>
        # The ServerName directive sets the request scheme, hostname and port that
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        ServerName mapserver.northernhorizon.org
        ServerAdmin azhao@northernacademy.org

        DocumentRoot /var/www/html

        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # For most configuration files from conf-available/, which are
        # enabled or disabled at a global level, it is possible to
        # include a line for only one particular virtual host. For example the
        # following line enables the CGI configuration for this host only
        # after it has been globally disabled with "a2disconf".
        #Include conf-available/serve-cgi-bin.conf

        # BDS - MapServer configurations begin here.

        # Enable CORS support
        Header set Access-Control-Allow-Origin "*"

        # Apache caching setup required for caching MapServer tiles
        #  Leave the CacheQuickHandler off for complete processing of caching rules:
        CacheQuickHandler off
        #  Setup a locking mechanism based on Apache docs:
        CacheLock on
        CacheLockPath /tmp/mod_cache-lock
        CacheLockMaxAge 5
        #  Don't store cookies in the cache to prevent leaking of user-specific cookies:
        CacheIgnoreHeaders Set-Cookie
        #  Web WorldWind requests require CacheIgnoreCacheControl to be enabled to
        #  obtain cache hits. This tells the server to attempt to serve the resource
        #  from the cache even if the request contains no-cache header values:
        CacheIgnoreCacheControl On


        # MapServer requires CGI
        ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
        <Directory "/usr/lib/cgi-bin">
                AllowOverride None
                Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
                Order allow,deny
                Allow from all
        </Directory>

        # MapServer /wms endpoint
        Alias /wms /usr/lib/cgi-bin/mapserv
        <Location /wms>
                CacheEnable disk
                CacheHeader on

                CacheDefaultExpire 600
                CacheMaxExpire 86400
                CacheLastModifiedFactor 0.5

                ExpiresActive on
                ExpiresDefault "access plus 1 week"

                Header merge Cache-Control public

                SetHandler cgi-script
                Options ExecCGI
                SetEnv MS_MAPFILE /opt/mapserver/map/imagery.map
        </Location>

        # MapServer /worldwind25/wms endpoint
        Alias /worldwind25/wms /usr/lib/cgi-bin/mapserv
        <Location /worldwind25/wms>
                CacheEnable disk
                CacheHeader on

                CacheDefaultExpire 600
                CacheMaxExpire 86400
                CacheLastModifiedFactor 0.5

                ExpiresActive on
                ExpiresDefault "access plus 1 week"

                Header merge Cache-Control public
                # Header set Cache-Control "max-age=290304000, public"

                SetHandler cgi-script
                Options ExecCGI
                SetEnv MS_MAPFILE /opt/mapserver/map/worldwind25.map
        </Location>

        # MapServer /worldwind26/elev endpoint
        Alias /worldwind26/elev /usr/lib/cgi-bin/mapserv
        <Location /worldwind26/elev>
                CacheEnable disk
                CacheHeader on

                CacheDefaultExpire 600
                CacheMaxExpire 86400
                CacheLastModifiedFactor 0.5

                ExpiresActive on
                ExpiresDefault "access plus 1 week"

                Header merge Cache-Control public
                # Header set Cache-Control "max-age=290304000, public"

                SetHandler cgi-script
                Options ExecCGI
                SetEnv MS_MAPFILE /opt/mapserver/map/worldwind26.map
        </Location>

        # MapServer /elev endpoint
        Alias /elev /usr/lib/cgi-bin/mapserv
        <Location /elev>
                CacheEnable disk
                CacheHeader on

                CacheDefaultExpire 600
                CacheMaxExpire 86400
                CacheLastModifiedFactor 0.5

                ExpiresActive on
                ExpiresDefault "access plus 1 week"

                Header merge Cache-Control public
                # Header set Cache-Control "max-age=290304000, public"

                SetHandler cgi-script
                Options ExecCGI
                SetEnv MS_MAPFILE /opt/mapserver/map/elevations.map
        </Location>


        # MapCache /mapcache endpoint
        <IfModule mapcache_module>
                <Directory /opt/mapcache>
                        Require all granted
                </Directory>
                MapCacheAlias /mapcache "/opt/mapcache/mapcache.xml"
        </IfModule>

	# Forward to https
	RewriteEngine on
	RewriteCond %{SERVER_NAME} =mapserver.northernhorizon.org
	RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
	
</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

Apache Cache

/etc/default/apache-htcacheclean

# This file must only contain KEY=VALUE lines. Do not use advanced
# shell script constructs!

## run mode: cron, daemon
## run in daemon mode or as daily cron job
## default: daemon
HTCACHECLEAN_MODE=daemon

## cache size
HTCACHECLEAN_SIZE=3000M

## interval: if in daemon mode, clean cache every x minutes
HTCACHECLEAN_DAEMON_INTERVAL=120

## path to cache
## must be the same as in CacheRoot directive
#HTCACHECLEAN_PATH=/var/cache/apache2/mod_cache_disk

## additional options:
## -n : be nice
## -t : remove empty directories
HTCACHECLEAN_OPTIONS="-n"

MapServer

Map Files

/opt/mapserver/map/imagery.map

MAP

  NAME ""
  STATUS ON
  SIZE 800 600
  #SYMBOLSET "../etc/symbols.txt"
  EXTENT -180 -90 180 90
  UNITS DD
  SHAPEPATH "../data"
  IMAGECOLOR 255 255 255
  #FONTSET "../etc/fonts.txt"

  DEBUG 1
  CONFIG "MS_ERRORFILE" "../tmp/ms_error.txt"


  WEB
    IMAGEPATH "/opt/mapserver/tmp/"
    IMAGEURL "/ms_tmp/"
    METADATA
      "ows_title"           "Northern Horizon Imagery Server"
      "ows_abstract"        "WMS Service for Northern Academy and the WorldWind Explorer"
      "ows_onlineresource"  "https://mapserver.northernhorizon.org/wms"
      "ows_enable_request"  "*"
      "ows_srs"             "EPSG:4326 EPSG:4269 EPSG:3857"
      "ows_updatesequence"  "2014-05-30T16:26:00Z"
      "ows_sld_enabled"     "false"
      "wms_contactperson"   "Anson Zhao"
      "wms_contactorganization" "Northern Academy"
      "wms_contactPosition" " "
      "wms_contactelectronicmailaddress" "azhao@northernacademy.org"
    END
    TEMPLATE "../templates/blank.html"
  END

  #define your output projection
  PROJECTION
    "init=epsg:4326"
  END

  #define output formats
  OUTPUTFORMAT
    NAME "png"
    DRIVER AGG/PNG
    MIMETYPE "image/png"
    IMAGEMODE RGB
    EXTENSION "png"
    FORMATOPTION "GAMMA=0.75"
  END

  OUTPUTFORMAT
    NAME "GTiff"
    DRIVER GDAL/GTiff
    MIMETYPE "image/tiff"
    IMAGEMODE RGB
    EXTENSION "tif"
  END
  OUTPUTFORMAT
    NAME "jpeg"
    DRIVER AGG/JPEG
    MIMETYPE "image/jpeg"
    IMAGEMODE RGB
    EXTENSION "jpg"
    FORMATOPTION "GAMMA=0.75"
    FORMATOPTION "QUALITY=95"
  END

  #OUTPUTFORMAT
  #  NAME "jpeg_png"
  #  MIMETYPE "image/vnd.jpeg-png"
  #  DRIVER AGG/MIXED
  #  IMAGEMODE RGBA
  #  FORMATOPTION "TRANSPARENT_FORMAT=png"
  #  FORMATOPTION "OPAQUE_FORMAT=jpeg"
  #END

  # JPEG2000 is not supported without a customized build of GDAL with DDS enabled
  #OUTPUTFORMAT
  #  NAME "JPEG2000"
  #  DRIVER "GDAL/JPEG2000"
  #  MIMETYPE "image/jp2k"
  #  IMAGEMODE "RGB"
  #  EXTENSION "jp2"
  #END

  # DDS is not supported without a customized build of GDAL with DDS enabled
  #OUTPUTFORMAT
  #  NAME "DDS"
  #  DRIVER GDAL/dds
  #  MIMETYPE "image/dds"
  #  IMAGEMODE RGBA
  #  EXTENSION "dds"
  #  FORMATOPTION "QUALITY=NORMAL" # Should be SUPERFAST, FAST, NORMAL (default), BETTER, UBER
  #  FORMATOPTION "FORMAT=DXT3" # Should be DXT1, DXT1A, DXT3 (default) or DXT5
  #END

  #
  # Start of layer definitions
  #

  INCLUDE "./layers/bm200401.lay"
  INCLUDE "./layers/bm200402.lay"
  INCLUDE "./layers/bm200403.lay"
  INCLUDE "./layers/bm200404.lay"
  INCLUDE "./layers/bm200405.lay"
  INCLUDE "./layers/bm200406.lay"
  INCLUDE "./layers/bm200407.lay"
  INCLUDE "./layers/bm200408.lay"
  INCLUDE "./layers/bm200410.lay"
  INCLUDE "./layers/bm200411.lay"
  INCLUDE "./layers/bm200412.lay"

  INCLUDE "./layers/i3.lay"

  #INCLUDE "earth-at-night.lay"

END # Map File

/opt/mapserver/map/elevations.map

MAP

  NAME ""                 # Short name prepended to map, legend and scalebar filenames
  STATUS ON
  SIZE 800 600
  EXTENT -180 -90 180 90
  UNITS DD
  SHAPEPATH "../data"      # Path to the directory holding the shapefiles or tiles
  IMAGECOLOR 255 255 255   # Background color for transparency

  CONFIG "MS_ERRORFILE" "../tmp/ms_error.txt"
  DEBUG 0 # 0-5; 0=Off 1-5=level of detail

  WEB
    IMAGEPATH "/opt/mapserver/tmp/" # Path for temp files and images; writable by www-data
    IMAGEURL "/ms_tmp/"             # Base URL for IMAGEPATH.
    METADATA
      "ows_title"           "Northern Horizon Elevation Server"
      "ows_abstract"        "WMS elevation server for Northern Academy and the WorldWind Explorer"
      "ows_onlineresource"  "https://mapserver.northernhorizon.org/elev"
      "ows_enable_request"  "*"
      "ows_srs"             "EPSG:4326 EPSG:4269 EPSG:3857"
      "ows_updatesequence"  "2015-02-27T16:26:00Z"
      "wms_contactperson"   "Anson Zhao"
      "wms_contactorganization" "Northern Academy"
      "wms_contactPosition" " "
      "wms_contactelectronicmailaddress" "azhao@northernacademy.org"
    END
    TEMPLATE "../templates/blank.html"

  END

  #define your output projection
  PROJECTION
    "init=epsg:4326"
  END

  #define output formats
  OUTPUTFORMAT
    NAME "png"
    DRIVER AGG/PNG
    MIMETYPE "image/png"
    IMAGEMODE RGB
    EXTENSION "png"
    FORMATOPTION "GAMMA=0.75"
  END

  OUTPUTFORMAT
    NAME "bil"
    DRIVER "GDAL/EHdr"
    MIMETYPE "application/bil16"
    IMAGEMODE INT16
    EXTENSION "bil"
  END

  OUTPUTFORMAT
    NAME GEOTIFF_16
    DRIVER "GDAL/GTiff"
    MIMETYPE "image/tiff"
    IMAGEMODE INT16
    EXTENSION "tif"
  END

  #
  # Start of layer definitions
  #

#  INCLUDE "./layers/ned10m.lay"
#  INCLUDE "./layers/asterv2.lay"
  INCLUDE "./layers/srtm-cgiar.lay"
  INCLUDE "./layers/gebco.lay"

END # Map File

/opt/mapserver/map/worldwind25.map

MAP

  NAME ""
  STATUS ON
  SIZE 800 600
  #SYMBOLSET "../etc/symbols.txt"
  EXTENT -180 -90 180 90
  UNITS DD
  SHAPEPATH "../data"
  IMAGECOLOR 255 255 255
  #FONTSET "../etc/fonts.txt"

  DEBUG 0
  CONFIG "MS_ERRORFILE" "../tmp/ww25_error.txt"


  WEB
    IMAGEPATH "/opt/mapserver/tmp/"
    IMAGEURL "/ms_tmp/"
    METADATA
      "ows_title"           "WorldWind25 Imagery Server"
      "ows_abstract"        "WorldWind WMS server compatible with worldwind25.arc.nasa.gov layers."
      "ows_onlineresource"  "https://mapserver.northernhorizon.org/worldwind25/wms"
      "ows_enable_request"  "*"
      "ows_srs"             "EPSG:4326 EPSG:4269 EPSG:3857"
      "ows_updatesequence"  "2014-05-30T16:26:00Z"
      "ows_sld_enabled"     "false"
      "wms_contactperson"   "Anson Zhao"
      "wms_contactorganization" "Northern Academy"
      "wms_contactPosition" " "
      "wms_contactelectronicmailaddress" "azhao@northernacademy.org"
    END
    TEMPLATE "../templates/blank.html"
  END

  #define your output projection
  PROJECTION
    "init=epsg:4326"
  END

  #define output formats
  OUTPUTFORMAT
    NAME "png"
    DRIVER AGG/PNG
    MIMETYPE "image/png"
    IMAGEMODE RGB
    EXTENSION "png"
    FORMATOPTION "GAMMA=0.75"
  END

  OUTPUTFORMAT
    NAME "GTiff"
    DRIVER GDAL/GTiff
    MIMETYPE "image/tiff"
    IMAGEMODE RGB
    EXTENSION "tif"
  END

#  OUTPUTFORMAT
#    NAME "JPEG2000"
#    DRIVER "GDAL/JP2OpenJPEG"
#    MIMETYPE "image/jp2k"
#    IMAGEMODE "RGB"
#    EXTENSION "jp2"
#  END

  # DDS is not supported without a customized build of GDAL with DDS enabled
  #OUTPUTFORMAT
  #  NAME "DDS"
  #  DRIVER GDAL/dds
  #  MIMETYPE "image/dds"
  #  IMAGEMODE RGBA
  #  EXTENSION "dds"
  #  FORMATOPTION "QUALITY=NORMAL" # Should be SUPERFAST, FAST, NORMAL (default), BETTER, UBER
  #  FORMATOPTION "FORMAT=DXT3" # Should be DXT1, DXT1A, DXT3 (default) or DXT5
  #END

  #
  # Start of layer definitions
  #

  INCLUDE "layers/bm200401.lay"
  INCLUDE "layers/bm200402.lay"
  INCLUDE "layers/bm200403.lay"
  INCLUDE "layers/bm200404.lay"
  INCLUDE "layers/bm200405.lay"
  INCLUDE "layers/bm200406.lay"
  INCLUDE "layers/bm200407.lay"
  INCLUDE "layers/bm200408.lay"
  INCLUDE "layers/bm200409.lay"
  INCLUDE "layers/bm200410.lay"
  INCLUDE "layers/bm200411.lay"
  INCLUDE "layers/bm200412.lay"

  INCLUDE "layers/i3.lay"

  #INCLUDE "earth-at-night.lay"

END # Map File

/opt/mapserver/map/worldwind26.map

MAP

  NAME ""                 # Short name prepended to map, legend and scalebar filenames
  STATUS ON
  SIZE 800 600
  EXTENT -180 -90 180 90
  UNITS DD
  SHAPEPATH "../data"      # Path to the directory holding the shapefiles or tiles
  IMAGECOLOR 255 255 255   # Background color for transparency

  CONFIG "MS_ERRORFILE" "../tmp/ww26_error.txt"
  DEBUG 0 # 0-5; 0=Off 1-5=level of detail

  WEB
    IMAGEPATH "/opt/mapserver/tmp/" # Path for temp files and images; writable by www-data
    IMAGEURL "/ms_tmp/"             # Base URL for IMAGEPATH.
    METADATA
      "ows_title"           "WorldWind26 Elevation Server"
      "ows_abstract"        "WorldWind WMS elevation server compatible with worldwind26.arc.nasa.gov layers"
      "ows_onlineresource"  "https://mapserver.emxsys.net/worldwind26/wms"
      "ows_enable_request"  "*"
      "ows_srs"             "EPSG:4326 EPSG:4269 EPSG:3857"
      "ows_updatesequence"  "2015-02-27T16:26:00Z"
      "wms_contactperson"   "Anson Zhao"
      "wms_contactorganization" "Northern Academy"
      #"wms_contactPosition" ""
      "wms_contactelectronicmailaddress" "azhao@northernacademy.org"
    END
    TEMPLATE "../templates/blank.html"

  END

  #define your output projection
  PROJECTION
    "init=epsg:4326"
  END

  #define output formats
  OUTPUTFORMAT
    NAME "png"
    DRIVER AGG/PNG
    MIMETYPE "image/png"
    IMAGEMODE RGB
    EXTENSION "png"
    FORMATOPTION "GAMMA=0.75"
  END

  OUTPUTFORMAT
    NAME "bil"
    DRIVER "GDAL/EHdr"
    MIMETYPE "application/bil16"
    IMAGEMODE INT16
    EXTENSION "bil"
  END

  OUTPUTFORMAT
    NAME GEOTIFF_16
    DRIVER "GDAL/GTiff"
    MIMETYPE "image/tiff"
    IMAGEMODE INT16
    EXTENSION "tif"
  END

  #
  # Start of layer definitions
  #

  INCLUDE "./layers/ned10m.lay"
  INCLUDE "./layers/asterv2.lay"
  INCLUDE "./layers/srtm-nasa.lay"
  INCLUDE "./layers/gebco.lay"

END # Map File

Layer Files

/opt/mapserver/layers/bm200405.lay

LAYER
  PROCESSING "RESAMPLE=BILINEAR"
  NAME "BlueMarble-200405"
  METADATA
    "wms_title"          "BlueMarble Topo-Bathy May 2004"
    "wms_abstract"       "BlueMarble topographic and bathymetry imagery for May 2004"
    "wms_keywordlist"    "LastUpdate= 2013-12-12T16:26:00Z"
    "wms_opaque"         "1"
  END
  TYPE RASTER
  STATUS ON
  TILEINDEX "bluemarble/bmng-topo-bathy-200405.shp"
  TILEITEM "Location"
  TYPE RASTER
#  MINSCALEDENOM 1785714
  PROJECTION
    "init=epsg:4326"
  END
END

/opt/mapserver/map/layers/i3.lay

LAYER
  PROCESSING "RESAMPLE=BILINEAR"
  NAME "esat"
  METADATA
    "wms_title"          "ESAT"
    "wms_abstract"       "I-Cubed ESAT World Landsat7 Mosaic"
    "wms_keywordlist"    "LastUpdate= 2013-12-12T16:26:00Z"
    "wms_opaque"         "1"
  END
  TYPE RASTER
  STATUS ON
  TILEINDEX "landsat/i3-index.shp"
  TILEITEM "Location"
  TYPE RASTER
#  MINSCALEDENOM 53571
  PROJECTION
    "init=epsg:4326"
  END
  EXTENT -180 -58 180 82
  OFFSITE 0 0 0
END

/opt/mapserver/map/layers/gebco.lay

LAYER
  PROCESSING "RESAMPLE=BILINEAR"
  NAME "GEBCO"
  METADATA
    "wms_title"       "GEBCO"
    "wms_abstract"    "General Bathymetric Chart of the Oceans"
    "wms_keywordlist"    "LastUpdate= 2015-02-27T12:00:00Z"
  END
  TYPE RASTER
  STATUS ON
  TILEINDEX "gebco/gebco-index.shp" # Path is relative to SHAPEPATH
  TILEITEM "Location"
  TYPE RASTER
  #MAXSCALEDENOM 500000
  PROJECTION
    "init=epsg:4326"
  END
END

/opt/mapserver/map/layers/asterv2.lay

LAYER
  PROCESSING "RESAMPLE=BILINEAR"
  NAME "aster_v2"
  METADATA
    "wms_title"       "ASTER V2"
    "wms_abstract"    "ASTER GDEM Version2"
    "wms_keywordlist" "LastUpdate= 2013-07-02T16:26:00Z"

    "wcs_label" "ASTER version 2"
    "wcs_extent" "-180 -83 180 83"
    "wcs_size" "100 100"
    "wcs_resolution" "0.0002 0.0002"
  END
  TYPE RASTER
  STATUS ON
  TILEINDEX "asterv2/asterv2-index.shp"
  TILEITEM "Location"
  TYPE RASTER
  MAXSCALEDENOM 1000000
  PROJECTION
    "init=epsg:4326"
  END
END

/opt/mapserver/map/layers/srtm-cgiar.lay

LAYER
  PROCESSING "RESAMPLE=BILINEAR"
  NAME "srtm-cgiar"
  METADATA
    "wms_title"       "SRTM CGIAR-CSI"
    "wms_abstract"    "SRTMv4.1 from CGIAR-CSI"
  END
  TYPE RASTER
  STATUS ON
  TILEINDEX "srtm-cgiar/srtm-cgiar-index.shp"
  TILEITEM "Location"
  TYPE RASTER
  #MAXSCALEDENOM 500000
  PROJECTION
    "init=epsg:4326"
  END
END

/opt/mapserver/map/layers/srtm-nasa.lay

This layer file provides name compatablity with NASA_SRTM30 but uses SRTM-CGIAR data.

LAYER
  PROCESSING "RESAMPLE=BILINEAR"
  NAME "NASA_SRTM30_900m_Tiled"
  METADATA
    "wms_title"       "NASA_SRTM30"
    "wms_abstract"    "NASA SRTM with bathy"
    "wms_keywordlist"    "LastUpdate= 2013-07-02T16:26:00Z"
  END
  TYPE RASTER
  STATUS ON
  TILEINDEX "srtm-cgiar/srtm-cgiar-index.shp"
  TILEITEM "Location"
  TYPE RASTER
  #MAXSCALEDENOM 500000
  PROJECTION
    "init=epsg:4326"
  END
END

/opt/mapserver/map/layers/ned10m.lay

LAYER
  PROCESSING "RESAMPLE=BILINEAR"
  NAME "USGS-NED"
  METADATA
    "wms_title"       "USGS NED"
    "wms_abstract"    "USGS NED 10m"
    "wms_keywordlist"    "LastUpdate= 2013-07-02T16:26:00Z"
  END
  TYPE RASTER
  STATUS ON
  TILEINDEX "ned10m/ned10m-index.shp"
  TILEITEM "Location"
  TYPE RASTER
  MAXSCALEDENOM 500000
  PROJECTION
    "init=epsg:4326"
  END
END

MapCache

/opt/mapcache/mapcache.xml

<?xml version="1.0" encoding="UTF-8"?>

<!-- See: https://mapserver.org/mapcache/config.html for more inforation -->

<mapcache>

  <metadata>
    <title>Northern Horizon WMTS Server</title>
    <abstract>WMTS imagery for Northern Academy and the WorldWind Explorer</abstract>
    <providername>Northern Academy</providername>
    <providerurl>https://northernacademy.org</providerurl>
    <contactname>Anson Zhao</contactname>
    <contactelectronicemailaddress>azhao@northernacademy.org</contactelectronicemailaddress>

  </metadata>

 <!--
  Cache location on disk for BlueMarble imagery using default layout.
  -->
  <cache name="BMNG" type="disk">
    <base>/var/cache/mapcache/bmng</base>
    <symlink_blank/>
  </cache>

 <!--
  Cache location on disk for Landsat imagery using default layout.
  -->
  <cache name="Landsat" type="disk">
    <base>/var/cache/mapcache/landsat</base>
    <symlink_blank/>
  </cache>

  <!--
  PNG format with FAST compression quality.
  Note: <colors>256</colors> constrains the num colors for better compression but will decimate the image quality.
  -->
  <format name="PNGQ_FAST" type="PNG">
    <compression>fast</compression>
  </format>

  <!--
  PNG format with BEST compression quality
  -->
  <format name="PNGQ_BEST" type="PNG">
    <compression>best</compression>
  </format>

  <!--
  JPEG (YCbCr colorspace) format for opaque images
  -->
  <format name="JPEG_YCBCR" type="JPEG">
    <quality>95</quality>
    <photometric>ycbcr</photometric>
  </format>

  <!--
  JPEG (RGB colorspace) format for opaque images
  -->
  <format name="JPEG_RGB" type="JPEG">
    <quality>95</quality>
    <photometric>rgb</photometric>
  </format>

  <!--
  Mixed PNG/JPEG format
  -->
  <format name="PNG_JPEG_MIX" type="MIXED">
    <opaque>JPEG_YCBCR</opaque>
    <transparent>PNGQ_BEST</transparent>
  </format>

  <!--
  Default format for unspported requests
  -->
  <default_format>JPEG_YCBCR</default_format>

  <!--
  Source for BlueMarble imagery is a local WorldWind MapServer.
  -->
  <source name="BlueMarble-200405" type="wms">
    <http>
      <!-- URL of the WMS service, without any parameters -->
      <url>http:/localhost/wms?</url>
    </http>
    <!--
    Extra parameters that will be added to the GetMap requests
    -->
    <getmap>
      <params>
        <LAYERS>BlueMarble-200405</LAYERS>
        <FORMAT>image/jpeg</FORMAT>
        <TRANSPARENT>false</TRANSPARENT>
      </params>
    </getmap>
    <!-- Connction timeout. default 30 -->
    <connection_timeout>30</connection_timeout>
    <!-- Rendering and download timeout. 300 -->
    <timeout>360</timeout>
  </source>

  <!--
  Source for Landsat/BlueMarble imagery is the local MapServer instance.
  -->
  <source name="BMNGLandsat" type="wms">
    <http>
      <!-- URL of the WMS service, without any parameters -->
      <url>http://localhost/wms?</url>
    </http>
    <!--
    Extra parameters that will be added to the GetMap requests
    -->
    <getmap>
      <params>
        <LAYERS>BlueMarble-200405,esat</LAYERS>
        <FORMAT>image/jpeg</FORMAT>
        <TRANSPARENT>false</TRANSPARENT>
      </params>
    </getmap>
    <!-- Connction timeout. default 30 -->
    <connection_timeout>30</connection_timeout>
    <!-- Rendering and download timeout. 300 -->
    <timeout>360</timeout>
  </source>

  <!-- For reference, the preconfigured WGS84 Grid:
  <grid name="WGS84">
     <metadata>
        <title>GoogleCRS84Quad</title>
        <WellKnownScaleSet>urn:ogc:def:wkss:OGC:1.0:GoogleCRS84Quad</WellKnownScaleSet>
     </metadata>
     <extent>-180 -90 180 90</extent>
     <srs>EPSG:4326</srs>
     <units>dd</units>
     <size>256 256</size>
     <resolutions>0.703125000000000 0.351562500000000 0.175781250000000 8.78906250000000e-2 4.39453125000000e-2 2.19726562500000e-2 1.09863281250000e-2 5.49316406250000e-3 2.74658203125000e-3 1.37329101562500e-3 6.86645507812500e-4 3.43322753906250e-4 1.71661376953125e-4 8.58306884765625e-5 4.29153442382812e-5 2.14576721191406e-5 1.07288360595703e-5 5.36441802978516e-6</resolutions>
  </grid>
  -->

  <grid name="WGS84-9">
     <metadata>
        <title>GoogleCRS84Quad - 9 zoom levels</title>
        <WellKnownScaleSet>urn:ogc:def:wkss:OGC:1.0:GoogleCRS84Quad</WellKnownScaleSet>
     </metadata>
     <extent>-180 -90 180 90</extent>
     <srs>EPSG:4326</srs>
     <units>dd</units>
     <size>256 256</size>
     <resolutions>0.703125000000000 0.351562500000000 0.175781250000000 8.78906250000000e-2 4.39453125000000e-2 2.19726562500000e-2 1.09863281250000e-2 5.49316406250000e-3 2.74658203125000e-3</resolutions>
  </grid>

  <grid name="WGS84-14">
     <metadata>
        <title>GoogleCRS84Quad - 14 levels</title>
        <WellKnownScaleSet>urn:ogc:def:wkss:OGC:1.0:GoogleCRS84Quad</WellKnownScaleSet>
     </metadata>
     <extent>-180 -90 180 90</extent>
     <srs>EPSG:4326</srs>
     <units>dd</units>
     <size>256 256</size>
     <resolutions>0.703125000000000 0.351562500000000 0.175781250000000 8.78906250000000e-2 4.39453125000000e-2 2.19726562500000e-2 1.09863281250000e-2 5.49316406250000e-3 2.74658203125000e-3 1.37329101562500e-3 6.86645507812500e-4 3.43322753906250e-4 1.71661376953125e-4 8.58306884765625e-5</resolutions>
  </grid>

  <!--
  Tileset from the BlueMarble source and stored in the BMNG cache
  -->
  <tileset name="BlueMarble-200405">
    <!-- Use the BlueMarble remote WMS (USGS) -->
    <source>BlueMarble-200405</source>

    <!--The an 9 level WGS84 grid with EPSG:4326 support -->
    <grid>WGS84-9</grid>

    <!-- Location and layout of the cache tiles -->
    <cache>BMNG</cache>

    <!-- Format and compression of tiles in the cache -->
    <format>JPEG_YCBCR</format>

    <!-- Use meta-tiles for fewer requests -->
    <metatile>4 4</metatile>

    <!-- No metabuffer required as no labels in imagery -->
    <metabuffer>0</metabuffer>

    <!-- Set the cache-control header to 30 days -->
    <expires>2592000</expires>

    <!-- Auto-expire tiles in the cache after n seconds (upon a re-request). Default: don't expire.
    <auto_expire>7776000</auto_expire>
    -->
  </tileset>

  <!--
  Tileset from the BlueMarble/Landsat source and stored in the BMNG cache
  -->
  <tileset name="BMNGLandsat">
    <!-- Use the BlueMarble/Landsat local WMS -->
    <source>BMNGLandsat</source>

    <!--The a 14 level WGS84 grid with EPSG:4326 support -->
    <grid minzoom="2">WGS84-14</grid>

    <!-- Location and layout of the cache tiles -->
    <cache>Landsat</cache>

    <!-- Format and compression of tiles in the cache -->
    <format>JPEG_YCBCR</format>

    <!-- Use meta-tiles for fewer requests -->
    <metatile>4 4</metatile>

    <!-- No metabuffer required as no labels in imagery -->
    <metabuffer>0</metabuffer>

    <!-- Set the cache-control header to 30 days -->
    <expires>2592000</expires>

    <!-- Auto-expire tiles in the cache after n seconds days (upon a re-request). Default: don't expire.
    <auto_expire>7776000</auto_expire>
    -->
  </tileset>


  <!--
  Proxy service for unsupported WMS GetMap requests. This service should actually be called "ogc".
  It is different from the other services as it does not listen on the /wms endpoint, but
  directly on /. It will intercept WMS GetMap requests that can be satisfied from configured
  tilesets, and can optionally forward all the rest to (an)other server(s) or assemble tiles
  from the tile cache.
  Note: Do not enable "allow_client_override" for the format. if enabled it will break image/dds support -->
  -->
  <service type="wms" enabled="true">
    <!--
    Forward WMS GetCapabilities requests directly to the MapServer for a properly formatted
    document with all available services. Otherwise, we get a WMT_MS_Capabilities document
    from MapCache with just the capbilities of the tile cache.

    <forwarding_rule name="first rule">
      <param name="SERVICE" type="values">WMS</param>
      <param name="REQUEST" type="values">GetCapabilities</param>
      <http>
        <url>http://localhost/wms</url>
      </http>
    </forwarding_rule>
    -->

    <!--
    Assemble unsupported grid GetMap requests from the cache
    -->
    <full_wms>assemble</full_wms>
    <resample_mode>bilinear</resample_mode>
    <format allow_client_override="true">JPEG_YCBCR</format>
    <maxsize>4096</maxsize>
  </service>

  <!--
  Tile services. See: https://mapserver.org/mapcache/services.html
  -->
  <service type="wmts" enabled="true"/>
  <service type="tms" enabled="true"/>
  <service type="kml" enabled="true"/>
  <service type="gmaps" enabled="true"/>
  <service type="ve" enabled="true"/>
  <service type="demo" enabled="true"/>


  <locker type="disk">  <!-- this is the default -->
     <!--
     Where to put lockfiles (to block other clients while a metatile is being
     rendered). Defaults to /tmp. This location should be writable by the
     Apache user.
     -->
    <directory>/tmp</directory>

    <!-- Check back every .01 seconds -->
    <retry>0.01</retry>

    <!-- Consider a lock stale after this many seconds. May cause issues if WMS rendering time exceeds this value. Default: 120 -->
    <timeout>360</timeout>
  </locker>

  <errors>report</errors>

  <log_level>debug</log_level>

  <auto_reload>true</auto_reload>


</mapcache>

Fail2Ban

Following is an elided listing of the fail2ban configuration file showing the most relative values:

/etc/fail2ban/jail.local

#
# WARNING: heavily refactored in 0.9.0 release.  Please review and
#          customize settings for your setup.
#
# Changes:  in most of the cases you should not modify this
#           file, but provide customizations in jail.local file,
#           or separate .conf files under jail.d/ directory, e.g.:
#
# HOW TO ACTIVATE JAILS:
#
# YOU SHOULD MODIFY THIS FILE, jail.local, NOT jail.conf
#
#
.
.
.
# "ignoreself" specifies whether the local resp. own IP addresses should be ignored
# (default is true). Fail2ban will not ban a host which matches such addresses.
#ignoreself = true

# "ignoreip" can be a list of IP addresses, CIDR masks or DNS hosts. Fail2ban
# will not ban a host which matches an address in this list. Several addresses
# can be defined using space (and/or comma) separator.
ignoreip = 127.0.0.1/8 ::1 157.245.134.29

# External command that will take an tagged arguments to ignore, e.g. <ip>,
# and return true if the IP is to be ignored. False otherwise.
#
# ignorecommand = /path/to/command <ip>
ignorecommand =

# "bantime" is the number of seconds that a host is banned.
# (604800 secs = 1 week)
bantime  = 1w

# A host is banned if it has generated "maxretry" during the last "findtime"
# seconds.
findtime  = 1h

# "maxretry" is the number of failures before a host get banned.
maxretry = 3
.
.
.

Postfix

Postfix mail transfer agent configuration file:

/etc/postfix/main.cf

# See /usr/share/postfix/main.cf.dist for a commented, more complete version


# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 2 on
# fresh installs.
compatibility_level = 2



# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_security_level=may

smtp_tls_CApath=/etc/ssl/certs
smtp_tls_security_level=may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache


smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = mapserver-01
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = mapserver-01, mapserver.northernhorizon.org, localhost.localdomain, localhost
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = localhost
inet_protocols = ipv4

LogWatch

LogWatch log analyzer configuration file; note the MailTo, Detail and Range settings:

/usr/share/logwatch/default.conf/logwatch.conf

########################################################
# This was written and is maintained by:
#    Kirk Bauer <kirk@kaybee.org>
#
# Please send all comments, suggestions, bug reports,
#    etc, to kirk@kaybee.org.
#
########################################################

# NOTE:
#   All these options are the defaults if you run logwatch with no
#   command-line arguments.  You can override all of these on the
#   command-line.

# You can put comments anywhere you want to.  They are effective for the
# rest of the line.

# this is in the format of <name> = <value>.  Whitespace at the beginning
# and end of the lines is removed.  Whitespace before and after the = sign
# is removed.  Everything is case *insensitive*.

# Yes = True  = On  = 1
# No  = False = Off = 0

# Default Log Directory
# All log-files are assumed to be given relative to this directory.
LogDir = /var/log

# You can override the default temp directory (/tmp) here
TmpDir = /var/cache/logwatch

#Output/Format Options
#By default Logwatch will print to stdout in text with no encoding.
#To make email Default set Output = mail to save to file set Output = file
Output = stdout
#To make Html the default formatting Format = html
Format = text
#To make Base64 [aka uuencode] Encode = base64
Encode = none

# Input Encoding
# Logwatch assumes that the input is in UTF-8 encoding.  Defining CharEncoding
# will use iconv to convert text to the UTF-8 encoding.  Set CharEncoding
# to an empty string to use the default current locale.  If set to a valid
# encoding, the input characters are converted to UTF-8, discarding any
# illegal characters.  Valid encodings are as used by the iconv program,
# and `iconv -l` lists valid character set encodings.
# Setting CharEncoding to UTF-8 simply discards illegal UTF-8 characters.
#CharEncoding = ""

# Default person to mail reports to.  Can be a local account or a
# complete email address.  Variable Output should be set to mail, or
# --output mail should be passed on command line to enable mail feature.
MailTo = azhao@northernacademy.org bruce@emxsys.com

# WHen using option --multiemail, it is possible to specify a different
# email recipient per host processed.  For example, to send the report
# for hostname host1 to user@example.com, use:
#Mailto_host1 = user@example.com
# Multiple recipients can be specified by separating them with a space.

# Default person to mail reports from.  Can be a local account or a
# complete email address.
MailFrom = Logwatch

# if set, the results will be saved in <filename> instead of mailed
# or displayed. Be sure to set Output = file also.
#Filename = /tmp/logwatch

# Use archives?  If set to 'Yes', the archives of logfiles
# (i.e. /var/log/messages.1 or /var/log/messages.1.gz) will
# be searched in addition to the /var/log/messages file.
# This usually will not do much if your range is set to just
# 'Yesterday' or 'Today'... it is probably best used with Range = All
# By default this is now set to Yes. To turn off Archives uncomment this.
#Archives = No

# The default time range for the report...
# The current choices are All, Today, Yesterday
Range = yesterday

# The default detail level for the report.
# This can either be Low, Med, High or a number.
# Low = 0
# Med = 5
# High = 10
Detail = Low


# The 'Service' option expects either the name of a filter
# (in /usr/share/logwatch/scripts/services/*) or 'All'.
# The default service(s) to report on.  This should be left as All for
# most people.
Service = All
# You can also disable certain services (when specifying all)
Service = "-zz-network"     # Prevents execution of zz-network service, which
                            # prints useful network configuration info.
Service = "-zz-sys"         # Prevents execution of zz-sys service, which
                            # prints useful system configuration info.
Service = "-eximstats"      # Prevents execution of eximstats service, which
                            # is a wrapper for the eximstats program.
# If you only cared about FTP messages, you could use these 2 lines
# instead of the above:
#Service = ftpd-messages   # Processes ftpd messages in /var/log/messages
#Service = ftpd-xferlog    # Processes ftpd messages in /var/log/xferlog
# Maybe you only wanted reports on PAM messages, then you would use:
#Service = pam_pwdb        # PAM_pwdb messages - usually quite a bit
#Service = pam             # General PAM messages... usually not many

# You can also choose to use the 'LogFile' option.  This will cause
# logwatch to only analyze that one logfile.. for example:
#LogFile = messages
# will process /var/log/messages.  This will run all the filters that
# process that logfile.  This option is probably not too useful to
# most people.  Setting 'Service' to 'All' above analyzes all LogFiles
# anyways...

#
# By default we assume that all Unix systems have sendmail or a sendmail-like MTA.
# The mailer code prints a header with To: From: and Subject:.
# At this point you can change the mailer to anything that can handle this output
# stream.
# TODO test variables in the mailer string to see if the To/From/Subject can be set
# From here with out breaking anything. This would allow mail/mailx/nail etc..... -mgt
mailer = "/usr/sbin/sendmail -t"

#
# With this option set to a comma separated list of hostnames, only log entries
# for these particular hosts will be processed.  This can allow a log host to
# process only its own logs, or Logwatch can be run once per a set of hosts
# included in the logfiles.
# Example: HostLimit = hosta,hostb,myhost
#
# The default is to report on all log entries, regardless of its source host.
# Note that some logfiles do not include host information and will not be
# influenced by this setting.
#
#HostLimit = myhost

#
# By default /var/adm is searched after LogDir.
#AppendVarAdmToLogDirs = 1

#
# By default /var/log is to be searched after LogDir and /var/adm/ .
#AppendVarLogToLogDirs = 1

#
# By default the current working directory is searched last after LogDir, /var/adm/, and /var/log/ .
#AppendCWDToLogDirs = 1

# vi: shiftwidth=3 tabstop=3 et
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment