Skip to content

Instantly share code, notes, and snippets.

@ph
Created February 8, 2022 20:49
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ph/49fdf7224cb0047fc4fd0f86d50a6923 to your computer and use it in GitHub Desktop.
Save ph/49fdf7224cb0047fc4fd0f86d50a6923 to your computer and use it in GitHub Desktop.
diff --git a/docker-compose.yml b/docker-compose.yml
index 28fdfb32f..113d3b0d2 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -10,7 +10,7 @@ services:
fleet-server: { condition: service_healthy }
elasticsearch:
- image: docker.elastic.co/elasticsearch/elasticsearch:8.1.0-1cb1f3ad-SNAPSHOT
+ image: docker.elastic.co/elasticsearch/elasticsearch:8.2.0-5d69c4c3-SNAPSHOT
ports:
- 9200:9200
healthcheck:
@@ -39,7 +39,7 @@ services:
- "./testing/docker/elasticsearch/users_roles:/usr/share/elasticsearch/config/users_roles"
kibana:
- image: docker.elastic.co/kibana/kibana:8.1.0-1cb1f3ad-SNAPSHOT
+ image: docker.elastic.co/kibana/kibana:8.2.0-5d69c4c3-SNAPSHOT
ports:
- 5601:5601
healthcheck:
@@ -56,14 +56,14 @@ services:
XPACK_SECURITY_ENCRYPTIONKEY: "fhjskloppd678ehkdfdlliverpoolfcr"
XPACK_ENCRYPTEDSAVEDOBJECTS_ENCRYPTIONKEY: "fhjskloppd678ehkdfdlliverpoolfcr"
XPACK_FLEET_AGENTS_ELASTICSEARCH_HOST: "http://elasticsearch:9200"
- XPACK_FLEET_AGENTS_TLSCHECKDISABLED: "true"
XPACK_FLEET_REGISTRYURL: "http://package-registry:8080"
depends_on:
elasticsearch: { condition: service_healthy }
package-registry: { condition: service_healthy }
-
+ volumes:
+ - "./kbn.yml:/usr/share/kibana/config/kibana.yml"
fleet-server:
- image: docker.elastic.co/beats/elastic-agent:8.1.0-1cb1f3ad-SNAPSHOT
+ image: docker.elastic.co/beats/elastic-agent:8.2.0-5d69c4c3-SNAPSHOT
ports:
- 8220:8220
healthcheck:
diff --git a/docs/apm-data-security.asciidoc b/docs/apm-data-security.asciidoc
index 65b9d9791..c84abc47d 100644
--- a/docs/apm-data-security.asciidoc
+++ b/docs/apm-data-security.asciidoc
@@ -7,14 +7,13 @@ When it does, we offer several different ways to filter, manipulate, or obfuscat
**Built-in data filters**
-Elastic APM provides built-in support for filtering the following types of data:
+APM agents provide built-in support for filtering the following types of data:
[options="header"]
|====
|Data type |Common sensitive data
|<<filters-http-header>> |Passwords, credit card numbers, authorization, etc.
|<<filters-http-body>> |Passwords, credit card numbers, etc.
-|<<filters-personal-data>> |Client IP address and user agent.
|<<filters-real-user-data>> |URLs visited, click events, user browser errors, resources used, etc.
|<<filters-database-statements>> |Sensitive user or business information
|====
@@ -37,7 +36,7 @@ Potential overhead implications on the instrumented service
[[filtering]]
=== Built-in data filtering
-Elastic APM provides built-in support for filtering or obfuscating the following types of data.
+APM agents provide built-in support for filtering or obfuscating the following types of data.
[discrete]
[[filters-http-header]]
@@ -93,21 +92,6 @@ which means the list of sanitized fields can be updated without needing to redep
* Python: {apm-py-ref-v}/configuration.html#config-capture-body[`capture_body`]
* Ruby: {apm-ruby-ref-v}/configuration.html#config-capture-body[`capture_body`]
-[discrete]
-[[filters-personal-data]]
-==== Personal data
-
-By default, the APM Server captures some personal data associated with trace events:
-
-* `client.ip`: The client's IP address. Typically derived from the HTTP headers of incoming requests.
-`client.ip` is also used in conjunction with the {ref}/geoip-processor.html[`geoip` processor] to assign
-geographical information to trace events. To learn more about how `client.ip` is derived,
-see <<derive-client-ip>>.
-* `user_agent`: User agent data, including the client operating system, device name, vendor, and version.
-
-The capturing of this data can be turned off by setting
-**Capture personal data** to `false`.
-
[discrete]
[[filters-real-user-data]]
==== Real user monitoring data
diff --git a/docs/legacy/configuration-anonymous.asciidoc b/docs/legacy/configuration-anonymous.asciidoc
index 4e7b058db..6b3c1e12c 100644
--- a/docs/legacy/configuration-anonymous.asciidoc
+++ b/docs/legacy/configuration-anonymous.asciidoc
@@ -33,50 +33,6 @@ is set to `true`, and <<config-auth-anon-enabled,`apm-server.auth.anonymous.enab
See <<configuration-rum>> for additional RUM configuration options.
-[float]
-[[config-auth-anon-mitigating]]
-=== Mitigating malicious requests
-
-There are a few configuration variables that can mitigate the impact of malicious requests to an
-unauthenticated APM Server endpoint.
-
-Use the <<config-auth-anon-allow-agent>> and <<config-auth-anon-allow-service>> configs to ensure that the
-`agent.name` and `service.name` of each incoming request match a specified list.
-
-Additionally, the APM Server can rate-limit unauthenticated requests based on the client IP address
-(`client.ip`) of the request with <<config-auth-anon-event-limit>>.
-This allows you to specify the maximum number of requests allowed per unique IP address, per second.
-
-[float]
-[[config-auth-anon-client-ip]]
-==== Deriving an incoming request's `client.ip` address
-
-The remote IP address of an incoming request might be different
-from the end-user's actual IP address, for example, because of a proxy. For this reason,
-the APM Server attempts to derive the IP address of an incoming request from HTTP headers.
-The supported headers are parsed in the following order:
-
-1. `Forwarded`
-2. `X-Real-Ip`
-3. `X-Forwarded-For`
-
-If none of these headers are present, the remote address for the incoming request is used.
-
-[float]
-[[config-auth-anon-client-ip-concerns]]
-==== Using a reverse proxy or load balancer
-
-HTTP headers are easily modified;
-it's possible for anyone to spoof the derived `client.ip` value by changing or setting,
-for example, the value of the `X-Forwarded-For` header.
-For this reason, if any of your clients are not trusted,
-we recommend setting up a reverse proxy or load balancer in front of the APM Server.
-
-Using a proxy allows you to clear any existing IP-forwarding HTTP headers,
-and replace them with one set by the proxy.
-This prevents malicious users from cycling spoofed IP addresses to bypass the
-APM Server's rate limiting feature.
-
[float]
[[config-auth-anon]]
=== Configuration reference
@@ -122,4 +78,14 @@ Default: `1000`
==== `rate_limit.event_limit`
The maximum number of events allowed per second, per agent IP address.
+The APM Server first attempts to derive the IP address from proxy headers. The
+supported headers are parsed in the following order:
+
+- `Forwarded`
+- `X-Real-Ip`
+- `X-Forwarded-For`
+
+If none of these headers is present, the remote address for the incoming
+request is used.
+
Default: `300`
diff --git a/docs/legacy/guide/data-security.asciidoc b/docs/legacy/guide/data-security.asciidoc
index 7d74c2ff8..2ee54d5c7 100644
--- a/docs/legacy/guide/data-security.asciidoc
+++ b/docs/legacy/guide/data-security.asciidoc
@@ -10,14 +10,13 @@ When it does, we offer several different ways to filter, manipulate, or obfuscat
**Built-in data filters**
-Elastic APM provides built-in support for filtering the following types of data:
+APM agents provide built-in support for filtering the following types of data:
[options="header"]
|====
|Data type |Common sensitive data
|<<filter-http-header>> |Passwords, credit card numbers, authorization, etc.
|<<filter-http-body>> |Passwords, credit card numbers, etc.
-|<<filter-personal-data>> |Client IP address and user agent.
|<<filter-real-user-data>> |URLs visited, click events, user browser errors, resources used, etc.
|<<filter-database-statements>> |Sensitive user or business information
|====
@@ -40,7 +39,7 @@ Potential overhead implications on the instrumented service
[[built-in-filtering]]
=== Built-in data filtering
-Elastic APM provides built-in support for filtering or obfuscating the following types of data.
+APM agents provide built-in support for filtering or obfuscating the following types of data.
[discrete]
[[filter-http-header]]
@@ -96,21 +95,6 @@ which means the list of sanitized fields can be updated without needing to redep
* Python: {apm-py-ref-v}/configuration.html#config-capture-body[`capture_body`]
* Ruby: {apm-ruby-ref-v}/configuration.html#config-capture-body[`capture_body`]
-[discrete]
-[[filter-personal-data]]
-==== Personal data
-
-By default, the APM Server captures some personal data associated with trace events:
-
-* `client.ip`: The client's IP address. Typically derived from the HTTP headers of incoming requests.
-`client.ip` is also used in conjunction with the {ref}/geoip-processor.html[`geoip` processor] to assign
-geographical information to trace events. To learn more about how `client.ip` is derived,
-see <<config-auth-anon-client-ip>>.
-* `user_agent`: User agent data, including the client operating system, device name, vendor, and version.
-
-The capturing of this data can be turned off by setting
-<<`capture_personal_data`,capture_personal_data>> to `false`.
-
[discrete]
[[filter-real-user-data]]
==== Real user monitoring data
diff --git a/docs/secure-agent-communication.asciidoc b/docs/secure-agent-communication.asciidoc
index cb5a36f4c..0eb085756 100644
--- a/docs/secure-agent-communication.asciidoc
+++ b/docs/secure-agent-communication.asciidoc
@@ -227,52 +227,9 @@ Elastic APM agents can send unauthenticated (anonymous) events to the APM Server
An event is considered to be anonymous if no authentication token can be extracted from the incoming request.
By default, these anonymous requests are rejected and an authentication error is returned.
-In some cases, however, it makes sense to allow anonymous requests -- for
-example, when using the Real User Monitoring (RUM) agent running in a browser,
-or the iOS/Swift agent running in a user application,
-it is not possible to hide or protect a secret token or API key.
-Thus, enabling anonymous authentication is required to ingest client-side APM data.
+In some cases, however, it makes sense to allow anonymous requests --
+for example, when using an agent that runs on the client, like the Real User Monitoring (RUM) agent running in a browser,
+or the iOS/Swift agent running in a user application.
-[float]
-[[anonymous-auth-config]]
-=== Configuring anonymous authentication
-
-There are a few configuration variables that can mitigate the impact of malicious requests to an
-unauthenticated APM Server endpoint.
-
-Use the **Allowed anonymous agents** and **Allowed anonymous services** configs to ensure that the
-`agent.name` and `service.name` of each incoming request match a specified list.
-
-Additionally, the APM Server can rate-limit unauthenticated requests based on the client IP address
-(`client.ip`) of the request.
-This allows you to specify the maximum number of requests allowed per unique IP address, per second.
-
-[float]
-[[derive-client-ip]]
-=== Deriving an incoming request's `client.ip` address
-
-The remote IP address of an incoming request might be different
-from the end-user's actual IP address, for example, because of a proxy. For this reason,
-the APM Server attempts to derive the IP address of an incoming request from HTTP headers.
-The supported headers are parsed in the following order:
-
-1. `Forwarded`
-2. `X-Real-Ip`
-3. `X-Forwarded-For`
-
-If none of these headers are present, the remote address for the incoming request is used.
-
-[float]
-[[derive-client-ip-concerns]]
-==== Using a reverse proxy or load balancer
-
-HTTP headers are easily modified;
-it's possible for anyone to spoof the derived `client.ip` value by changing or setting,
-for example, the value of the `X-Forwarded-For` header.
-For this reason, if any of your clients are not trusted,
-we recommend setting up a reverse proxy or load balancer in front of the APM Server.
-
-Using a proxy allows you to clear any existing IP-forwarding HTTP headers,
-and replace them with one set by the proxy.
-This prevents malicious users from cycling spoofed IP addresses to bypass the
-APM Server's rate limiting feature.
+Because anyone can send anonymous events to the APM Server,
+additional configuration variables are available to rate limit the number anonymous events the APM Server processes.
diff --git a/kbn.yml b/kbn.yml
new file mode 100644
index 000000000..0fe18acfd
--- /dev/null
+++ b/kbn.yml
@@ -0,0 +1,44 @@
+server.name: kibana
+server.host: "0.0.0.0"
+
+elasticsearch.hosts: [ "http://elasticsearch:9200" ]
+
+monitoring.ui.container.elasticsearch.enabled: true
+
+xpack.fleet.registryUrl: "http://package-registry:8080"
+xpack.fleet.agents.enabled: true
+xpack.fleet.agents.elasticsearch.hosts: ["http://elasticsearch:9200"]
+xpack.fleet.agents.fleet_server.hosts: ["http://fleet-server:8220"]
+
+xpack.encryptedSavedObjects.encryptionKey: "12345678901234567890123456789012"xpack.fleet.packages:
+ - name: system
+ version: latest
+ - name: elastic_agent
+ version: latest
+ - name: fleet_server
+ version: latest
+
+xpack.fleet.agentPolicies:
+ - name: Elastic-Agent (elastic-package)
+ id: elastic-agent-managed-ep
+ is_default: true
+ is_managed: false
+ namespace: default
+ monitoring_enabled:
+ - logs
+ - metrics
+ package_policies:
+ - name: system-1
+ id: default-system
+ package:
+ name: system
+ - name: Fleet Server (elastic-package)
+ id: fleet-server-managed-ep
+ is_default_fleet_server: true
+ is_managed: false
+ namespace: default
+ package_policies:
+ - name: fleet_server-1
+ id: default-fleet-server
+ package:
+ name: fleet_server
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment