Skip to content

Instantly share code, notes, and snippets.

View poldeuce-sys's full-sized avatar

poldeuce-sys

1 follower · 0 following
View GitHub Profile
@poldeuce-sys
poldeuce-sys / useful_docker.txt
Last active November 2, 2017 07:49
Some useful things with docker
When building a container and having some issues, you can always launch interactively with
(--rm removes the container when you are done so that you can remove the image if you need to rebuild):
docker run --rm -it --entrypoint=/bin/bash <my-container>
Remember that the default 'bridge' network does not allow hostname resolution. To resolve host names, you need to
create a user network and ensure the containers attatch to that.
@poldeuce-sys
poldeuce-sys / log_trace_multi_query.json
Created October 26, 2017 12:51
With both zipkin traces and logs in ElasticSearch, you can query both places by span id or similar. For example:
GET /zipkin:span-2017-10-25,logstash-2017.10.25/_search
{
"query": {
"multi_match": {
"query": "1baed6f769e2648a",
"fields": ["id", "spanid"]
}
}
}
@poldeuce-sys
poldeuce-sys / monitoring_spring_boot_with_beats.txt
Created October 26, 2017 09:36
A couple notes on the setting a spring boot log format to include trace and span id for zipkin, and a logstash pipeline for reading these logs
######### application.properties
# trace and span taken from MDC context. Hostname could be redundant is using FileBeat on same host,
# but maybe you are feeding logs to ES in another way. Including it in the log covers this possibility
logging.pattern.file=%d{ABSOLUTE} [%X{traceId}-%X{spanId}] %-5p ${HOSTNAME} ${PID} [%t] [%C{2}] [%F:%L] - %m%n
######### pipeline/logstash.conf
# There is an old saying. You had a problem. You solved it with regex. Now you have 2 problems.
@poldeuce-sys
poldeuce-sys / monitoring_stack.yml
Last active December 31, 2023 17:00
docker compose file for default elasticsearch, logstash, kibana and zipkin
version: '3'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:5.6.3
environment:
- discovery.type=single-node
ports:
- "9200:9200"
- "9300:9300"
logstash:
@poldeuce-sys
poldeuce-sys / docker_spark.md
Created September 19, 2017 08:58
Docker Spark Notes

Networking

The docker containers create their own internal IP addresses. These are bridged, however, when the slave container connects to the master, it gives the master this address as its' location. By default this is not visible to the master container (on another machine) and so the worker is not visible.

Some examples involve parsing the hosts file and so forth however, newer versions of Docker do not require this. It is possible instead to use the docker swarm functionality to effectively create dns entries for the containers and make them visible to each other without messing with the container itself. Note the steps below are for interactively running. The service docker-compose.yml will have to do some things differently, but will get to that later.