ddewaele

• No need to install Docker yourself
• Solution for multi-container / multi-host
• Integrates very well with AutoScaling / Target Groups / Loadbalancers / other AWS services
• Allows you to focus on your containers
• Cluster is lightweight concept (just a name). Builds on top of AWS constructs like ec2 / autoscaling / user-data
• Private AWS docker registry
• Supports docker compose (kinda) / Custom json format for describing task definitions
• Services as a high order component for managing tasks. Always tries to meet the desired count
• Self-healing / Auto-restart : if LB marks your service as unhealthy, it will drain / restart the service until it is up
• Devops minded : CloudFormation support / Well documented SDK / CLI tools to interact with the cluster

ddewaele

Title : Docker in the cloud (change it ?)

Description :

With the growing popularity of containerized applications, and the need or desire for companies to shift (part of) their infrastructure to the cloud, a good cloud strategy for your multi-container, multi-host Docker architecture becomes very important.

I'll give you a short intro on how Docker is used on-premise, and quickly outline some cloud scenarios where Docker will really shine.

Docker is built for the cloud and forces you to partition your applications, services and architecture in such a way. We'll discuss the various cloud provider offerings when it comes to Docker and try to derive some meaningful conclusions

ddewaele

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
Falling back to built-in command interpreter.
OK
>ets Jun  8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
Falling back to built-in command interpreter.

ddewaele

2016-12-28 16:58:33,561 INFO [Timer-Driven Process Thread-4] o.a.n.processors.standard.LogAttribute LogAttribute[id=01591074-6575-1391-9453-fc3988d88965] logging for flow file StandardFlowFileRecord[uuid=71a6ceed-312d-4d81-bb2d-f222f88f7b30,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1482936126260-2, container=default, section=2], offset=211997, length=1594],offset=0,name=system-diagnostics,size=1594]
--------------------------------------------------
Standard FlowFile Attributes
Key: 'entryDate'
        Value: 'Wed Dec 28 16:58:33 UTC 2016'
Key: 'lineageStartDate'
        Value: 'Wed Dec 28 16:58:33 UTC 2016'
Key: 'fileSize'
        Value: '1594'
FlowFile Attribute Map Content

ddewaele

We have a node nifi cluster running with 3 zookeeper instances (replicated) in a Docker Swarm Cluster.

Most of time the cluster is operating fine, but from time to time we notice that Nifi stops processing messages completely. It eventually resumes after a while (sometimes after a couple of seconds, sometimes after a couple of minutes).

When I do a grep o.a.n.c.l.e.CuratorLeaderElectionManager /srv/nifi/logs/nifi-app.log on the primary node, I see a lof of suspended / reconnected messages.

  2017-06-13 21:29:35,622 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@778f44c6 Connection State changed to SUSPENDED
  2017-06-13 21:29:35,623 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@3485c1a5 Connection State changed to SUSPENDED
  2017-06-13 21:29:35,628 INFO [Le

ddewaele

[root@CentOS-72-64-minimal etc]# virt-install --name centos7 --ram 4096 --disk path=/var/kvm/images/centos7.img,size=30 --vcpus 2 --os-type linux --os-variant rhel7 --network bridge=virbr0 --graphics none --console pty,target_type=serial --location 'http://ftp.iij.ad.jp/pub/linux/centos/7/os/x86_64/' --extra-args 'console=ttyS0,115200n8 serial'

Starting install...
Retrieving file vmlinuz...                                                                                                                                                                                                       | 5.1 MB  00:00:13     
Retrieving file initrd.img...                                                                                                                                                                                                    |  41 MB  00:00:26     
Allocating 'centos7.img'                                                                                                                                      

ddewaele

Manager:

[root@centosvm ~]# docker ps
CONTAINER ID        IMAGE                                    COMMAND                  CREATED             STATUS              PORTS               NAMES
a593d485050a        ddewaele/springboot.crud.sample:latest   "sh -c 'java $JAVA_OP"   7 minutes ago       Up 7 minutes                            springbootcrudsample.1.5syc6j4c8i3bnerdqq4e1yelm

Node1:

[root@node1 ~]# docker ps
CONTAINER ID        IMAGE                                    COMMAND                  CREATED             STATUS              PORTS               NAMES

ddewaele

[centos@ip-172-30-0-249 system]$ cat /etc/systemd/system/nifi.service 
[Unit]
Description=Apache NiFi
After=network.target

[Service]
Type=forking
User=nifi
Group=nifi
ExecStart=/opt/nifi-latest/bin/nifi.sh start

ddewaele

sudo lsblk --output NAME,TYPE,SIZE,FSTYPE,MOUNTPOINT,LABEL

NAME    TYPE SIZE FSTYPE MOUNTPOINT LABEL
xvda    disk   8G                   
└─xvda1 part   8G xfs    /          
xvdf    disk  30G                   
└─xvdf1 part  30G xfs             

ddewaele

var AWS = require('aws-sdk');
var AWSCognito = require('amazon-cognito-identity-js');
var util = require('util');

var params = {
    Username: '',
    Password: '',
    UserPoolId: 'us-west-2_aLI134pRo',
    ClientId: '3jmku5aeaqe6pdkqa5q18trjk5',
    IdentityPoolId: 'us-west-2:93be5994-d1aa-4e3a-b088-3ed28fa4b068',