myriad demo

myriad-demo.txt

setup:  You have a multi-purpose cluster environment used for end user web traffic and in-house analytics.  In this example we have 4 running docker instances of nginx hosting our web application fronted by haproxy and 2 small and 2 medium instances of YARN running on Mapr Hadoop with MapRFS.

note: most organizations underutilize their datacenter resources by separating these two concerns.   In this demonstration we are co-locating these separate needs.
<setup scripts>
1. look at master port 80 (web app)
     - technical dive: look at /etc/haproxy/haproxy.cfg on master

2. lets run a terrasort job
<launch terrasort job>

3. just as the job is running we have scaling demand for web traffic.   We need to scale down YARN and scale up our web application.
<scale down command>
     - look at mesos UI (see tasks running that finish)
     - potential look at the command-line terrasort job change
     - look at the YARN UI for nodes
     - look at YARN UI for application status

<scale up marathon>
     - look at marathon

4. Whew... traffic has died down, its the end of the day and we want to scale our YARN up for evening processing
<scale down marathon>

<scale up 2 medium nodes of YARN>
     - look at mesos UI
     - look at YARN UI

5. show the resize of cluster of DCOS + visualizer
     - next version of myriad will be integrated in

**************
running tera sort
**************
sudo su - mapr

export YARN_HOME=/opt/mapr/hadoop/hadoop-2.5.1
export YARN_EXAMPLES=$YARN_HOME/share/hadoop/mapreduce

yarn jar $YARN_EXAMPLES/hadoop-mapreduce-examples-2.5.1-mapr-1501.jar pi 16 1000

hadoop jar $YARN_EXAMPLES/hadoop-mapreduce-examples-2.5.1-mapr-1501.jar teragen 50000000 /user/mapr/terasort-input

hadoop jar $YARN_EXAMPLES/hadoop-mapreduce-examples-2.5.1-mapr-1501.jar  terasort /user/mapr/terasort-input /user/mapr/terasort-output

******
flexing myriad
******

curl -H 'Content-Type: application/json' -X PUT -d '{ "instances":1, "profile": "small" }' http://104.154.73.141:8192/api/cluster/flexup
curl -H 'Content-Type: application/json' -X PUT -d '{ "instances":2, "profile": "medium" }' http://104.154.73.141:8192/api/cluster/flexup
** only 1 per node for now **