Skip to content

Instantly share code, notes, and snippets.

@cbaenziger
Last active May 18, 2019 00:34
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save cbaenziger/9521ffc4e97589cd6220579857ae8642 to your computer and use it in GitHub Desktop.
Save cbaenziger/9521ffc4e97589cd6220579857ae8642 to your computer and use it in GitHub Desktop.
HDFS Balancer Oozie Workflow
nameNode=hdfs://<cluster>
jobTracker=<cluster>
queueName=defult
workflowRoot=${nameNode}/user/hdfs/hdfs_balancer
oozie.wf.application.path=${workflowRoot}/workflow.xml
<workflow-app xmlns="uri:oozie:workflow:0.5" name="Run_Balancer">
<global>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
</global>
<start to="Run_Balancer"/>
<action name="Run_Balancer">
<shell xmlns="uri:oozie:shell-action:0.3">
<exec>/usr/bin/hdfs</exec>
<argument>balancer</argument>
<argument>-Ddfs.blocksize=134217728</argument>
<argument>-Ddfs.balancer.max-size-to-move=10737418240</argument>
<argument>-Ddfs.balancer.moverThreads=50</argument>
<argument>-Ddfs.balancer.dispatcherThreads=50</argument>
<argument>-Ddfs.datanode.balance.max.concurrent.moves=10</argument>
<argument>-threshold</argument>
<argument>5</argument>
<capture-output />
</shell>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Balancer failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment