airawat

This gist includes components of a simple workflow application that created a directory and moves files within 
hdfs to this directory; 
Emails are sent out to notify designated users of success/failure of workflow.  There is a prepare section, 
to allow re-run of the  action..the prepare essentially negates the move done by a potential prior run 
of the action.  Sample data is also included.

The sample application includes:
--------------------------------
1.  Oozie actions: hdfs action and email action 
2.  Oozie workflow controls: start, end, and kill. 

airawat

This gist includes components of a simple workflow application (oozie 3.3.0) that 
pipes data in a Hive table to mysql; 
 
The sample application includes:
--------------------------------
1.  Oozie actions: sqoop action 
2.  Oozie workflow controls: start, end, and kill. 
3.  Workflow components: job.properties and workflow.xml
4.  Sample data
5.  Prep tasks in Hive

airawat

This gist includes components of a oozie workflow - scripts/code, sample data 
and commands;  Oozie actions covered: java mapreduce action; Oozie controls 
covered: start, kill, end; The java program uses regex to parse the logs, and
also extracts the path of the mapper input directory path and includes in the 
key emitted.

Note: The reducer can be specified as a combiner as well.
 
Usecase
-------

airawat

This gist includes components of a oozie workflow - scripts/code, sample data 
and commands;  Oozie actions covered: java main action; Oozie controls 
covered: start, kill, end; The java program uses regex to parse the logs, and
also extracts pat of the mapper input directory path and includes in the key
emitted.
 
Usecase
-------
Parse Syslog generated log files to generate reports;
 

airawat

Introduction
-------------
This gist includes sample data, application components, and components to execute a bundle application.

The sample bundle application is time triggered.  The start time is defined in the bundle job.properties 
file.  The bundle application starts two coordinator applications- as defined in the bundle definition file - 
bundleConfirguration.xml.

The first coordinator job is time triggered.  The start time is defined in the bundle job.properties file.  
It runs a workflow, that includes a java main action.  The java program parses some log files and generates 

airawat

This gist includes components of a oozie workflow application - scripts/code, sample data
and commands;  Oozie actions covered: sub-workflow, email java main action, 
sqoop action (to mysql);  Oozie controls covered: decision;

Pictorial overview:
--------------------
http://hadooped.blogspot.com/2013/07/apache-oozie-part-8-subworkflow.html

Usecase:
--------

airawat

import java.util.Properties;

import org.apache.oozie.client.OozieClient;
import org.apache.oozie.client.WorkflowJob;

public class myOozieWorkflowJavaAPICall {

	public static void main(String[] args) {
		OozieClient wc = new OozieClient("http://cdh-dev01:11000/oozie");

airawat

This gist includes components of a oozie workflow - scripts/code, sample data 
and commands;  Oozie actions covered: shell action, email action

Action 1: The shell action executes a shell script that does a line count for files in a 
glob provided, and writes the line count to standard output
Action 2: The email action emails the output of action 1
 
 
Pictorial overview of job:
--------------------------

airawat

This gist demonstrates how to create a map file, from a text file.

Includes:
---------
1. Input data and script download
2. Input data-review
3. Data load commands                                
4. Java program to create the map file out of a text file in HDFS
5. Command to run Java program
6. Results of the program run to create map file 

airawat

This gist demonstrates how to create a sequence file (compressed and uncompressed), from a text file.
 
Includes:
---------
1. Input data and script download
2. Input data-review
3. Data load commands                                
4. Mapper code
5. Driver code to create the sequence file out of a text file in HDFS
6. Command to run Java program