object SparkHadoopWriter extends Logging { | |
… … | |
/** | |
* Basic work flow of this command is: | |
* 1. Driver side setup, prepare the data source and hadoop configuration for the write job to | |
* be issued. | |
* 2. Issues a write job consists of one or more executor side tasks, each of which writes all | |
* rows within an RDD partition. | |
* 3. If no exception is thrown in a task, commits that task, otherwise aborts that task; If any | |
* exception is thrown during task commitment, also aborts that task. | |
* 4. If all tasks are committed, commit the job, otherwise aborts the job; If any exception is | |
* thrown during job commitment, also aborts the job. | |
*/ | |
def write[K, V: ClassTag]... |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment