Created
July 7, 2016 03:58
-
-
Save sshrestha-datalicious/e3fc8ebb4916f27735a97e9fcc42136c to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ spark-submit --verbose --packages org.apache.spark:spark-streaming-kinesis-asl_2.10:1.6.2 demo_kinesis_streaming.py --awsAccessKeyId XXXXX --awsSecretKey XXXX | |
| Using properties file: null | |
| Parsed arguments: | |
| master local[*] | |
| deployMode null | |
| executorMemory null | |
| executorCores null | |
| totalExecutorCores null | |
| propertiesFile null | |
| driverMemory null | |
| driverCores null | |
| driverExtraClassPath null | |
| driverExtraLibraryPath null | |
| driverExtraJavaOptions null | |
| supervise false | |
| queue null | |
| numExecutors null | |
| files null | |
| pyFiles null | |
| archives null | |
| mainClass null | |
| primaryResource file:/home/sshrestha/KinesisMA/example/demo_kinesis_streaming.py | |
| name demo_kinesis_streaming.py | |
| childArgs [--awsAccessKeyId XXXXX --awsSecretKey XXXXX | |
| jars null | |
| packages org.apache.spark:spark-streaming-kinesis-asl_2.10:1.6.2 | |
| packagesExclusions null | |
| repositories null | |
| verbose true | |
| Spark properties used, including those specified through | |
| --conf and those from the properties file null: | |
| Ivy Default Cache set to: /home/sshrestha/.ivy2/cache | |
| The jars for the packages stored in: /home/sshrestha/.ivy2/jars | |
| :: loading settings :: url = jar:file:/usr/lib/spark/lib/spark-assembly-1.6.1-hadoop2.7.2.jar!/org/apache/ivy/core/settings/ivysettings.xml | |
| org.apache.spark#spark-streaming-kinesis-asl_2.10 added as a dependency | |
| :: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0 | |
| confs: [default] | |
| found org.apache.spark#spark-streaming-kinesis-asl_2.10;1.6.2 in central | |
| found com.amazonaws#amazon-kinesis-client;1.4.0 in central | |
| found com.amazonaws#aws-java-sdk;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-support;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-core;1.9.37 in central | |
| found commons-logging#commons-logging;1.1.3 in central | |
| found org.apache.httpcomponents#httpclient;4.3.2 in central | |
| found org.apache.httpcomponents#httpcore;4.3.2 in central | |
| found commons-codec#commons-codec;1.10 in central | |
| found com.fasterxml.jackson.core#jackson-databind;2.4.4 in central | |
| found com.fasterxml.jackson.core#jackson-annotations;2.4.4 in central | |
| found com.fasterxml.jackson.core#jackson-core;2.4.4 in central | |
| found joda-time#joda-time;2.9.3 in central | |
| found com.amazonaws#aws-java-sdk-simpledb;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-simpleworkflow;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-storagegateway;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-route53;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-s3;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-kms;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-importexport;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-sts;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-sqs;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-rds;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-redshift;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-elasticbeanstalk;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-glacier;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-sns;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-iam;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-datapipeline;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-elasticloadbalancing;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-emr;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-elasticache;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-elastictranscoder;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-ec2;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-dynamodb;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-cloudtrail;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-cloudwatch;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-logs;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-cognitoidentity;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-cognitosync;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-directconnect;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-cloudformation;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-cloudfront;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-kinesis;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-opsworks;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-ses;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-autoscaling;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-cloudsearch;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-cloudwatchmetrics;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-swf-libraries;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-codedeploy;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-config;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-lambda;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-ecs;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-cloudhsm;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-ssm;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-workspaces;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-machinelearning;1.9.37 in central | |
| found com.amazonaws#aws-java-sdk-directory;1.9.37 in central | |
| found com.google.protobuf#protobuf-java;2.5.0 in central | |
| found commons-lang#commons-lang;2.6 in central | |
| found org.spark-project.spark#unused;1.0.0 in central | |
| :: resolution report :: resolve 1732ms :: artifacts dl 44ms | |
| :: modules in use: | |
| com.amazonaws#amazon-kinesis-client;1.4.0 from central in [default] | |
| com.amazonaws#aws-java-sdk;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-autoscaling;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-cloudformation;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-cloudfront;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-cloudhsm;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-cloudsearch;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-cloudtrail;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-cloudwatch;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-cloudwatchmetrics;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-codedeploy;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-cognitoidentity;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-cognitosync;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-config;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-core;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-datapipeline;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-directconnect;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-directory;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-dynamodb;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-ec2;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-ecs;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-elasticache;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-elasticbeanstalk;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-elasticloadbalancing;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-elastictranscoder;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-emr;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-glacier;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-iam;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-importexport;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-kinesis;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-kms;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-lambda;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-logs;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-machinelearning;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-opsworks;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-rds;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-redshift;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-route53;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-s3;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-ses;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-simpledb;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-simpleworkflow;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-sns;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-sqs;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-ssm;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-storagegateway;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-sts;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-support;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-swf-libraries;1.9.37 from central in [default] | |
| com.amazonaws#aws-java-sdk-workspaces;1.9.37 from central in [default] | |
| com.fasterxml.jackson.core#jackson-annotations;2.4.4 from central in [default] | |
| com.fasterxml.jackson.core#jackson-core;2.4.4 from central in [default] | |
| com.fasterxml.jackson.core#jackson-databind;2.4.4 from central in [default] | |
| com.google.protobuf#protobuf-java;2.5.0 from central in [default] | |
| commons-codec#commons-codec;1.10 from central in [default] | |
| commons-lang#commons-lang;2.6 from central in [default] | |
| commons-logging#commons-logging;1.1.3 from central in [default] | |
| joda-time#joda-time;2.9.3 from central in [default] | |
| org.apache.httpcomponents#httpclient;4.3.2 from central in [default] | |
| org.apache.httpcomponents#httpcore;4.3.2 from central in [default] | |
| org.apache.spark#spark-streaming-kinesis-asl_2.10;1.6.2 from central in [default] | |
| org.spark-project.spark#unused;1.0.0 from central in [default] | |
| --------------------------------------------------------------------- | |
| | | modules || artifacts | | |
| | conf | number| search|dwnlded|evicted|| number|dwnlded| | |
| --------------------------------------------------------------------- | |
| | default | 62 | 0 | 0 | 0 || 62 | 0 | | |
| --------------------------------------------------------------------- | |
| :: retrieving :: org.apache.spark#spark-submit-parent | |
| confs: [default] | |
| 0 artifacts copied, 62 already retrieved (0kB/41ms) | |
| Main class: | |
| org.apache.spark.deploy.PythonRunner | |
| Arguments: | |
| file:/home/sshrestha/KinesisMA/example/demo_kinesis_streaming.py | |
| /home/sshrestha/.ivy2/jars/org.apache.spark_spark-streaming-kinesis-asl_2.10-1.6.2.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_amazon-kinesis-client-1.4.0.jar,/home/sshrestha/.ivy2/jars/org.spark-project.spark_unused-1.0.0.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.google.protobuf_protobuf-java-2.5.0.jar,/home/sshrestha/.ivy2/jars/commons-lang_commons-lang-2.6.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-support-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-simpledb-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-simpleworkflow-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-storagegateway-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-route53-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-s3-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-importexport-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-sts-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-sqs-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-rds-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-redshift-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elasticbeanstalk-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-glacier-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-iam-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-datapipeline-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elasticloadbalancing-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-emr-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elasticache-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elastictranscoder-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ec2-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-dynamodb-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-sns-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudtrail-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudwatch-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-logs-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cognitoidentity-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cognitosync-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-directconnect-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudformation-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudfront-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-kinesis-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-opsworks-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ses-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-autoscaling-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudsearch-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudwatchmetrics-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-swf-libraries-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-codedeploy-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-kms-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-config-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-lambda-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ecs-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudhsm-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ssm-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-workspaces-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-machinelearning-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-directory-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-core-1.9.37.jar,/home/sshrestha/.ivy2/jars/commons-logging_commons-logging-1.1.3.jar,/home/sshrestha/.ivy2/jars/org.apache.httpcomponents_httpclient-4.3.2.jar,/home/sshrestha/.ivy2/jars/com.fasterxml.jackson.core_jackson-databind-2.4.4.jar,/home/sshrestha/.ivy2/jars/joda-time_joda-time-2.9.3.jar,/home/sshrestha/.ivy2/jars/org.apache.httpcomponents_httpcore-4.3.2.jar,/home/sshrestha/.ivy2/jars/commons-codec_commons-codec-1.10.jar,/home/sshrestha/.ivy2/jars/com.fasterxml.jackson.core_jackson-annotations-2.4.4.jar,/home/sshrestha/.ivy2/jars/com.fasterxml.jackson.core_jackson-core-2.4.4.jar | |
| --awsAccessKeyId | |
| XXXXX | |
| --awsSecretKey | |
| XXXXX | |
| System properties: | |
| SPARK_SUBMIT -> true | |
| spark.submit.pyFiles -> /home/sshrestha/.ivy2/jars/org.apache.spark_spark-streaming-kinesis-asl_2.10-1.6.2.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_amazon-kinesis-client-1.4.0.jar,/home/sshrestha/.ivy2/jars/org.spark-project.spark_unused-1.0.0.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.google.protobuf_protobuf-java-2.5.0.jar,/home/sshrestha/.ivy2/jars/commons-lang_commons-lang-2.6.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-support-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-simpledb-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-simpleworkflow-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-storagegateway-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-route53-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-s3-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-importexport-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-sts-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-sqs-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-rds-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-redshift-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elasticbeanstalk-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-glacier-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-iam-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-datapipeline-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elasticloadbalancing-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-emr-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elasticache-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elastictranscoder-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ec2-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-dynamodb-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-sns-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudtrail-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudwatch-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-logs-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cognitoidentity-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cognitosync-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-directconnect-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudformation-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudfront-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-kinesis-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-opsworks-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ses-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-autoscaling-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudsearch-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudwatchmetrics-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-swf-libraries-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-codedeploy-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-kms-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-config-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-lambda-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ecs-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudhsm-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ssm-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-workspaces-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-machinelearning-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-directory-1.9.37.jar,/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-core-1.9.37.jar,/home/sshrestha/.ivy2/jars/commons-logging_commons-logging-1.1.3.jar,/home/sshrestha/.ivy2/jars/org.apache.httpcomponents_httpclient-4.3.2.jar,/home/sshrestha/.ivy2/jars/com.fasterxml.jackson.core_jackson-databind-2.4.4.jar,/home/sshrestha/.ivy2/jars/joda-time_joda-time-2.9.3.jar,/home/sshrestha/.ivy2/jars/org.apache.httpcomponents_httpcore-4.3.2.jar,/home/sshrestha/.ivy2/jars/commons-codec_commons-codec-1.10.jar,/home/sshrestha/.ivy2/jars/com.fasterxml.jackson.core_jackson-annotations-2.4.4.jar,/home/sshrestha/.ivy2/jars/com.fasterxml.jackson.core_jackson-core-2.4.4.jar | |
| spark.files -> file:/home/sshrestha/KinesisMA/example/demo_kinesis_streaming.py,file:/home/sshrestha/.ivy2/jars/org.apache.spark_spark-streaming-kinesis-asl_2.10-1.6.2.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_amazon-kinesis-client-1.4.0.jar,file:/home/sshrestha/.ivy2/jars/org.spark-project.spark_unused-1.0.0.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.google.protobuf_protobuf-java-2.5.0.jar,file:/home/sshrestha/.ivy2/jars/commons-lang_commons-lang-2.6.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-support-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-simpledb-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-simpleworkflow-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-storagegateway-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-route53-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-s3-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-importexport-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-sts-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-sqs-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-rds-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-redshift-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elasticbeanstalk-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-glacier-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-iam-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-datapipeline-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elasticloadbalancing-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-emr-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elasticache-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elastictranscoder-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ec2-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-dynamodb-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-sns-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudtrail-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudwatch-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-logs-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cognitoidentity-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cognitosync-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-directconnect-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudformation-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudfront-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-kinesis-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-opsworks-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ses-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-autoscaling-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudsearch-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudwatchmetrics-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-swf-libraries-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-codedeploy-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-kms-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-config-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-lambda-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ecs-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudhsm-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ssm-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-workspaces-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-machinelearning-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-directory-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-core-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/commons-logging_commons-logging-1.1.3.jar,file:/home/sshrestha/.ivy2/jars/org.apache.httpcomponents_httpclient-4.3.2.jar,file:/home/sshrestha/.ivy2/jars/com.fasterxml.jackson.core_jackson-databind-2.4.4.jar,file:/home/sshrestha/.ivy2/jars/joda-time_joda-time-2.9.3.jar,file:/home/sshrestha/.ivy2/jars/org.apache.httpcomponents_httpcore-4.3.2.jar,file:/home/sshrestha/.ivy2/jars/commons-codec_commons-codec-1.10.jar,file:/home/sshrestha/.ivy2/jars/com.fasterxml.jackson.core_jackson-annotations-2.4.4.jar,file:/home/sshrestha/.ivy2/jars/com.fasterxml.jackson.core_jackson-core-2.4.4.jar | |
| spark.app.name -> demo_kinesis_streaming.py | |
| spark.jars -> file:/home/sshrestha/.ivy2/jars/org.apache.spark_spark-streaming-kinesis-asl_2.10-1.6.2.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_amazon-kinesis-client-1.4.0.jar,file:/home/sshrestha/.ivy2/jars/org.spark-project.spark_unused-1.0.0.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.google.protobuf_protobuf-java-2.5.0.jar,file:/home/sshrestha/.ivy2/jars/commons-lang_commons-lang-2.6.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-support-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-simpledb-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-simpleworkflow-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-storagegateway-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-route53-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-s3-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-importexport-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-sts-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-sqs-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-rds-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-redshift-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elasticbeanstalk-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-glacier-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-iam-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-datapipeline-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elasticloadbalancing-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-emr-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elasticache-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elastictranscoder-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ec2-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-dynamodb-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-sns-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudtrail-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudwatch-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-logs-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cognitoidentity-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cognitosync-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-directconnect-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudformation-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudfront-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-kinesis-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-opsworks-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ses-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-autoscaling-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudsearch-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudwatchmetrics-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-swf-libraries-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-codedeploy-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-kms-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-config-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-lambda-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ecs-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudhsm-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ssm-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-workspaces-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-machinelearning-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-directory-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-core-1.9.37.jar,file:/home/sshrestha/.ivy2/jars/commons-logging_commons-logging-1.1.3.jar,file:/home/sshrestha/.ivy2/jars/org.apache.httpcomponents_httpclient-4.3.2.jar,file:/home/sshrestha/.ivy2/jars/com.fasterxml.jackson.core_jackson-databind-2.4.4.jar,file:/home/sshrestha/.ivy2/jars/joda-time_joda-time-2.9.3.jar,file:/home/sshrestha/.ivy2/jars/org.apache.httpcomponents_httpcore-4.3.2.jar,file:/home/sshrestha/.ivy2/jars/commons-codec_commons-codec-1.10.jar,file:/home/sshrestha/.ivy2/jars/com.fasterxml.jackson.core_jackson-annotations-2.4.4.jar,file:/home/sshrestha/.ivy2/jars/com.fasterxml.jackson.core_jackson-core-2.4.4.jar | |
| spark.submit.deployMode -> client | |
| spark.master -> local[*] | |
| Classpath elements: | |
| /home/sshrestha/.ivy2/jars/org.apache.spark_spark-streaming-kinesis-asl_2.10-1.6.2.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_amazon-kinesis-client-1.4.0.jar | |
| /home/sshrestha/.ivy2/jars/org.spark-project.spark_unused-1.0.0.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.google.protobuf_protobuf-java-2.5.0.jar | |
| /home/sshrestha/.ivy2/jars/commons-lang_commons-lang-2.6.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-support-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-simpledb-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-simpleworkflow-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-storagegateway-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-route53-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-s3-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-importexport-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-sts-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-sqs-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-rds-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-redshift-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elasticbeanstalk-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-glacier-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-iam-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-datapipeline-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elasticloadbalancing-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-emr-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elasticache-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-elastictranscoder-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ec2-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-dynamodb-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-sns-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudtrail-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudwatch-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-logs-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cognitoidentity-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cognitosync-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-directconnect-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudformation-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudfront-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-kinesis-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-opsworks-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ses-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-autoscaling-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudsearch-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudwatchmetrics-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-swf-libraries-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-codedeploy-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-kms-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-config-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-lambda-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ecs-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-cloudhsm-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-ssm-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-workspaces-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-machinelearning-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-directory-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/com.amazonaws_aws-java-sdk-core-1.9.37.jar | |
| /home/sshrestha/.ivy2/jars/commons-logging_commons-logging-1.1.3.jar | |
| /home/sshrestha/.ivy2/jars/org.apache.httpcomponents_httpclient-4.3.2.jar | |
| /home/sshrestha/.ivy2/jars/com.fasterxml.jackson.core_jackson-databind-2.4.4.jar | |
| /home/sshrestha/.ivy2/jars/joda-time_joda-time-2.9.3.jar | |
| /home/sshrestha/.ivy2/jars/org.apache.httpcomponents_httpcore-4.3.2.jar | |
| /home/sshrestha/.ivy2/jars/commons-codec_commons-codec-1.10.jar | |
| /home/sshrestha/.ivy2/jars/com.fasterxml.jackson.core_jackson-annotations-2.4.4.jar | |
| /home/sshrestha/.ivy2/jars/com.fasterxml.jackson.core_jackson-core-2.4.4.jar | |
| 16/07/07 03:40:25 INFO akka.event.slf4j.Slf4jLogger: Slf4jLogger started | |
| 16/07/07 03:40:25 INFO Remoting: Starting remoting | |
| 16/07/07 03:40:25 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriverActorSystem@10.240.0.8:45165] | |
| 16/07/07 03:40:26 INFO org.spark-project.jetty.server.Server: jetty-8.y.z-SNAPSHOT | |
| 16/07/07 03:40:26 INFO org.spark-project.jetty.server.AbstractConnector: Started SelectChannelConnector@0.0.0.0:4040 | |
| 16/07/07 03:40:26 INFO org.spark-project.jetty.server.Server: jetty-8.y.z-SNAPSHOT | |
| 16/07/07 03:40:26 INFO org.spark-project.jetty.server.AbstractConnector: Started SocketConnector@0.0.0.0:40027 | |
| 16/07/07 03:40:29 INFO com.amazonaws.services.kinesis.leases.impl.LeaseCoordinator: With failover time 10000ms and epsilon 25ms, LeaseCoordinator will renew leases every 3308ms and take leases every 20050ms | |
| 16/07/07 03:40:29 INFO com.amazonaws.AmazonWebServiceClient: {kinesis, ap-southeast-2} was not found in region metadata, trying to construct an endpoint using the standard pattern for this region: 'kinesis.ap-southeast-2.amazonaws.com'. | |
| 16/07/07 03:40:29 WARN com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker: Received configuration for both region name as ap-southeast-2, and Amazon Kinesis endpoint as https://kinesis.ap-southeast-2.amazonaws.com. Amazon Kinesis endpoint will overwrite region name. | |
| 16/07/07 03:40:29 INFO com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker: Initialization attempt 1 | |
| 16/07/07 03:40:29 INFO com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker: Initializing LeaseCoordinator | |
| 16/07/07 03:40:30 INFO com.amazonaws.services.kinesis.leases.impl.LeaseManager: Table test-kcl-m already exists. | |
| 16/07/07 03:40:31 INFO com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker: Syncing Kinesis shard info | |
| 16/07/07 03:40:32 INFO com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker: Starting LeaseCoordinator | |
| 16/07/07 03:40:42 INFO com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker: Initialization complete. Starting worker loop. | |
| 16/07/07 03:40:43 INFO com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable: Successfully published 16 datums. | |
| 16/07/07 03:40:52 INFO com.amazonaws.services.kinesis.leases.impl.LeaseTaker: Worker localhost:dd106430-b910-4e20-b239-61992bf5abc1 saw 1 total leases, 1 available leases, 1 workers. Target is 1 leases, I have 0 leases, I will take 1 leases | |
| 16/07/07 03:40:52 INFO com.amazonaws.services.kinesis.leases.impl.LeaseTaker: Worker localhost:dd106430-b910-4e20-b239-61992bf5abc1 successfully took 1 leases: shardId-000000000000 | |
| 16/07/07 03:40:53 INFO com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable: Successfully published 16 datums. | |
| 16/07/07 03:40:53 INFO com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker: Created new shardConsumer for : ShardInfo [shardId=shardId-000000000000, concurrencyToken=cd21a94c-aa99-4ec1-8227-af839566c959, parentShardIds=[]] | |
| 16/07/07 03:40:53 INFO com.amazonaws.services.kinesis.clientlibrary.lib.worker.BlockOnParentShardTask: No need to block on parents [] of shard shardId-000000000000 | |
| 16/07/07 03:40:54 INFO com.amazonaws.services.kinesis.clientlibrary.lib.worker.KinesisDataFetcher: Initializing shard shardId-000000000000 with 49559401579349950919709182430103521419284180247706075138 | |
| ========= 2016-07-07 03:41:00 ========= | |
| Empty RDD... Returning | |
| 16/07/07 03:41:03 INFO com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable: Successfully published 12 datums. | |
| 16/07/07 03:41:13 INFO com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable: Successfully published 11 datums. | |
| 16/07/07 03:41:23 INFO com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable: Successfully published 6 datums. | |
| 16/07/07 03:41:30 INFO com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker: Current stream shard assignments: shardId-000000000000 | |
| 16/07/07 03:41:30 INFO com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker: Sleeping ... | |
| 16/07/07 03:41:33 INFO com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable: Successfully published 11 datums. | |
| 16/07/07 03:41:44 INFO com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable: Successfully published 6 datums. | |
| 16/07/07 03:41:54 INFO com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable: Successfully published 11 datums. | |
| ========= 2016-07-07 03:42:00 ========= | |
| Empty RDD... Returning | |
| 16/07/07 03:42:04 INFO com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable: Successfully published 6 datums. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment