Skip to content

Instantly share code, notes, and snippets.

@melvinsoft
Created September 14, 2017 15:55
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save melvinsoft/9d4d03b2adf3cc146fce69f678c53f91 to your computer and use it in GitHub Desktop.
Save melvinsoft/9d4d03b2adf3cc146fce69f678c53f91 to your computer and use it in GitHub Desktop.
2017-09-14 13:12:36,055 ERROR 9901 [luigi-interface] worker.py:304 - [pid 9901] Worker Worker(salt=197142346, host=prod-asustarbucks-ficus-insights-0, username=hadoop, pid=9901) failed HiveTableFromParameterQueryTask(warehouse_path=hdfs://localhost:9000/edx-analytics-pipeline/warehouse/, insert_query=
SELECT
act.course_id as course_id,
CONCAT(cal.iso_week_start, ' 00:00:00') as interval_start,
CONCAT(cal.iso_week_end, ' 00:00:00') as interval_end,
act.category as label,
COUNT(DISTINCT username) as count
FROM user_activity_daily act
JOIN calendar cal ON act.`date` = cal.`date`
WHERE "2017-09-04" <= cal.`date` AND cal.`date` < "2017-09-18"
GROUP BY
act.course_id,
cal.iso_week_start,
cal.iso_week_end,
act.category;
, table=course_activity, columns=(('course_id', 'STRING'), ('interval_start', 'TIMESTAMP'), ('interval_end', 'TIMESTAMP'), ('label', 'STRING'), ('count', 'INT')), partition=dt=2017-09-18)
Traceback (most recent call last):
File "/var/lib/analytics-tasks/analyticstack/venv/local/lib/python2.7/site-packages/luigi/worker.py", line 292, in _run_task
task.run()
File "/var/lib/analytics-tasks/analyticstack/venv/local/lib/python2.7/site-packages/luigi/hadoop.py", line 612, in run
self.job_runner().run_job(self)
File "/var/lib/analytics-tasks/analyticstack/venv/local/lib/python2.7/site-packages/edx/analytics/tasks/util/hive.py", line 353, in run_job
return super(OverwriteAwareHiveQueryRunner, self).run_job(job)
File "/var/lib/analytics-tasks/analyticstack/venv/local/lib/python2.7/site-packages/luigi/hive.py", line 359, in run_job
return luigi.hadoop.run_and_track_hadoop_job(arglist)
File "/var/lib/analytics-tasks/analyticstack/venv/local/lib/python2.7/site-packages/luigi/hadoop.py", line 318, in run_and_track_hadoop_job
return track_process(arglist, tracking_url_callback, env)
File "/var/lib/analytics-tasks/analyticstack/venv/local/lib/python2.7/site-packages/luigi/hadoop.py", line 302, in track_process
raise HadoopJobError(message + 'Also, no tracking url found.', out, err)
HadoopJobError: ('Streaming job failed with exit code 230. Also, no tracking url found.', '', "17/09/14 13:12:32 INFO Configuration.deprecation: mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive\n17/09/14 13:12:32 INFO Configuration.deprecation: mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize\n17/09/14 13:12:32 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize\n17/09/14 13:12:32 INFO Configuration.deprecation: mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack\n17/09/14 13:12:32 INFO Configuration.deprecation: mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node\n17/09/14 13:12:32 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces\n17/09/14 13:12:32 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative\n17/09/14 13:12:32 WARN conf.Configuration: org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@71bbf57e:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.\n17/09/14 13:12:32 WARN conf.Configuration: org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@71bbf57e:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.\n17/09/14 13:12:32 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name\n\nLogging initialized using configuration in jar:file:/edx/app/hadoop/hive-0.11.0-bin/lib/hive-common-0.11.0.jar!/hive-log4j.properties\nHive history file=/tmp/hadoop/hive_job_log_hadoop_10474@prod-asustarbucks-ficus-insights-0.c.asu-starbucks.internal_201709141312_1920388014.txt\nSLF4J: Class path contains multiple SLF4J bindings.\nSLF4J: Found binding in [jar:file:/edx/app/hadoop/hadoop-2.3.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]\nSLF4J: Found binding in [jar:file:/edx/app/hadoop/hive-0.11.0-bin/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]\nSLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.\nSLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]\nOK\nTime taken: 1.75 seconds\nOK\nTime taken: 0.924 seconds\nOK\nTime taken: 0.271 seconds\nFAILED: SemanticException [Error 10214]: Invalid partition spec specified dt not found in table's partition spec: {`dt`=2017-09-18}\n")
2017-09-14 13:12:36,056 INFO 9901 [luigi-interface] notifications.py:96 - Skipping error email. Set `error-email` in the `core` section of the luigi config file to receive error emails.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment