Mahout SHDP Tool Config

job output

17:33:32,579  INFO main context.TestContextManager - @TestExecutionListeners is not present for class [class org.springframework.data.hadoop.MahoutTests]: using defaults.
17:33:32,710  INFO main xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [org/springframework/data/hadoop/MahoutTests-context.xml]
17:33:32,917  INFO main xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [org/springframework/data/hadoop/hadoop-ctx.xml]
17:33:33,074  INFO main support.GenericApplicationContext - Refreshing org.springframework.context.support.GenericApplicationContext@59556d12: startup date [Mon Sep 24 17:33:33 EEST 2012]; root of context hierarchy
17:33:33,196  INFO main config.PropertyPlaceholderConfigurer - Loading properties file from class path resource [test.properties]
17:33:33,234  INFO main support.DefaultListableBeanFactory - Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@7a2431b9: defining beans [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,hadoopFs,org.springframework.data.hadoop.mapreduce.MapReducePropertyEditorRegistrar.ns.registration,hadoopResourceLoader,hadoopConfiguration,org.springframework.data.hadoop.scripting.HdfsScriptFactoryBean#0,mahout-tool,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor]; root of factory hierarchy
17:33:33,279  WARN main conf.Configuration - DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
17:33:33,688  WARN main util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17:33:33,688  WARN main snappy.LoadSnappy - Snappy native library not loaded
Resource type class org.springframework.core.io.UrlResource file:/libs/hadoop/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jar 30037387
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/libs/hadoop/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/costin/.gradle/caches/artifacts-14/filestore/org.slf4j/slf4j-log4j12/1.6.6/jar/5cd9b4fbc3ff6a97beaade3206137d76f65df805/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
log4j:ERROR A "org.apache.log4j.ConsoleAppender" object is not assignable to a "org.apache.log4j.Appender" variable.
log4j:ERROR The class "org.apache.log4j.Appender" was loaded by 
log4j:ERROR [ParentLastURLCL
URLs: [file:/libs/hadoop/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jar]
Parent CL: sun.misc.Launcher$AppClassLoader@63c78e57
System CL: sun.misc.Launcher$ExtClassLoader@425224ee
] whereas object of type 
log4j:ERROR "org.apache.log4j.ConsoleAppender" was loaded by [sun.misc.Launcher$AppClassLoader@63c78e57].
log4j:ERROR Could not instantiate appender named "stdout".
17:33:35,091  INFO main mapreduce.ToolRunner - Invoking [org.apache.mahout.cf.taste.hadoop.item.RecommenderJob@252a78ee] from jar [file:/libs/hadoop/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jar] with args [[-s SIMILARITY_LOGLIKELIHOOD, -i/tmp/mah/intro.csv, -o recommendations]]
log4j:WARN No appenders could be found for logger (org.apache.mahout.common.AbstractJob).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
17:33:35,634  INFO main input.FileInputFormat - Total input paths to process : 1
17:33:36,079  INFO main mapred.JobClient - Running job: job_local_0001
17:33:36,147  INFO Thread-10 mapred.Task -  Using ResourceCalculatorPlugin : null
17:33:36,176  INFO Thread-10 mapred.MapTask - io.sort.mb = 100
17:33:36,221  INFO Thread-10 mapred.MapTask - data buffer = 79691776/99614720
17:33:36,221  INFO Thread-10 mapred.MapTask - record buffer = 262144/327680
17:33:36,254  WARN Thread-10 mapred.LocalJobRunner - job_local_0001
java.lang.ArrayIndexOutOfBoundsException: 1
	at org.apache.mahout.cf.taste.hadoop.item.ItemIDIndexMapper.map(ItemIDIndexMapper.java:47)
	at org.apache.mahout.cf.taste.hadoop.item.ItemIDIndexMapper.map(ItemIDIndexMapper.java:31)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
17:33:37,083  INFO main mapred.JobClient -  map 0% reduce 0%
17:33:37,086  INFO main mapred.JobClient - Job complete: job_local_0001
17:33:37,086  INFO main mapred.JobClient - Counters: 0
17:33:37,189  WARN main mapreduce.ExecutionUtils - Trying to patch leaked cl [ParentLastURLCL
URLs: [file:/libs/hadoop/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jar]
Parent CL: sun.misc.Launcher$AppClassLoader@63c78e57
System CL: sun.misc.Launcher$ExtClassLoader@425224ee
] in thread [Thread[communication thread,5,main]]
17:33:37,189  WARN main mapreduce.ExecutionUtils - Trying to patch leaked cl [ParentLastURLCL
URLs: [file:/libs/hadoop/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jar]
Parent CL: sun.misc.Launcher$AppClassLoader@63c78e57
System CL: sun.misc.Launcher$ExtClassLoader@425224ee
] in thread [Thread[SpillThread,5,main]]
17:33:37,200  INFO Thread-3 support.GenericApplicationContext - Closing org.springframework.context.support.GenericApplicationContext@59556d12: startup date [Mon Sep 24 17:33:33 EEST 2012]; root of context hierarchy
17:33:37,201  INFO Thread-3 support.DefaultListableBeanFactory - Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@7a2431b9: defining beans [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,hadoopFs,org.springframework.data.hadoop.mapreduce.MapReducePropertyEditorRegistrar.ns.registration,hadoopResourceLoader,hadoopConfiguration,org.springframework.data.hadoop.scripting.HdfsScriptFactoryBean#0,mahout-tool,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor]; root of factory hierarchy

MahoutTests-context.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:hdp="http://www.springframework.org/schema/hadoop"
	xmlns:batch="http://www.springframework.org/schema/batch"
	xmlns:util="http://www.springframework.org/schema/util"
	xmlns:p="http://www.springframework.org/schema/p"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
		http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd
		http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd">

	<import resource="./hadoop-ctx.xml"/>

	<hdp:script run-at-startup="true" language="javascript">
		if (fsh.test("temp"))
			fsh.rmr("temp")
		
		if (fsh.test("tmp/mah/intro.csv"))
			fsh.put("/libs/hadoop/mahout/mahout-distribution-0.7/intro.csv", "/tmp/mah/intro.csv"
        </hdp:script>
	 
	<hdp:tool-runner id="mahout-tool"
		tool-class="org.apache.mahout.cf.taste.hadoop.item.RecommenderJob"
		jar="/libs/hadoop/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jar">
		<hdp:arg value="-s SIMILARITY_LOGLIKELIHOOD" />
		<hdp:arg value="-i/tmp/mah/intro.csv" />
		<hdp:arg value="-o recommendations" />
	</hdp:tool-runner>

</beans>