Skip to content

Instantly share code, notes, and snippets.

@danbri
Created February 11, 2012 20:32
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save danbri/1804016 to your computer and use it in GitHub Desktop.
bash-3.2$ mahout spectralkmeans -i wiki/ -o output1 -k 20 -d 4192499 --maxIter 10
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
Running on hadoop, using HADOOP_HOME=/Users/danbri/working/hadoop/hadoop-0.20.203.0/
HADOOP_CONF_DIR=/Users/danbri/working/hadoop/hadoop-0.20.203.0/conf
MAHOUT-JOB: /Users/danbri/working/mahout/trunk/examples/target/mahout-examples-0.7-SNAPSHOT-job.jar
12/02/11 21:29:33 WARN driver.MahoutDriver: No spectralkmeans.props found on classpath, will use command-line arguments only
12/02/11 21:29:33 INFO common.AbstractJob: Command line arguments: {--clusters=20, --convergenceDelta=0.5, --dimensions=4192499, --distanceMeasure=org.apache.mahout.common.distance.SquaredEuclideanDistanceMeasure, --endPhase=2147483647, --input=wiki/, --maxIter=10, --output=output1, --startPhase=0, --tempDir=temp}
12/02/11 21:29:34 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
12/02/11 21:29:34 INFO input.FileInputFormat: Total input paths to process : 0
12/02/11 21:29:35 INFO mapred.JobClient: Running job: job_201202111723_0014
12/02/11 21:29:36 INFO mapred.JobClient: map 0% reduce 0%
12/02/11 21:30:01 INFO mapred.JobClient: map 0% reduce 100%
12/02/11 21:30:06 INFO mapred.JobClient: Job complete: job_201202111723_0014
12/02/11 21:30:06 INFO mapred.JobClient: Counters: 16
12/02/11 21:30:06 INFO mapred.JobClient: Job Counters
12/02/11 21:30:06 INFO mapred.JobClient: Launched reduce tasks=1
12/02/11 21:30:06 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=9779
12/02/11 21:30:06 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
12/02/11 21:30:06 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
12/02/11 21:30:06 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=4141
12/02/11 21:30:06 INFO mapred.JobClient: File Output Format Counters
12/02/11 21:30:06 INFO mapred.JobClient: Bytes Written=97
12/02/11 21:30:06 INFO mapred.JobClient: FileSystemCounters
12/02/11 21:30:06 INFO mapred.JobClient: FILE_BYTES_WRITTEN=22062
12/02/11 21:30:06 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=97
12/02/11 21:30:06 INFO mapred.JobClient: Map-Reduce Framework
12/02/11 21:30:06 INFO mapred.JobClient: Reduce input groups=0
12/02/11 21:30:06 INFO mapred.JobClient: Combine output records=0
12/02/11 21:30:06 INFO mapred.JobClient: Reduce shuffle bytes=0
12/02/11 21:30:06 INFO mapred.JobClient: Reduce output records=0
12/02/11 21:30:06 INFO mapred.JobClient: Spilled Records=0
12/02/11 21:30:06 INFO mapred.JobClient: Total committed heap usage (bytes)=85000192
12/02/11 21:30:06 INFO mapred.JobClient: Combine input records=0
12/02/11 21:30:06 INFO mapred.JobClient: Reduce input records=0
12/02/11 21:30:06 INFO common.HadoopUtil: Deleting output1/calculations/diagonal
12/02/11 21:30:06 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
12/02/11 21:30:08 INFO input.FileInputFormat: Total input paths to process : 1
12/02/11 21:30:09 INFO mapred.JobClient: Running job: job_201202111723_0015
12/02/11 21:30:10 INFO mapred.JobClient: map 0% reduce 0%
12/02/11 21:30:30 INFO mapred.JobClient: map 100% reduce 0%
12/02/11 21:30:43 INFO mapred.JobClient: map 100% reduce 100%
12/02/11 21:30:48 INFO mapred.JobClient: Job complete: job_201202111723_0015
12/02/11 21:30:48 INFO mapred.JobClient: Counters: 26
12/02/11 21:30:48 INFO mapred.JobClient: Job Counters
12/02/11 21:30:48 INFO mapred.JobClient: Launched reduce tasks=1
12/02/11 21:30:48 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=13014
12/02/11 21:30:48 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
12/02/11 21:30:48 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
12/02/11 21:30:48 INFO mapred.JobClient: Launched map tasks=1
12/02/11 21:30:48 INFO mapred.JobClient: Data-local map tasks=1
12/02/11 21:30:48 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=10527
12/02/11 21:30:48 INFO mapred.JobClient: File Output Format Counters
12/02/11 21:30:48 INFO mapred.JobClient: Bytes Written=98
12/02/11 21:30:48 INFO mapred.JobClient: FileSystemCounters
12/02/11 21:30:48 INFO mapred.JobClient: FILE_BYTES_READ=6
12/02/11 21:30:48 INFO mapred.JobClient: HDFS_BYTES_READ=241
12/02/11 21:30:48 INFO mapred.JobClient: FILE_BYTES_WRITTEN=44459
12/02/11 21:30:48 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=98
12/02/11 21:30:48 INFO mapred.JobClient: File Input Format Counters
12/02/11 21:30:48 INFO mapred.JobClient: Bytes Read=97
12/02/11 21:30:48 INFO mapred.JobClient: Map-Reduce Framework
12/02/11 21:30:48 INFO mapred.JobClient: Map output materialized bytes=6
12/02/11 21:30:48 INFO mapred.JobClient: Map input records=0
12/02/11 21:30:48 INFO mapred.JobClient: Reduce shuffle bytes=0
12/02/11 21:30:48 INFO mapred.JobClient: Spilled Records=0
12/02/11 21:30:48 INFO mapred.JobClient: Map output bytes=0
12/02/11 21:30:48 INFO mapred.JobClient: Total committed heap usage (bytes)=269619200
12/02/11 21:30:48 INFO mapred.JobClient: Combine input records=0
12/02/11 21:30:48 INFO mapred.JobClient: SPLIT_RAW_BYTES=144
12/02/11 21:30:48 INFO mapred.JobClient: Reduce input records=0
12/02/11 21:30:48 INFO mapred.JobClient: Reduce input groups=0
12/02/11 21:30:48 INFO mapred.JobClient: Combine output records=0
12/02/11 21:30:48 INFO mapred.JobClient: Reduce output records=0
12/02/11 21:30:48 INFO mapred.JobClient: Map output records=0
12/02/11 21:30:48 INFO common.VectorCache: Loading vector from: output1/calculations/diagonal/part-r-00000
Exception in thread "main" java.util.NoSuchElementException
at com.google.common.collect.AbstractIterator.next(AbstractIterator.java:152)
at org.apache.mahout.clustering.spectral.common.VectorCache.load(VectorCache.java:115)
at org.apache.mahout.clustering.spectral.common.MatrixDiagonalizeJob.runJob(MatrixDiagonalizeJob.java:78)
at org.apache.mahout.clustering.spectral.kmeans.SpectralKMeansDriver.run(SpectralKMeansDriver.java:133)
at org.apache.mahout.clustering.spectral.kmeans.SpectralKMeansDriver.run(SpectralKMeansDriver.java:85)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.mahout.clustering.spectral.kmeans.SpectralKMeansDriver.main(SpectralKMeansDriver.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment