Skip to content

Instantly share code, notes, and snippets.

@danbri
Created September 7, 2011 12:27
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save danbri/1200428 to your computer and use it in GitHub Desktop.
TellyClub:trunk danbri$ sh spectral.sh
Running on hadoop, using HADOOP_HOME=/Users/danbri/working/hadoop/hadoop-0.20.2
HADOOP_CONF_DIR=/Users/danbri/working/hadoop/hadoop-0.20.2/conf
MAHOUT-JOB: /Users/danbri/working/mahout/trunk/examples/target/mahout-examples-0.6-SNAPSHOT-job.jar
11/09/07 14:22:46 WARN driver.MahoutDriver: No spectralkmeans.props found on classpath, will use command-line arguments only
11/09/07 14:22:46 INFO common.AbstractJob: Command line arguments: {--clusters=2, --convergenceDelta=0.5, --dimensions=37, --distanceMeasure=org.apache.mahout.common.distance.SquaredEuclideanDistanceMeasure, --endPhase=2147483647, --input=speccy, --maxIter=10, --output=specout, --startPhase=0, --tempDir=temp}
11/09/07 14:22:46 INFO common.HadoopUtil: Deleting specout/calculations/seqfile-248
11/09/07 14:22:47 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
11/09/07 14:22:51 INFO input.FileInputFormat: Total input paths to process : 2
11/09/07 14:22:56 INFO mapred.JobClient: Running job: job_201109071300_0008
11/09/07 14:22:57 INFO mapred.JobClient: map 0% reduce 0%
11/09/07 14:24:07 INFO mapred.JobClient: map 100% reduce 0%
11/09/07 14:24:19 INFO mapred.JobClient: map 100% reduce 100%
11/09/07 14:24:21 INFO mapred.JobClient: Job complete: job_201109071300_0008
11/09/07 14:24:22 INFO mapred.JobClient: Counters: 15
11/09/07 14:24:22 INFO mapred.JobClient: Job Counters
11/09/07 14:24:22 INFO mapred.JobClient: Launched reduce tasks=1
11/09/07 14:24:22 INFO mapred.JobClient: Launched map tasks=2
11/09/07 14:24:22 INFO mapred.JobClient: FileSystemCounters
11/09/07 14:24:22 INFO mapred.JobClient: FILE_BYTES_READ=6
11/09/07 14:24:22 INFO mapred.JobClient: FILE_BYTES_WRITTEN=82
11/09/07 14:24:22 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=97
11/09/07 14:24:22 INFO mapred.JobClient: Map-Reduce Framework
11/09/07 14:24:22 INFO mapred.JobClient: Reduce input groups=0
11/09/07 14:24:22 INFO mapred.JobClient: Combine output records=0
11/09/07 14:24:22 INFO mapred.JobClient: Map input records=0
11/09/07 14:24:22 INFO mapred.JobClient: Reduce shuffle bytes=6
11/09/07 14:24:22 INFO mapred.JobClient: Reduce output records=0
11/09/07 14:24:22 INFO mapred.JobClient: Spilled Records=0
11/09/07 14:24:22 INFO mapred.JobClient: Map output bytes=0
11/09/07 14:24:22 INFO mapred.JobClient: Combine input records=0
11/09/07 14:24:22 INFO mapred.JobClient: Map output records=0
11/09/07 14:24:22 INFO mapred.JobClient: Reduce input records=0
11/09/07 14:24:22 INFO common.HadoopUtil: Deleting specout/calculations/diagonal
11/09/07 14:24:23 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
11/09/07 14:24:30 INFO input.FileInputFormat: Total input paths to process : 1
11/09/07 14:24:32 INFO mapred.JobClient: Running job: job_201109071300_0009
11/09/07 14:24:33 INFO mapred.JobClient: map 0% reduce 0%
11/09/07 14:24:52 INFO mapred.JobClient: map 100% reduce 0%
11/09/07 14:25:04 INFO mapred.JobClient: map 100% reduce 100%
11/09/07 14:25:06 INFO mapred.JobClient: Job complete: job_201109071300_0009
11/09/07 14:25:06 INFO mapred.JobClient: Counters: 17
11/09/07 14:25:06 INFO mapred.JobClient: Job Counters
11/09/07 14:25:06 INFO mapred.JobClient: Launched reduce tasks=1
11/09/07 14:25:06 INFO mapred.JobClient: Launched map tasks=1
11/09/07 14:25:06 INFO mapred.JobClient: Data-local map tasks=1
11/09/07 14:25:06 INFO mapred.JobClient: FileSystemCounters
11/09/07 14:25:06 INFO mapred.JobClient: FILE_BYTES_READ=6
11/09/07 14:25:06 INFO mapred.JobClient: HDFS_BYTES_READ=97
11/09/07 14:25:06 INFO mapred.JobClient: FILE_BYTES_WRITTEN=44
11/09/07 14:25:06 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=98
11/09/07 14:25:06 INFO mapred.JobClient: Map-Reduce Framework
11/09/07 14:25:06 INFO mapred.JobClient: Reduce input groups=0
11/09/07 14:25:06 INFO mapred.JobClient: Combine output records=0
11/09/07 14:25:06 INFO mapred.JobClient: Map input records=0
11/09/07 14:25:06 INFO mapred.JobClient: Reduce shuffle bytes=0
11/09/07 14:25:06 INFO mapred.JobClient: Reduce output records=0
11/09/07 14:25:06 INFO mapred.JobClient: Spilled Records=0
11/09/07 14:25:06 INFO mapred.JobClient: Map output bytes=0
11/09/07 14:25:06 INFO mapred.JobClient: Combine input records=0
11/09/07 14:25:06 INFO mapred.JobClient: Map output records=0
11/09/07 14:25:06 INFO mapred.JobClient: Reduce input records=0
11/09/07 14:25:06 INFO common.VectorCache: Loading vector from: specout/calculations/diagonal/part-r-00000
Exception in thread "main" java.util.NoSuchElementException
at com.google.common.collect.AbstractIterator.next(AbstractIterator.java:152)
at org.apache.mahout.clustering.spectral.common.VectorCache.load(VectorCache.java:121)
at org.apache.mahout.clustering.spectral.common.MatrixDiagonalizeJob.runJob(MatrixDiagonalizeJob.java:75)
at org.apache.mahout.clustering.spectral.kmeans.SpectralKMeansDriver.run(SpectralKMeansDriver.java:141)
at org.apache.mahout.clustering.spectral.kmeans.SpectralKMeansDriver.run(SpectralKMeansDriver.java:93)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.mahout.clustering.spectral.kmeans.SpectralKMeansDriver.main(SpectralKMeansDriver.java:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
TellyClub:trunk danbri$
TellyClub:trunk danbri$ mahout seqdumper --seqFile specout/calculations/diagonal/part-r-00000
Running on hadoop, using HADOOP_HOME=/Users/danbri/working/hadoop/hadoop-0.20.2
HADOOP_CONF_DIR=/Users/danbri/working/hadoop/hadoop-0.20.2/conf
MAHOUT-JOB: /Users/danbri/working/mahout/trunk/examples/target/mahout-examples-0.6-SNAPSHOT-job.jar
Input Path: specout/calculations/diagonal/part-r-00000
Key class: class org.apache.hadoop.io.NullWritable Value Class: class org.apache.mahout.math.VectorWritable
Count: 0
11/09/07 14:26:04 INFO driver.MahoutDriver: Program took 340 ms
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment