Skip to content

Instantly share code, notes, and snippets.

@solidpple
Created July 26, 2016 00:09
val links = sc.objectFile[(String, Seq[String])]("links").partitionBy(new HashPartitioner(100)).persist()
val ranks = links.mapValues(v => 1.0)
for (i <- 0 until 10) {
val contributions = links.join(ranks).flatMap {
case (pageId, (link, rank)) =>
links.map(dest => (dest, rank / links.size))
}
rank = contributions.reduceByKey((x, y) => x + y).mapValues(v => 0.15 + 0.85*v)
}
ranks.saveAsTextFile("ranks")
@victron777
Copy link

Hi, I would appreciate if anyone provide me the ObjectFile that this script is running. Thanks in advance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment