This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val osAovCount = data.filter { x => interestedBrowsers.contains(x.osFamily)} | |
.filter (_.categoryId == 128) | |
.map { x => x.osFamily } | |
.countByValue | |
.toSeq | |
draw(column(osAovCount.toList)) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
//CODE of MIC | |
import data.VarPairData | |
import mine.core.MineParameters | |
import analysis.Analysis | |
import analysis.results.BriefResult | |
import scala.util.Random | |
//Code a discrete value by randomly changing the order of the "codes | |
def encode(col: Array[String]): Array[Double] = { |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
//CODE | |
import org.ddahl.rscala._ | |
import ru.retailrocket.ispark._ | |
def connect() = RClient("R", false) | |
@transient | |
val r = connect() | |
R.plot(r, "barplot(cs, names.arg=names, col = 'red' )", Map("cs"->os.Aov.map(_._2), "names" -> osAov.map(_._1)) ) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
//CODE of Highcharts | |
import com.quantifind.charts.Highcharts._ | |
import ru.retailrocket.ispark.wisp._ | |
draw(column(osAov.toList)) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
//CODE: | |
val interestedBrowsers = List("Android", "OS X", "iOS", "Linux", "Windows") | |
val osAov = dataAov.filter(x => interestedBrowsers.contains(x.osFamily)) //we leave only the desired OS | |
.filter(_.categoryId == 128) // filter categories | |
.map(x => (x.osFamily, (x.aov, 1.0))) // need to calculate average purchase amount | |
.reduceByKey((x, y) => (x._1 + y._1, x._2 + y._2)) | |
.map{ case(osFamily, (revenue, orders)) => (osFamily, revenue/orders) } | |
.collect() | |
//OUT |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
//CODE: | |
//The most popular category. | |
dataAov.map { x => x.categoryId } // select the categoryId field | |
.countByValue() // calculate how often each categoryId appears | |
.toSeq | |
.sortBy( - _._2) // sort by frequency in descending order | |
.take(10) //take the top 10 records | |
//OUT: | |
//format: (categoryId, count) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
//CODE: | |
import org.apache.spark.rdd.RDD | |
import org.apache.spark.sql._ | |
import org.tribbloid.ispark.display.dsl._ | |
import scala.util.Try | |
val sqlContext = new org.apache.spark.sql.SQLContext(sc) | |
import sqlContext.implicits._ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
//CODE: | |
import org.apache.spark.rdd.RDD | |
import org.apache.spark.sql._ | |
import org.tribbloid.ispark.display.dsl._ | |
import scala.util.Try | |
val sqlContext = new org.apache.spark.sql.SQLContext(sc) | |
import sqlContext.implicits._ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import org.nd4j.linalg.api.ndarray.INDArray | |
import org.nd4j.linalg.api.ops.impl.broadcast.BroadcastAddOp | |
import org.nd4j.linalg.api.rng.distribution.impl.UniformDistribution | |
import org.nd4j.linalg.factory.Nd4j | |
import scala.util.Random | |
object Xavier { | |
def init(shape: Array[Int]): INDArray = { | |
var n = shape.dropRight(1).product |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package org.deeplearning4j.learn | |
import java.util.Arrays | |
import org.nd4j.api.linalg.DSL._ | |
import org.nd4j.linalg.api.ndarray.INDArray | |
import org.nd4j.linalg.factory.Nd4j | |
import org.nd4j.linalg.indexing.{NDArrayIndex, BooleanIndexing} | |
import org.nd4j.linalg.indexing.conditions.Conditions | |
import org.nd4j.linalg.indexing.functions.Value |
NewerOlder