rickcrawford

Create 2 files: main.go and docker-compose.yml. Once both are created use docker-compose up and it will build the necessary files and start Elastic Search.

I found it necessary to add elastic.SetSniff(false) or I could not connect. Also remember the docker containers have security enabled for Elasticsearch with the password set as elastic:changeme. You can test it using curl:

curl http://127.0.0.1:9200/_cat/health -u elastic:changeme

Once it is running you can successfully run the main.go file using go run main.go.

zouzias

import org.apache.spark.sql.Row
import org.apache.spark.sql.types.{StructField,StructType,IntegerType, LongType}

val  df = sc.parallelize(Seq((1.0, 2.0), (0.0, -1.0), (3.0, 4.0), (6.0, -2.3))).toDF("x", "y")

// Append "rowid" column of type Long
val newSchema = StructType(df.schema.fields ++ Array(StructField("rowid", LongType, false)))

// Zip on RDD level
val rddWithId = df.rdd.zipWithIndex