anirtek

Swagger Setup for Embedded Jetty Server

In setting up a Jetty server with Jersey servlets, you may choose to use an embedded Jetty server setup. (See here for how to setup an embedded Jetty server). In this gist, we'll go through how to setup Swagger for this setup. I am using Swagger 1.5, Maven 3.3.3, Jersey 1.8, and Jetty 7.3. Make sure you add all dependencies to your pom.xml.

In the Swagger Core setup, the current official recommendations involve an Application class, or a web.xml, neither of which are used in an embedded Jetty server setup. To add Swagger to your embedded Jetty Server, you must do 3 things:

1. Add the package scanning packages to your servlets which will serve your REST API.
2. Add the Swagger package scanning servlet.
3. P

anirtek

Custom S3 endpoints with Spark

To be able to use custom endpoints with the latest Spark distribution, one needs to add an external package (hadoop-aws). Then, custum endpoints can be configured according to docs.

Use the hadoop-aws package

bin/spark-shell --packages org.apache.hadoop:hadoop-aws:2.7.2

SparkContext configuration

anirtek

val spark = SparkSession
  .builder()
  .appName("SparkScalaApp")
  .master("local")
  .getOrCreate()

val df = spark.read.csv("/Users/home/Dataset/webLog.csv")

df.createOrReplaceTempView("view")