This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
['Pink Locust', 'Olive Aphid', 'Black Basilisk', 'Amethyst Silverfish', 'Aqua Butterfly', 'Aquamarine Sparrow', 'Copper Asp', 'Turquoise Kangaroo', 'Silver Hookworm', 'Gold Toad', 'Tomato Parrot', 'Olive Sheep', 'Aqua Roundworm', 'Emerald Salamander', 'Blush Grouse', 'Yellow Cow', 'Brown Unicorn', 'Tan Amphibian', 'Cyan Eagle', 'Jade Emu', 'Amethyst Python', 'Purple Newt', 'Crimson Wildebeest', 'Fuchsia Xerinae', 'Rose Llama', 'Turquoise Crow', 'Blush Puma', 'Fuchsia Hyena', 'Silver Cuckoo', 'Green Rabbit', 'Red Trout', 'Purple Catfish', 'Harlequin Seahorse', 'Coffee Tapir', 'Yellow Toucan', 'Amaranth Kingfisher', 'Silver Rattlesnake', 'Gold Unicorn', 'Pink Orca', 'Harlequin Rooster', 'Azure Kite', 'Copper Silverfish', 'Green Harrier', 'Bronze Hornet', 'Aquamarine Gibbon', 'Chocolate Whale', 'Apricot Minnow', 'Crimson Mackerel', 'Aqua Whitefish', 'Yellow Goat', 'Teal Tortoise', 'Blush Rabbit', 'Peach Lark', 'Plum Planarian', 'Plum Caterpillar', 'Blush Heron', 'Red Orangutan', 'Sapphire Fish', 'Yellow Sailfish |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
schema = StructType([ | |
StructField("RecordNumber",IntegerType(),True), | |
StructField("Zipcode",IntegerType(),True), | |
StructField("ZipCodeType",StringType(),True), | |
StructField("City",StringType(),True), | |
StructField("State",StringType(),True), | |
StructField("LocationType",StringType(),True), | |
StructField("Lat",DoubleType(),True), | |
StructField("Long",DoubleType(),True), | |
StructField("Xaxis",IntegerType(),True), |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Spark dependencies | |
from pyspark.sql import SparkSession | |
from pyspark.sql.functions import * | |
from pyspark.context import SparkContext | |
# Glue dependecies | |
from awsglue.dynamicframe import DynamicFrame | |
from awsglue.dynamicframe import DynamicFrame | |
from awsglue.context import GlueContext | |
from pyspark.sql.types import * |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import pyspark | |
from pyspark.sql import SparkSession | |
from pyspark.sql.functions import * | |
spark = SparkSession.builder.master("local[1]").appName("AWSpark").getOrCreate() | |
df = spark.read.option("header","true").csv("s3a://aws-stocks-dataset/AAL.csv") | |
df.show() | |
df = df.withColumn("date_dt", to_date(col("Date"),"dd-MM-yyyy")) | |
df = df.withColumn("high_cents", col("High") * 100) | |
df = df.orderBy(col("High").desc()).limit(10) | |
df = df.withColumnRenamed("Adjusted Close", "Adjusted_close") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Parameter | Description | Type | | |
|----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------| | |
| spark.hadoop.fs.s3a.access.key | The access key of your user. | textarea | | |
| spark.hadoop.fs.s3a.secret.key | The secret key of your user. | textarea | | |
| spark.hadoop.fs.s3a.endpoint |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aws cloudformation update-stack --stack-name ${StackName} --template-body file://./ec2.yaml --parameters ParameterKey=KeyName,ParamaterValue=testkeypair > cfn.err 2>&1 | |
update_output=$(cat cfn.err) | |
if [[ $update_output =~ ValidationError && $update_output =~ "No updates" ]]; then | |
echo "No updates to perform" | |
exit 0 | |
fi | |
aws cloudformation wait stack-update-complete --stack-name ${StackName} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Replication controller: | |
================================= | |
apiVersion: v1 | |
kind: ReplicationController | |
metadata: | |
labels: | |
name: nginx | |
name: nginx | |
spec: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env bash | |
TAG="kafka-brokers-demo" | |
sudo yum install -y jq | |
sudo yum install -y wget | |
sudo yum install -y java | |
cd ~ | |
wget "http://archive.apache.org/dist/kafka/2.1.0/kafka_2.11-2.1.0.tgz" | |
tar -xvf kafka_2.11-2.1.0.tgz |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class ZeroValue extends Error {} | |
class OtherValue extends Error {} | |
function resolveValue(val) { | |
return Promise.resolve().then(() => { | |
if (!val) throw new ZeroValue(); | |
throw new OtherValue(val); | |
}); | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
factorial 0 = 1 | |
factorial n = n * factorial (n — 1) |
NewerOlder