Skip to content

Instantly share code, notes, and snippets.

View agrawalreetika's full-sized avatar

Reetika Agrawal agrawalreetika

10 followers · 1 following
View GitHub Profile
@agrawalreetika
agrawalreetika / presto_cli_install
Created March 3, 2020 09:35
Install Presto-cli on Presto coordinator
#Install presto-cli
cd /usr/lib/
sudo wget https://repo1.maven.org/maven2/io/prestosql/presto-cli/325/presto-cli-325-executable.jar
sudo mv presto-cli-325-executable.jar /usr/bin/presto
sudo chmod 0755 /usr/bin/presto
@agrawalreetika
agrawalreetika / Query_presto
Last active March 3, 2020 12:46
Steps to query through presto-cli.
[root@presto-coordinator ~]$ presto
presto> create schema hive.test_db;
presto> CREATE TABLE hive.test_db.lineitem (
orderkey bigint NOT NULL,
partkey bigint NOT NULL,
suppkey bigint NOT NULL,
linenumber integer NOT NULL,
quantity double NOT NULL,
@agrawalreetika
agrawalreetika / cloud_init_installation.sh
Last active December 22, 2019 15:32
This script is used for cloud init installation
sudo yum makecache fast
sudo yum install -y gdisk cloud-utils-growpart
sudo yum install -y cloud-init
sudo sed -i 's/Provisioning.Enabled=y/Provisioning.Enabled=n/g' /etc/waagent.conf
sudo sed -i 's/Provisioning.UseCloudInit=n/Provisioning.UseCloudInit=y/g' /etc/waagent.conf
sudo sed -i 's/ResourceDisk.Format=y/ResourceDisk.Format=n/g' /etc/waagent.conf
sudo sed -i 's/ResourceDisk.EnableSwap=y/ResourceDisk.EnableSwap=n/g' /etc/waagent.conf
sudo cloud-init clean
@agrawalreetika
agrawalreetika / cloud-init.txt
Created December 22, 2019 13:26
Cloud-init script
#cloud-config
runcmd:
- sudo UUID=`uuidgen`
- sudo nodeId=`grep node.id= /usr/lib/presto/etc/node.properties`
- sudo sed -i "s|$nodeId|node.id=$UUID|" /usr/lib/presto/etc/node.properties
- /usr/lib/presto/bin/launcher start
@agrawalreetika
agrawalreetika / hive.properties
Created December 8, 2019 18:32
Configuring hive catalog for presto
connector.name=hive-hadoop2
hive.metastore.uri=thrift://<metastore-vm>:9083
hive.config.resources=<location for azure-site.xml>
@agrawalreetika
agrawalreetika / azure-site.xml
Last active February 13, 2020 11:43
This file contains configuration for Azure app that will be required to access azure Gen1 and Gen2 storage
<configuration>
<property>
<name>dfs.adls.oauth2.access.token.provider.type</name>
<value>ClientCredential</value>
</property>
<property>
<name>dfs.adls.oauth2.refresh.url</name>
<value>{Oauth2.0-endpoint}</value>
</property>
@agrawalreetika
agrawalreetika / vmss_create.sh
Last active January 3, 2020 09:22
In this we will create scale set and set auto scaling rules for same.
ImageName="presto-worker-image"
ScalesetName="presto-worker-scaleset"
ScalesetProfileName="presto-worker-scaleset-autoscale-profile"
AdminPassword="PasswordOfYourChoice"
ResourceGroupName="PrestoResourceGroup"
Subscription="Azure-Subscription-Name"
Location="eastus"
MIN_COUNT=2
MAX_COUNT=10
NsgName="presto-nsg"
@agrawalreetika
agrawalreetika / worker_imager_create.sh
Last active February 3, 2020 09:31
This includes steps to create presto worker image.
# De-allocate and generalize the VMs in order to create an image.
# A VM that has been generalized can no longer be restarted.
ResourceGroupName="PrestoResourceGroup"
WorkerName="presto-worker-vm"
ImageName="presto-worker-image"
Subscription="Azure-Subscription-Name"
az vm deallocate \
--subscription $Subscription \
@agrawalreetika
agrawalreetika / worker_setup.sh
Last active January 3, 2020 09:14
This has all the steps to setup presto worker on VM.
# Install wget
sudo yum install -y wget
# Setup Presto
sudo mkdir -m757 -p /var/presto/data
cd /usr/lib/
sudo wget https://repo1.maven.org/maven2/io/prestosql/presto-server/325/presto-server-325.tar.gz
sudo tar xvzf presto-server-325.tar.gz
sudo mv presto-server-325 presto
sudo rm presto-server-325.tar.gz
@agrawalreetika
agrawalreetika / coordinator_setup.sh
Last active March 3, 2020 09:37
This has all the steps to setup presto coordinator on VM.
# Install wget
sudo yum install -y wget
# Setup Presto
sudo mkdir -m757 -p /var/presto/data
cd /usr/lib/
sudo wget https://repo1.maven.org/maven2/io/prestosql/presto-server/325/presto-server-325.tar.gz
sudo tar xvzf presto-server-325.tar.gz
sudo mv presto-server-325 presto
sudo rm presto-server-325.tar.gz