This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import boto3 | |
import io | |
import zipfile | |
def unzip_s3_source_data( | |
s3_source_bucket: str, | |
s3_source_folder: str, | |
s3_destination_folder: str | |
) -> None: | |
""" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import pandas as pd | |
def set_pd_display(): | |
pd.set_option('display.max_rows', None) | |
pd.set_option('display.max_columns', None) | |
pd.set_option('display.width', None) | |
pd.set_option('display.max_colwidth', None) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
select city_name, current_date, get_weather_udf(city_name) | |
from cities; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CREATE OR REPLACE EXTERNAL FUNCTION get_weather_udf (city varchar) | |
RETURNS varchar STABLE | |
LAMBDA 'my-function' | |
IAM_ROLE '<Role ARN>'; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"Version": "2012-10-17", | |
"Statement": [ | |
{ | |
"Sid": "Invoke", | |
"Effect": "Allow", | |
"Action": [ | |
"lambda:InvokeFunction" | |
], | |
"Resource": "arn:aws:lambda:us-west-2:123456789012:function:my-function" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import http.client | |
from geopy.geocoders import Nominatim | |
import json | |
def geo_code(city): | |
geo_locator = Nominatim(user_agent="medium_article_example") | |
location = geo_locator.geocode(city) | |
print(location.latitude, location.longitude) | |
return (location.latitude, location.longitude) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
create table cities (city_name varchar); | |
insert into dev.public.cities | |
(city_name) | |
values ('Cape Town'),('Pretoria'),('Johannesburg'),('Bloemfontein'); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sudo apt-get install apache2-utils | |
htpasswd -B -C 10 password.db newusername | |
# To verify that the password was created correctly, you can run: | |
htpasswd -v password.db newusername |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
password-authenticator.name=file | |
file.password-file=/etc/presto/password.db | |
file-refresh-period=1m | |
file.auth-token-cache.max-size=1000 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
http-server.authentication.type=PASSWORD | |
web-ui.authentication.type=form |
NewerOlder