Skip to content

Instantly share code, notes, and snippets.

View audhiaprilliant's full-sized avatar
🎯
Focusing

Audhi Aprilliant audhiaprilliant

🎯
Focusing
View GitHub Profile
@audhiaprilliant
audhiaprilliant / README.md
Last active April 23, 2021 02:45
GitHub README File

Hi there, I'm [Audhi Aprilliant][website] 👋

I'm a Man, Friend, and Researcher!

  • 🔭 I’m currently working on a [personal project][website]!
  • 🌱 I’m currently learning Data Science and Analytics 🤣
  • 👯 I’m looking to collaborate with other content creators
  • 🥅 2020 Goals: Give impact to society
  • ⚡ Fun fact: I love blogging, reading books, and drawing

Connect with me:

name: Latest Medium story workflow
on:
schedule:
# Runs every hour
- cron: '0 * * * *'
workflow_dispatch:
jobs:
update-readme-with-blog:
name: Update this repo's README with latest Medium post
@audhiaprilliant
audhiaprilliant / autocorrelation_simulation.R
Last active December 19, 2020 02:54
Simulation of Autocorrelation in Linear Regression Model
# Function to conduct simulation
generate_beta = function(ti, rho, repetition) {
# Vector intercept and slope
vector_intercept_auto = rep(x = 0, len = repetition)
vector_slope_auto = rep(x = 0, len = repetition)
vector_intercept_non = rep(x = 0, len = repetition)
vector_slope_non = rep(x = 0, len = repetition)
j = 1
while (j < (repetition + 1)) {
@audhiaprilliant
audhiaprilliant / autocorrelation_simulation_preprocessing.R
Last active December 18, 2020 17:11
Simulation of Autocorrelation in Linear Regression Model
# Install package
install.packages('tidyr')
# Load the package
library(tidyr)
# Data melting
long = df %>%
gather(Coefficient, Value, -c(Rho))
# Create a new column indicating whether it is intercept or slope
long$Status = ifelse(substring(long$Coefficient, 1, 9) == 'intercept', 'Intercept', 'Slope')
@audhiaprilliant
audhiaprilliant / autocorrelation_simulation_statistics.R
Last active December 18, 2020 17:10
Simulation of Autocorrelation in Linear Regression Model
# Install package
install.packages('dplyr')
# Load the package
library(dplyr)
# Create a data aggregation
statDf = df %>%
group_by(Rho) %>%
summarize(
# Without autocorrelation
@audhiaprilliant
audhiaprilliant / autocorrelation_simulation_visualization.R
Last active December 19, 2020 02:56
Simulation of Autocorrelation in Linear Regression Model
# Install package
install.packages('ggplot2')
# Load the package
library(ggplot2)
# Without autocorrelation - intercept
df_InterceptNonAutocorrelation = long %>%
filter(Coefficient == 'interceptNonAutocorrelation')
unique(df_InterceptNonAutocorrelation$Rho)
ggplot(data = df_InterceptNonAutocorrelation)+
@audhiaprilliant
audhiaprilliant / apache_airflow_covid19_import_modules.py
Last active December 13, 2020 08:48
Apache Airflow as Job Orchestration
# Modules for web scraping
import requests
from bs4 import BeautifulSoup
# Module for data manipulation
import pandas as pd
# Module for regular expression
import re
# Module for file management
import os
# Module for timing
@audhiaprilliant
audhiaprilliant / apache_airflow_covid19_get_url.py
Created December 13, 2020 08:56
Apache Airflow as Job Orchestration
def get_url():
# URL
url = 'https://www.kompas.com/covid-19'
page = requests.get(url)
# Wrangling HTML with BeautifulSoup
soup = BeautifulSoup(page.content,'html.parser')
return(soup)
@audhiaprilliant
audhiaprilliant / apache_airflow_covid19_get_current_date.py
Created December 13, 2020 08:59
Apache Airflow as Job Orchestration
# Create dictionary for date manipulation
dict_month = {'Januari':'01','Februari':'02','Maret':'03','April':'04','Mei':'05','Juni':'06','Juli':'07','Agustus':'08','September':'09','Oktober':'10','November':'11','Desember':'12'}
# Function to get the current date
def get_current_date(**kwargs):
date_scrape = soup.find('span',class_='covid__date').text
date_scrape = re.findall(r'Update terakhir: (\S+.+WIB)',date_scrape)[0].replace(', ',',')
date = date_scrape.split(',')[0]
time = date_scrape.split(',')[1]
# Date manipulation
date_format = re.findall(r'\w+',date)[0]
@audhiaprilliant
audhiaprilliant / apache_airflow_covid19_get_daily_summary.py
Created December 13, 2020 09:00
Apache Airflow as Job Orchestration
# Function to get the daily aggregated data
def get_daily_summary(**kwargs):
soup = get_url()
date,time = get_current_date()
# Get summary
# Regular expression pattern
pattern_summary = re.compile(r'\d[^\s]+')
for job_elem in soup.find_all('div',class_='covid__box'):
# Each job_elem is a new BeautifulSoup object.
terkonfirmasi_elem = job_elem.find('div',class_='covid__box2 -cases')