Skip to content

Instantly share code, notes, and snippets.

@wjrmffldrhrl
wjrmffldrhrl / aws4_hmac_sha256.py
Created February 8, 2023 16:14
AWS4-HMAC-SHA256 request
import urllib.parse
import hashlib
import hmac
from datetime import datetime
def sign_request(
method: str,
headers: dict,
path: str,
@wjrmffldrhrl
wjrmffldrhrl / kubernetes_pod_operator.py
Last active August 15, 2022 10:47
Airflow KubernetesPodOperator
import time
from datetime import datetime
from airflow.decorators import dag
from airflow.providers.cncf.kubernetes.operators.kubernetes_pod import (
KubernetesPodOperator,
)
doc = """
@wjrmffldrhrl
wjrmffldrhrl / superset_config_for_celery_worker.py
Last active May 24, 2022 08:17
superset_config_for_celery_worker.py
class CeleryConfig(object):
broker_url = "redis://localhost:6379/0"
imports = (
"superset.sql_lab",
"superset.tasks",
)
result_backend = "redis://localhost:6379/0"
worker_log_level = "DEBUG"
worker_prefetch_multiplier = 10
task_acks_late = True
from flask_appbuilder.security.manager import (AUTH_DB, AUTH_OAUTH)
AUTH_TYPE = AUTH_OAUTH
OAUTH_PROVIDERS = [
{
"name": "google",
"whitelist": [ "@voithru.com" ],
"icon": "fa-google",
"token_key": "access_token",
from flask_appbuilder.security.manager import (AUTH_DB, AUTH_OAUTH)
AUTH_TYPE = AUTH_OAUTH
OAUTH_PROVIDERS = [
{
"name": "google",
"whitelist": [ "@voithru.com" ],
"icon": "fa-google",
"token_key": "access_token",
## Configuration values for the postgresql dependency.
## ref: https://github.com/kubernetes/charts/blob/master/stable/postgresql/README.md
postgresql:
##
## Use the PostgreSQL chart dependency.
## Set to false if bringing your own PostgreSQL.
enabled: false
##
## The name of an existing secret that contains the postgres password.
existingSecret: superset-postgresql
@wjrmffldrhrl
wjrmffldrhrl / StreamingAnalytics2.md
Last active March 30, 2022 13:42
Streaming Analytics on Flink 2.md

Window

플링크는 매우 풍부한 윈도우 시멘틱을 제공합니다.

Introduction

스트림 처리를 수행할 때 아래와 같은 질문에 답하기 위해 스트림의 제한된 하위 집합에 대해 집계된 분석을 계산하려는 것은 자연스러운 일입니다.

It is natural when doing stream processing to want to compute aggregated analytics on bounded subsets of the streams in order to answer questions like these:

  • 분당 페이지 뷰 수
  • 주당 사용자 별 세션 수
  • 분당 센서별 최대 온도
@wjrmffldrhrl
wjrmffldrhrl / StreamingAnalytics1.md
Last active March 27, 2022 04:53
Streaming Analytics on Flink 1

Event Time and Wartermarks

Introduction

플링크는 시간에 관한 명확한 세 가지 개념을 지원합니다.

  • Event Time
    • 이벤트가 발생한 시간, 기기에 의해 생산(혹은 저장)되어 기록된 이벤트
  • Ingetion Time
    • 이벤트가 전달된 순간 플링크에 의해 기록된 시간
  • Processing Time
  • 파이프라인의 특정한 연산자에 의해 이벤트가 처리된 시간
@wjrmffldrhrl
wjrmffldrhrl / NaverStockDiscussionCrawler.scala
Created February 23, 2021 03:50
NaverStockDiscussionCrawler
package crawler
import crawler.manager.file.FileManager
import net.ruippeixotog.scalascraper.browser.JsoupBrowser
import net.ruippeixotog.scalascraper.dsl.DSL._
import net.ruippeixotog.scalascraper.dsl.DSL.Extract._
import java.io.{BufferedWriter, File, FileNotFoundException, FileReader, FileWriter}
import java.time.LocalDateTime
import scala.io.Source