zkan

# Libraries and infrastructure
sudo apt update -y
sudo apt install -y \
	docker.io docker-buildx \
	build-essential pkg-config autoconf bison rustc cargo clang \
	libssl-dev libreadline-dev zlib1g-dev libyaml-dev libreadline-dev libncurses5-dev libffi-dev libgdbm-dev libjemalloc2 \
	libvips imagemagick libmagickwand-dev mupdf mupdf-tools \
	redis-tools sqlite3 libsqlite3-0 libmysqlclient-dev \
	rbenv apache2-utils

zkan

MongoDB Shell Commands Cheat Sheet.

This is a Cheat Sheet for interacting with the Mongo Shell ( mongo on your command line). This is for MongoDB Community Edition.

Preface:

Mongo Manual can help you with getting started using the Shell.

FAQ for MongoDB Fundamentals and other FAQs can be found in the side-bar after visiting that link.

zkan

layout	title	description	tags
default	SQL Style Guide	A guide to writing clean, clear, and consistent SQL.	data process

Purpose

zkan

import duckdb
from google.cloud import bigquery

bqclient = bigquery.Client()
table = bigquery.TableReference.from_string(
    "bigquery-public-data.utility_us.country_code_iso"
)
rows = bqclient.list_rows(table)
country_code_iso = rows.to_arrow(create_bqstorage_client=True)
cursor = duckdb.connect()

zkan

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetObject"
            ],

zkan

FROM jupyter/pyspark-notebook
USER root
# Add essential packages
RUN apt-get update && apt-get install -y build-essential curl git gnupg2 nano apt-transport-https software-properties-common
# Set locale
RUN apt-get update && apt-get install -y locales \
    && echo "en_US.UTF-8 UTF-8" > /etc/locale.gen \
    && locale-gen
# Add config to Jupyter notebook
COPY jupyter/jupyter_notebook_config.py /home/jovyan/.jupyter/

zkan

admin > connections > +

conn_type = s3
conn_name = local_minio

Extra: a JSON object with the following properties: 
 {
    "aws_access_key_id":"your_minio_access_key",
    "aws_secret_access_key": "your_minio_secret_key",
    "host": "http://127.0.0.1:9000"

zkan

"""
### My Example DAG using Taskflow API

This document will show on the Tree View, Graph View, and Calendar View.
"""
from airflow.decorators import dag, task
from airflow.operators.dummy import DummyOperator
from airflow.utils import timezone

zkan

# Instructions
# 1. Write down Python code in say() method
#    - This method might have one parameter `number` which possibly 1-99
#    - This method need to return an English word for provided number e.g. If the number is 3, should return "three"
# 2. More examples on test data (numbers and words) are in the "Test Data Example" section
# 3. How to run this file through unit tests
#    - Open the terminal
#    - Run this command "python <path/to/file.py>" e.g. python PyConUS2021Challenge.py
#    - If the tests are failed, it will display "FAILED (failures=16)" for example, on the terminal
#    - If the tests are pass, it will display "Ran 16 tests in 0.001s OK"

zkan

from airflow.models import Variable

# Common (Not-so-nice way)
# 3 DB connections when the file is parsed
var1 = Variable.get("var1")
var2 = Variable.get("var2")
var3 = Variable.get("var3")

# Recommended Way
# Just 1 Database call