Skip to content

Instantly share code, notes, and snippets.

@leoricklin

leoricklin/res_snowflake.md

Last active October 8, 2022 01:57
Show Gist options
  • Save leoricklin/6572c51ae5ddd31d613f1b64aa8e0785 to your computer and use it in GitHub Desktop.
Save leoricklin/6572c51ae5ddd31d613f1b64aa8e0785 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
res_snowflake.md

Snowflake DW

Snowflake Doc

Teradata to Snowflake Migration Guide

Using Snowflake

Databases, Tables, View

Understanding Snowflake Table Structures

Micro-partitions & Data Clustering

Clustering Keys & Clustered Tables

Working with Temporary and Transient Tables

Temporary tables only exist within the session in which they were created and persist only for the remainder of the session. As such, they are not visible to other users or sessions. Once the session ends, data stored in the table is purged completely from the system and, therefore, is not recoverable, either by the user who created the table or Snowflake.

Snowflake supports creating transient tables that persist until explicitly dropped and are available to all users with the appropriate privileges. Transient tables are similar to permanent tables with the key difference that they do not have a Fail-safe period. As a result, transient tables are specifically designed for transitory data that needs to be maintained beyond each session (in contrast to temporary tables), but does not need the same level of data protection and recovery provided by permanent tables.

Overview of Views
Working with Secure Views
Working with Materialized Views
Table Design Considerations
Cloning Considerations
Data Storage Considerations

Developing Applications with Snowflake

Introduction to Developing Applications in Snowflake — Introduction to developing applications in Snowflake.

Overview of Connectors, Drivers, and Client APIs — Information about the connectors, drivers, and client APIs that you use to develop client applications that connect to Snowflake.

UDFs (User-Defined Functions)

Snowpark

External Functions

Stored Procedures

Snowflake Scripting

Overview of the Ecosystem

Loading Data into Snowflake

Overview of Data Loading
Summary of Data Loading Features
Data Loading Considerations
Using File Formats
Examples

Preparing to Load Data

Querying Metadata for Staged Files
Transforming Data During a Load

Unloading Data from Snowflake

Overview of Data Unloading
Unloading into a Snowflake Stage

Using snowflake

** Managing Security in Snowflake

** General Reference

*** Transactions

https://docs.snowflake.net/manuals/sql-reference/transactions.html

  • Scope of a Snowflake Transaction
  • Isolation Level
  • Resource Locking
  • Transaction Commands and Functions
  • Aborting Transactions

*** Information Schema

https://docs.snowflake.net/manuals/sql-reference/info-schema.html

SQL Command Reference

https://docs.snowflake.net/manuals/sql-reference/sql-all.html

[
WITH
      <cte_name1> [ ( <cte_column_list> ) ] AS ( SELECT ...  )
  [ , <cte_name2> [ ( <cte_column_list> ) ] AS ( SELECT ...  ) ]
  [ , <cte_nameN> [ ( <cte_column_list> ) ] AS ( SELECT ...  ) ]
]
SELECT ...

SQL Function

Good example will be: "average_depth" : 21.8698, "partition_depth_histogram" : { "00000" : 0, "00001" : 13849, // there are 13849 partitions with 1 keys inside this partition ...

Stored Procedures

When you query a view like the one you defined above, predicates get pushed down to the individual tables, and in addition only necessary columns (ones referenced by the query) are retrieved. So the view is never fully materialized.

Tutorials

Getting Started With Snowpark
Raw
res_snowflake_resources.md

1.Resources

20210823 A Spark Developer's Guide to Snowpark | phData

FAQ: DOES SNOWFLAKE SUPPORT DATA VAULT 2.0, https://community.snowflake.com/s/article/faq-does-snowflake-support-data-vault-20
FAQ: HOW TO INGEST & ENRICH IOT DATA AT SCALE INTO SNOWFLAKE WITH APACHE NIFI, https://community.snowflake.com/s/article/How-to-Ingest-Enrich-IoT-Data-at-Scale-into-Snowflake-with-Apache-NiFi

Snowflake & Spark

https://www.snowflake.com/blog/snowflake-and-spark-part-1-why-spark/
https://www.snowflake.com/blog/snowflake-spark-part-2-pushing-query-processing/
https://databricks.com/blog/2018/08/27/by-customer-demand-databricks-and-snowflake-integration.html

Snowflake and Sagemaker

https://www.snowflake.com/blog/quickstart-guide-for-sagemaker-snowflake-part-one/
https://www.snowflake.com/blog/building-a-sagemaker-instance-from-scratch/
https://www.snowflake.com/blog/connecting-a-jupyter-notebook-to-snowflake-through-python-part-3/
https://www.snowflake.com/blog/connecting-a-jupyter-notebook-to-snowflake-via-spark-part-4/

2.Tutorial

GETTING STARTED WITH SNOWFLAKE

Follow along with our tutorials to get you up and running with the Snowflake Data Cloud.

Accelerating Data Teams with Snowflake and dbt Cloud Hands On Lab
Accelerating Data Teams with dbt Core & Snowflake

Data Engineering with Apache Airflow, Snowflake & dbt

Getting Started with Snowpark Python

Getting started video

Snowflake University

  • onboarding

    • Snowflake Overview
    • Snowflake User Interface Walkthrough
    • OK, Snowflake Security Overview
    • OK, Snowflake Data Loading Best Practices
    • OK, Snowflake Administration
    • OK, Support Services

  • adv

    • OK, Snowflake Architecture-Cloud Service
    • OK, Clients and Ecosystem
    • Account Considerations
    • OK, Resource Monitor
    • OK, Backup & Recovery

  • transform

    • Data Sharing in Snowflake
    • SQL Support and Usage in Snowflake
    • OK, Data Warehouse Migration Planning
    • Cloning Data in Snowflake

  • Cloud Data Warehouse Benchmark Redshift vs Snowflake vs BigQuery | DataEngConf SF '18

  • Cloud Analytics Academy Featured Courses for snowflake (https://www.cloudanalyticsacademy.com/)

    • Featured Courses
      • Cloud Data Warehousing for Beginners
      • Analytics in the Cloud--Rising
      • Understanding cloud infrastructure
      • Unifying Your Data and Teams to Solve the Analytics Chaos
    • Cloud Foundation Track
      • Bringing Your Data Together in the Cloud
      • In the Cloud: ELT vs. ETL
      • The Correct Way to Build a Data Lake
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment