Skip to content

Instantly share code, notes, and snippets.

View orellabac's full-sized avatar

Orellabac orellabac

View GitHub Profile
@orellabac
orellabac / CUSTOMER_RAW.sql
Created June 30, 2024 02:45
SNOWPIPE WITH AUTOINGEST
CREATE STORAGE INTEGRATION s3_json_ingestion_int
TYPE = EXTERNAL_STAGE
STORAGE_PROVIDER = 'S3'
ENABLED = TRUE
STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::00000000000:role/s3-json-ingestion-role'
STORAGE_ALLOWED_LOCATIONS = ('s3://s3-json-ingestion/customers');
CREATE STAGE CUSTOMERS
URL = 's3://s3-json-ingestion/customers'
STORAGE_INTEGRATION = S3_JSON_INGESTION_INT
@orellabac
orellabac / customers1.json
Created June 30, 2024 00:06
Example of nested JSON
[
{
"customer_id": 3,
"name": "Alice Johnson",
"contact": {
"email": "alice@example.com",
"phone": "555-1234"
},
"address": {
"street": "123 Elm Street",
@orellabac
orellabac / initialize_snowpark_python.ipynb
Last active June 26, 2024 03:23
Sample Notebook that can be used to use Snowpark In a Databricks Notebook
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@orellabac
orellabac / chatbot.py
Created June 20, 2024 14:54
Simple Functional Chatbot with Streamlit and Cortex API in Snowflake.
import streamlit as st
import json
from snowflake.snowpark import Session
session = Session.builder.getOrCreate()
st.title("☃️ Frosty")
# Initialize the chat messages history
if "messages" not in st.session_state.keys():
st.session_state.messages = [{"role": "assistant", "content": "How can I help?"}]
@orellabac
orellabac / erwin_extractor.sql
Created June 19, 2024 17:40
Snowpark Example Extracting Tables Info from an ERWIN xml file:
COPY INTO @MYSTAGE/erwin_sample.xml
FROM (
SELECT $$<?xml version="1.0" standalone="yes"?>
<erwin
xmlns="http://www.erwin.com/dm"
xmlns:UDP="http://www.erwin.com/dm/metadata"
xmlns:EMX="http://www.erwin.com/dm/data"
xmlns:EM2="http://www.erwin.com/dm/EM2data" FileVersion="9.98.29174" Format="erwin">
<EMX:Model xmlns="http://www.erwin.com/dm/data" id="{700A0000-D99A-CCC5-999A-7CCCE2230F33}+00000001" name="Model_1">
<ModelEnvProps>
@orellabac
orellabac / session_with_alchemy.py
Created May 24, 2024 16:15
Using your snowpark session connection with sqlalchemy
from sqlalchemy import create_engine
import sqlalchemy
import logging
def get_engine_from_session(session):
# Your existing Snowflake connection (replace with your actual connection)
existing_snowflake_connection = session._conn._conn
setattr(existing_snowflake_connection,"_interpolate_empty_sequences",False)
# sql alchemy needs pyformat binding
existing_snowflake_connection._paramstyle = "pyformat"
@orellabac
orellabac / EXTRACT_XML.sql
Last active April 4, 2024 22:11
Function to extract xml fragments from an XML document. The input file can be compressed (ZIP format, GZIP format or TAR.GZ format are supported)
CREATE OR REPLACE FUNCTION EXTRACT_XML(FILE_PATH STRING, QUERY STRING, ITER BOOLEAN DEFAULT FALSE, OPTIONS OBJECT DEFAULT {})
RETURNS TABLE(DATA VARIANT)
LANGUAGE PYTHON
RUNTIME_VERSION='3.8'
PACKAGES = ('snowflake-snowpark-python','lxml')
HANDLER = 'XmlProcess'
AS
$$
import gzip
import tarfile
@orellabac
orellabac / ingest_table_from_dbx.sql
Created January 12, 2024 04:34
Loading a whole table from DBX to snowflake using jdbc
create or replace procedure ingest_table_from_dbx(
username string,password string, host string, qualified_table_name string, page_size int default 5000, max_rows int default -1)
returns string language python runtime_version = 3.11
packages =('snowflake-snowpark-python')
handler = 'main' execute as caller as $$
import snowflake.snowpark as snowpark
from snowflake.snowpark.functions import col, lit, object_construct
def main(session: snowpark.Session, username, password, host, qualified_table_name, page_size, max_rows):
# connection_settings
@orellabac
orellabac / export_vs_code_search_results_to_csv.py
Created January 10, 2024 17:06
Export VS Code Search Results as a CSV
# How to use:
# 1. In VS Code, perform a search.
# 2. Click "Open in editor" to open the search results in a `.code-search` file
# 3. The Save the file
# 4. In terminal, run `python export-vscode-search-to-csv.py search-results.code-search path/to/exported.csv`
import csv
import sys
import os
import re
@orellabac
orellabac / XPATHGET_TO_TABLE.sql
Created November 26, 2023 02:36
Snowpark Java UTDF to apply an XPATH to an XML and return the matches as rows
/*
XPATHGET_TO_TABLE Function
Description:
The XPATHGET_TO_TABLE function is designed to extract and return XML nodes as a table based on the provided XPath expression. It takes as input an XML string, an XPath expression, and an object specifying namespace prefixes.
Parameters:
xml (String): The XML string from which the function extracts nodes.
xpath (String): The XPath expression used to navigate and locate nodes in the XML document.