Skip to content

Instantly share code, notes, and snippets.

View orellabac's full-sized avatar

Orellabac orellabac

View GitHub Profile
@orellabac
orellabac / initialize_snowpark_python.ipynb
Last active June 26, 2024 03:23
Sample Notebook that can be used to use Snowpark In a Databricks Notebook
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@orellabac
orellabac / chatbot.py
Created June 20, 2024 14:54
Simple Functional Chatbot with Streamlit and Cortex API in Snowflake.
import streamlit as st
import json
from snowflake.snowpark import Session
session = Session.builder.getOrCreate()
st.title("☃️ Frosty")
# Initialize the chat messages history
if "messages" not in st.session_state.keys():
st.session_state.messages = [{"role": "assistant", "content": "How can I help?"}]
@orellabac
orellabac / erwin_extractor.sql
Created June 19, 2024 17:40
Snowpark Example Extracting Tables Info from an ERWIN xml file:
COPY INTO @MYSTAGE/erwin_sample.xml
FROM (
SELECT $$<?xml version="1.0" standalone="yes"?>
<erwin
xmlns="http://www.erwin.com/dm"
xmlns:UDP="http://www.erwin.com/dm/metadata"
xmlns:EMX="http://www.erwin.com/dm/data"
xmlns:EM2="http://www.erwin.com/dm/EM2data" FileVersion="9.98.29174" Format="erwin">
<EMX:Model xmlns="http://www.erwin.com/dm/data" id="{700A0000-D99A-CCC5-999A-7CCCE2230F33}+00000001" name="Model_1">
<ModelEnvProps>
@orellabac
orellabac / session_with_alchemy.py
Created May 24, 2024 16:15
Using your snowpark session connection with sqlalchemy
from sqlalchemy import create_engine
import sqlalchemy
import logging
def get_engine_from_session(session):
# Your existing Snowflake connection (replace with your actual connection)
existing_snowflake_connection = session._conn._conn
setattr(existing_snowflake_connection,"_interpolate_empty_sequences",False)
# sql alchemy needs pyformat binding
existing_snowflake_connection._paramstyle = "pyformat"
@orellabac
orellabac / EXTRACT_XML.sql
Last active April 4, 2024 22:11
Function to extract xml fragments from an XML document. The input file can be compressed (ZIP format, GZIP format or TAR.GZ format are supported)
CREATE OR REPLACE FUNCTION EXTRACT_XML(FILE_PATH STRING, QUERY STRING, ITER BOOLEAN DEFAULT FALSE, OPTIONS OBJECT DEFAULT {})
RETURNS TABLE(DATA VARIANT)
LANGUAGE PYTHON
RUNTIME_VERSION='3.8'
PACKAGES = ('snowflake-snowpark-python','lxml')
HANDLER = 'XmlProcess'
AS
$$
import gzip
import tarfile
@orellabac
orellabac / ingest_table_from_dbx.sql
Created January 12, 2024 04:34
Loading a whole table from DBX to snowflake using jdbc
create or replace procedure ingest_table_from_dbx(
username string,password string, host string, qualified_table_name string, page_size int default 5000, max_rows int default -1)
returns string language python runtime_version = 3.11
packages =('snowflake-snowpark-python')
handler = 'main' execute as caller as $$
import snowflake.snowpark as snowpark
from snowflake.snowpark.functions import col, lit, object_construct
def main(session: snowpark.Session, username, password, host, qualified_table_name, page_size, max_rows):
# connection_settings
@orellabac
orellabac / export_vs_code_search_results_to_csv.py
Created January 10, 2024 17:06
Export VS Code Search Results as a CSV
# How to use:
# 1. In VS Code, perform a search.
# 2. Click "Open in editor" to open the search results in a `.code-search` file
# 3. The Save the file
# 4. In terminal, run `python export-vscode-search-to-csv.py search-results.code-search path/to/exported.csv`
import csv
import sys
import os
import re
@orellabac
orellabac / XPATHGET_TO_TABLE.sql
Created November 26, 2023 02:36
Snowpark Java UTDF to apply an XPATH to an XML and return the matches as rows
/*
XPATHGET_TO_TABLE Function
Description:
The XPATHGET_TO_TABLE function is designed to extract and return XML nodes as a table based on the provided XPath expression. It takes as input an XML string, an XPath expression, and an object specifying namespace prefixes.
Parameters:
xml (String): The XML string from which the function extracts nodes.
xpath (String): The XPath expression used to navigate and locate nodes in the XML document.
@orellabac
orellabac / XPATHGET.sql
Created November 26, 2023 02:27
Snowpark Java UDF to get a value from an XML using XPATH
/*
XPATHGET Function
==============
Description:
The XPATHGET function is designed to extract values from XML documents using XPath expressions. It takes as input an XML string, an XPath expression, an optional instance parameter, and an optional prefixes parameter.
Parameters:
============
xml (String): The XML string from which the function extracts values.
xpath (String): The XPath expression used to navigate and locate elements in the XML document.
@orellabac
orellabac / deploy_spss_reader_metadata.java
Created November 23, 2023 13:31
Snowpark Java SPSS\.sav files metadata extractor
/*
Go to https://mvnrepository.com/artifact/com.bedatadriven.spss/spss-reader
and download the jar file.
And upload it into an stage
*/
create or replace function read_spss_metadata(filename string)
RETURNS TABLE(COLUMN_NAME TEXT,TYPE TEXT,NULLABLE BOOLEAN,EXPRESSION TEXT,FILENAMES TEXT,ORDER_ID INT)
language java
imports = ('@mystage/spss-reader-1.3.jar')