Skip to content

Instantly share code, notes, and snippets.

View xescuder's full-sized avatar
💻
Making software more reliable

xescuder

💻
Making software more reliable
View GitHub Profile
@xescuder
xescuder / load_all_exchanges_etl.py
Created June 13, 2024 16:45
Spark Load CSV into DB
import os
from urllib.request import urlopen
from pyspark.sql import SparkSession
from pyspark.sql.types import StructType, StructField, StringType
from config import AppConfig
import findspark
from pipeline_utils import rename_cols, snake_case
from trading_data_pipeline.load_daily_bars_etl import JDBC_PROPERTIES
"""
@xescuder
xescuder / pandas_utils.py
Last active June 12, 2024 21:05
Pandas Storing Only Changes to DB (SqlAlchemy 2.0, Pandas 2.0)
import sqlalchemy as db
import pandas as pd
def create_upsert_method(meta: db.MetaData):
"""
Create upsert method that satisfied the pandas's to_sql API. Adapted to 2.0
"""
def method(table, conn, keys, data_iter):
@xescuder
xescuder / StockChart.py
Created March 28, 2024 08:59
Trading View Chart to show in a Streamlit App
import json
from streamlit_lightweight_charts import renderLightweightCharts
import pandas_ta as ta
class StockChart:
@staticmethod
def show_chart(df, symbol):
COLOR_BULL = 'rgba(38,166,154,0.9)' # #26a69a
@xescuder
xescuder / ib_insync_streamlit.py
Last active March 15, 2024 20:32
Example of streamlit using ib_insync and streamlit
import logging
import pandas as pd
from dateutil.relativedelta import relativedelta
from dateutil.utils import today
from app.charts import render_candlestick_chart
import streamlit as st
import asyncio
import plotly.graph_objects as go
@xescuder
xescuder / base_strategy.py
Last active March 7, 2024 17:54
Backtrader Base Strategy (adapted from Esteban Thiliez for multiinstruments and some simplifications)
from abc import abstractmethod
import backtrader as bt
class BaseStrategy(bt.Strategy):
params = dict(
verbose=True,
longs_enabled=True,
shorts_enabled=True
)
@xescuder
xescuder / golden_cross_stop_trailer.py
Created March 3, 2024 11:04
A backtrader multidata example with golden cross strategy and stop trailer (based on strategy entries and exits)
import backtrader as bt
from trading_backtesting.stop_trailer import StopTrailer
class GoldenCrossStrategy(bt.Strategy):
SHORT, NONE, LONG = -1, 0, 1
params = dict(
fast_length=50,
@xescuder
xescuder / top_instruments.ipynb
Last active February 15, 2024 17:43
Top Instruments - DuckDB and Polars
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@xescuder
xescuder / return_changes_1year.sql
Created January 21, 2024 13:13
Return changes between today and 1 year ago
SELECT s.symbol, s.sector, first(close, date) AS first_close, last(close, date) AS last_close,
MIN(date) AS first_date,
MAX(date) AS last_date,
100 * (last(close, date) - first(close, date)) / first(close, date) AS return_change
FROM
daily_bars db, stocks s
WHERE
db.date >= now() - interval '1 year' AND db.symbol = s.symbol
GROUP BY
s.symbol
@xescuder
xescuder / load_sectors_etfs.py
Created January 16, 2024 17:45
Load CSV into Table
import os
from pathlib import Path
from sqlalchemy import create_engine
import pandas as pd
from trading_data_pipeline.config import AppConfig
config = AppConfig(os.environ)
connection_url = config.get_postgres_uri()
@xescuder
xescuder / docker-compose-airflow.yaml
Created December 31, 2023 13:32
Docker Compose Airflow
version: "3"
x-airflow-common: &airflow-common
# In order to add custom dependencies or upgrade provider packages you can use your extended image.
# Comment the image line, place your Dockerfile in the directory where you placed the docker-compose.yaml
# and uncomment the "build" line below, Then run `docker-compose build` to build the images.
image: trading-airflow #${AIRFLOW_IMAGE_NAME:-apache/airflow}
# build: .
environment: &airflow-common-env
AIRFLOW__CORE__EXECUTOR: LocalExecutor
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://airflow:airflow@postgres/airflow