This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sudo dpkg-reconfigure systemd-resolved |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import json | |
import pandas as pd | |
class MultiColumnLabelEncoder: | |
def __init__(self): | |
self.mappings = {} | |
self.unknown_token = "unknown" | |
def fit(self, df: pd.DataFrame, columns: list[str]): |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from datetime import datetime, timedelta | |
def string2timestamp(str_dt: str, format: str = "%Y-%m-%dT%H:%M:%S", mill_decimals: bool = True) -> int: | |
""" | |
Convert a datetime string to an integer timestamp. | |
Args: | |
str_dt (str): The datetime string to convert. | |
format (str, optional): The format of the datetime string. Defaults to "%Y-%m-%dT%H:%M:%S". |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# get all pods from namespace | |
kubectl get pods -n spark-ds | |
# container port forwarding | |
kubectl port-forward service/airflow-ds 8095:8080 -n spark-ds | |
# stream logs from container | |
kubectl -n spark-ds logs -f om-extraction-task.93d8e47352dc4777b600bfa7b41f2f8a | |
# connect to container |