sohang3112

Sync OneDrive in Linux

Using the official OneDrive Linux client:

$ sudo dnf install -y onedrive               # or "apt install", etc. - depends on distro
$ onedrive                                   # gives a link, open it in browser to login to OneDrive
$ onedrive --synchronize                     # sync files to ~/OneDrive ; downloading files first time takes some time
$ systemctl enable onedrive@<username>.service --now        # sync automatically in background

Check more on its usage in the docs.

sohang3112

Ollama Notes

Ollama is an open-source tool that allows self-hosting Large Language Models (LLMs). Normally these LLMs are not feasible to deploy on consumer hardware, however Ollama optimizes the models by removing unused layers, rounding decimal points in weights to reduce model size, etc.

Note: Some details about the Ollama service are Linux-specific, but most things are same on all platforms.

Downloading Ollama in Linux

In Linux, Ollama can be installed using the command curl -fsSL https://ollama.com/install.sh | sh.

sohang3112

import os
import openai

# Call GPT
os.environ['OPENAI_API_KEY'] = 'PUT-OPENAI-API-KEY-HERE'
client = openai.OpenAI()
prompt = "Say Hi!"        # put instructions to GPT here
gpt_response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": prompt}],

sohang3112

# Create a silent MP3 audio
# Requires ffmpeg library, install with: sudo apt install -y ffmpeg
from pydub import AudioSegment
silence = AudioSegment.silent(duration=5000)  # duration in milliseconds
silence.export("silence.mp3", format="mp3")

# Extract audio from video
import moviepy.editor
video = moviepy.editor.VideoFileClip("/path/to/video.mp4")
if video.audio is None:

sohang3112

# TODO: Optimize Dockerfile by combining multiple RUN commands into one, and using Multi-Stage Builds
FROM public.ecr.aws/lambda/python:3.11
RUN yum install -y freetype-devel harfbuzz-devel fribidi-devel meson gtk-doc
RUN yum install -y wget
RUN wget https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/l/libraqm-0.7.0-4.el7.x86_64.rpm && yum localinstall -y libraqm-0.7.0-4.el7.x86_64.rpm

RUN yum install -y cairo-devel pkg-config python3-devel
# install gcc required for building wheels of pycairo
RUN yum install -y gcc
RUN pip3 install --upgrade pip setuptools wheel

sohang3112

Creating AWS Lambda Layer for wordcloud Python library

wordcloud depends on matplotlib and numpy

Steps adapted from how to create AWS Lambda Layer for Python.

Note: Ran all the below steps in an EC2 Linux server.

• mk wordcloud_layer && cd wordcloud_layer
• Create requirements.txt:

wordcloud>=1.9.3

sohang3112

import os
os.environ["NLTK_DATA"] = "/opt/nltk_data"              # Optional - set custom path for nltk data (to download & use)

import os.path
import nltk
from nltk.corpus import stopwords

# identify stopwords (eg. for, the, a, etc. in English) - words that can be removed without changing the meaning of the data
nltk.download('stopwords')
print(stopwords.words('english'))

sohang3112

Doubling Rule (math chaos experiment)

In his talk Four Ways of Thinking, mathematician David Sumpter did a Chaos experiment with his audience. Here, I have plotted it in Python that shows the chaotic result plot.

Chaos definition: In maths, Chaos means a system where a small difference in a system's starting conditions can lead to huge changes in the final result.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

sohang3112

"""
Install requirements: pip install requests pandas parse
Set your Github Personal Access Token in environment variable GITHUB_PERSONAL_ACCESS_TOKEN before running this script.

Output Files:
- gh_issues.xlsx (MAIN OUTPUT) - issues' info in spreadsheet form
- gh_issues.json - original JSON response from Github Issues API
"""
from typing import Any, List, Dict
import os

sohang3112

import os
import traceback
import smptlib
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from email.utils import formatdate

# specify credentials in the below environment variables
EMAIL_SERVER = 'email-smtp.us-west-2.amazonaws.com'   # change this according to requirement
EMAIL_SERVER_PORT = 465