Skip to content

Instantly share code, notes, and snippets.

View timothymugayi's full-sized avatar
🎯
Focusing

Timothy Mugayi timothymugayi

🎯
Focusing
View GitHub Profile
@timothymugayi
timothymugayi / tqdm_threadpool.py
Created December 6, 2019 15:37
How to run tqdm in multiple threads
import time
from random import randrange
from multiprocessing.pool import ThreadPool
from tqdm import tqdm
def func_call(position, total):
text = 'progressbar #{position}'.format(position=position)
@timothymugayi
timothymugayi / langchain_openapi_schema_prompt_debug.log
Created June 10, 2023 02:16
langchain_openapi_schema_prompt_debug.log
Answer the following questions as best you can. You have access to the following tools:
spoonacular_API.searchRecipes: I'm an AI from spoonacular API. Instruct what you want, and I'll assist via an API with description: Search through hundreds of thousands of recipes using advanced filtering and ranking. NOTE: This method combines searching by query, by ingredients, and by nutrients into one endpoint.
spoonacular_API.searchRecipesByIngredients: I'm an AI from spoonacular API. Instruct what you want, and I'll assist via an API with description:
Ever wondered what recipes you can cook with the ingredients you have in your fridge or pantry? This endpoint lets you find recipes that either maximize the usage of ingredients you have at hand (pre shopping) or minimize the ingredients that you don't currently have (post shopping).
spoonacular_API.searchRecipesByNutrients: I'm an AI from spoonacular API. Instruct what you want, and I'll assist via an API with description: Find a set of recipes that adhere t
@timothymugayi
timothymugayi / shared_dict_update.py
Last active April 7, 2022 11:05
How to update python multiprocessing shared dict
import uuid
import sys
import multiprocessing
lock = multiprocessing.Lock()
if __name__ == '__main__':
print(sys.version);
# Manager().dict() = is a dict that spans multiple processes it exists in shared memory
import os
import pathlib
import time
import urllib.request
import zipfile
import dask.dataframe as dd
from dask.distributed import Client, LocalCluster
from memory_profiler import profile
from tqdm import tqdm
import os
import uuid
import pathlib
import zipfile
import pandas as pd
import urllib.request
from tqdm import tqdm
from memory_profiler import profile
import json
import os
import time
import boto3
# Partition keys are only required when you have more than 1 shard defined in the data stream but always required. Kinesis computes the MD5 hash of a partition key to decide what shard to store the record on (if you describe the stream you'll see the hash range as part of the shard description).
#
# Each shard can only accept 1,000 records and/or 1 MB per second. Hence If you attempt to write to a single shard faster than the default rate limit you'll get a ProvisionedThroughputExceededException Using With multiple shards, you can scale this limit: 8 shards gives you 8,000 records and/or 8 MB per second.
#
import calendar
import json
import os
import random
import time
import boto3
from datetime import datetime
import os
import sys
import magic
import boto3
import threading
from pathlib import Path
from boto3.s3.transfer import TransferConfig
# Subscribers are created with ZMQ.SUB socket types.
# A zmq subscriber can connect to many publishers.
import sys
import zmq
port = "5556"
if len(sys.argv) > 1:
port = sys.argv[1]
int(port)
# Publishers are created with ZMQ.PUB socket types
# Data is published along with a topic.The subscribers usually
# sets a filter on these topics for topic of their interests.
# A publisher has no connected subscribers, then it will simply drop all messages.
# If you’re using TCP, and a subscriber is slow, messages will queue up on the publisher.
# In the current versions of ØMQ, filtering happens at the subscriber side, not the publisher side.
import zmq
import random